LDOS - Large Data Object Storage  

The Large Data Object Storage project was a proof of concept for the DoD that modern cloud technologies could change the methodologies and price points for the capture and analysis of large amounts of reconnaissance imagery and make those data available without latency to the user community. The project was highly successful and led to many of the current cloud, and edge-of-the-cloud solutions.

Concepts that were proven possible and feasible in the project:

• Petabyte Ingest

• Object Storage

• Name Space technology

• Connectivity to Commercial Clouds (AWS & Microsoft Azure)

• Imagery Search via cloud search algorithms

• Application of analytics to data

• Distribution

• Exposure to the internet from any location

• Sync with DoD networks

• Meta-tagging at extremely high rates
  
  

The core of the offering was an earlier YottaDrive storage device. Each YottaDrive operating unit providing up to 10PB of Large Data Object Storage. 

The YottaDrive offering delivered three global namespace services: Ingest, Search and Dissemination. These services were optimized for different data types stored in unique global namespaces. For this IOC Assessment, the Ingest, Search and Dissemination were optimized for WAMI and FMV and will support other ISR data types such as SAR, EO/IR and other geospatial data types after FOC. 

In addition, the underlying global namespace was a naturally federated, distributed namespace which can span multiple locations providing a resilient and durable object store. For this IOC Assessment, the global namespace was planned for deployment in a single CONUS location.

Figure 1 - High-Level Overview of the YottaDrive for ISR

LDOS Ingest Service

The Ingest Service allows customers to ingest their specific data type into the global namespace. This Ingest Service was optimized for the target data types and continually updated throughout the period of service to incorporate new and modified industry standards. As part of the Ingest process the object metadata was extracted and provided to the Search process through industry standard XML services. This ensures that as the Ingest velocity increases the Search index is current with all objects stored in the global namespace. Because of the YottaDrive implementation behind the enterprise firewall, the underlying global namespace is specific to each customer and only the customer’s data types are ingested into the system. This provides another level of security for classified data.

Figure 2 - Graphical representation of the YottaDrive Ingest Service. 

LDOS Search Service 

The Search Service allows authorized uses to execute search queries against the global namespace. The YottaDrive Search Service is provided by the Google Search Appliance (GSA). The GSA provides an intuitive search application, with a familiar interface that is recognizable to many analysts.  For each specific data type the search user interface is optimized to support the users’ mission requirements. For the geospatial data types, our Commercial Search Service provides a powerful geospatial search capability that allows analysts to enter both text as well as geospatial search arguments. In addition, through a geospatial visualization paradigm, analysts can explore the global namespace dramatically, thereby enhancing capability for analysts to quickly find and access relevant geospatial data types to support mission requirements.

Figure 3 - Graphical representation of the YottaDrive Search Service.

LDOS Dissemination Service 

The Dissemination Service allows authorized users to extract data from the global namespace and bring it back to their operational environments for further processing and exploitation. The Dissemination Service supports multiple ways of accessing objects in the global namespace. The two (2) Dissemination Services provided for this IOC Assessment are as an integrated capability in the Search Service and as an industry standard WebDAV interface.

Figure 4 - Graphical representation of the YottaDrive Dissemination Service.