Skip to end of metadata
Go to start of metadata
This is the home page for the Data Management (DM) subsystem Integrated Product Team. This subsystem is constructed in Releases 1 through 3 of OOI CI construction.

Welcome to the Data Management Subsystem Page

The information distribution services of the Data Management subsystem provide information distribution, persistence and access across space and time, making data accessible across the observatory network over extended periods of time. Information includes observational data and derived data products along with other information resources required for the operation of the integrated observatory, such as ancillary instrument information, user identities, workflow definitions, and executable virtual machine images. All information products are managed together with their metadata. Data collection and processing activities are based on the distributed exchange framework, supporting flexible access to data at all steps of the activities.

Team

Name Org Status
Role
Campbell, Luke RPS ASA active Senior Software Engineer

Subsystem Objectives

Release 1 Objectives

  • Provide data distribution services leveraging publish-subscribe communication
  • Define the common OOI data and metadata model and provide common object model tool chain
  • Provide basic Persistent Archive Services

Release 2 Objectives

(see also R2 DM Construction Plan)

  • Supporting advanced but essential components of resource and data life cycle: versioning, association and provenance; archiving; semantic tagging and vocabulary management; data processing services - in a way that is tuned to large scale data processing, large catalogs, using the messaging infrastructure
  • Support data processing workflows and life cycle elements in a way that meets science requirements
    • Support SA to provide data product generation workflows
    • Support AS to provide visualization product generation workflows
    • Prepare for AS to provide a generic workflow execution architecture in Release 3
    • Work with CEI on process execution for real-time data transformations
  • Refine and extend the OOI Common data and metadata model, in particular geared towards resource life cycle, more feature types, provenance, versioning and semantic capabilities
    • Define feature type integration strategy for new/additional feature types
    • Provide support for most relevant feature types to convert in and out of the common data model
  • Expand the tools provided for search-by-query of OOI data resources, introducing more sophisticated metadata search capabilities, filtering and organizing search results, saved query and prospective query capabilities
  • Enable interoperability with external communities and applications by providing integration capabilities and transformation processes.
  • Provide flexible, robust manageable, multi-domain of authority support for persistent data stores, leveraging existing storage infrastructures and archives

Release 3 Objectives

See the R3 DM Construction Plan.

Documentation

Requirements

[DM R1 Requirements]
[DM R2 Requirements]

[DM R1 User Scenarios]
DM R2 User Scenarios

Architecture and Design

The System Engineering section of this wiki contains the architectural materials for the DM subsystem.

Development and Implementation Progress

Release 1

Release 2

The finalized iteration task plan (13 May) for Data Management and Distribution is here. The evolving list of tasks is on this GoogleDocs spreadsheet.

DM R2 Designs
R2 Status Report
References and Reading

Release 3

The evolving list of tasks is on this Google spreadsheet.

Prototypes

Labels

r2-subsystem-objectives r2-subsystem-objectives Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.