|This page explains the scope of the Release 3 Integrated Observatory Network, the On-Demand Measurement Processing release.
Note: Scope of Release 3 increments the scope of Release 2.
This release adds end-to-end control data collection methods, and emphasizes the end-to-end control of processing, with on-demand and event-driven measurements — including defining observation plans, marine resource coordination, and autonomous marine asset control — and advanced workflows for OOI users, instrument providers, all data product producers, and data product consumers. It also extends governance and federation features, enabling sophisticated interactions across multiple domains and user communities.
Primary Themes and Users: This release advances the ability to control OOI observatory infrastructure and core instrumentation, and adds end-to-end control of data processing methods. This release also allows participants beyond the developer community to generate real-time workflows, producing quality data products to the user community on behalf of OOI or themselves. This release provides sophisticated capabilities for Observatory operators and managers, and emphasizes the needs of OOI and end user community actors: instrument providers and operators, scientific data users, data product generation algorithm developers, data analysts, and providers of resources like storage and computer cycles.
Secondary Themes and Users: Expanded data modeling, workflow, and integration tools for observational, modeling, and analytic data consumers. Deploying advanced features including policy application, event response and autonomous behaviors, automated failure response, and Integrated Observatory interfaces with handheld devices.
Feature Categories: The release includes features in the following major categories:
- Interactively Annotate Data
- Register End User Resources (data, software, models, instruments)
- Automate Observatory Science Missions
- Manage OOI Network
- Integrate External Software
- Complete Data Integration
- Operate Virtual Workspaces
- Mobile Observatory Services
Release 3 delivers capabilities from the Common Operating Infrastructure (COI), Common Execution Infrastructure (CEI), Data Management (DM), Sensing & Acqusition (S&A), Analysis and Synthesis (AS), Planning and Prosecution (PP), External Observatory Integration (EOI) and User Experience (UX) subsystems.
For each of these subsystems, the extent of capabilities is explicitly stated as deliverables in the Release 3 Construction Plan (see also CI Construction Plans, and Construction Plan Overview). The user-visible and infrastructure capabilities are defined as use cases in the Product Description (see below). Both Construction Plan and Use Cases cross-reference this architecture specification.
In addition to new capabilities in Release 3, the system will maintain the capabilities of Release 2. Some internal restructuring and rework of existing software components will be necessary to adapt the system to its expanded scope. The User Interfaces in Release 3 will encompass the Release 2 functions, but may present them in a different integrated user experience.
OOI CI places a high priority on a transformative Integrated Observatory Network architecture that is scalable, secure and extensible. The architectural foundations together with accompanying infrastructure capabilities are of paramount importance to achieve the goal of maintaining and extending this system for more than 25 years. Substantial effort is spent on building out this architecture and its infrastructure components, in particular in the early releases. User visible capabilities make up only a fraction of the actual system capabilities designed and implemented. The actual implementation of user capabilities is often delivered limited at first and is refined and deepened in subsequent releases based on user feedback and within the architecture.
Some of these architecture foundations and core infrastructure capabilities in this release include:
- Multi-Domain of Authority Governance: The Integrated Observatory Network will support the definition of resource policy and user membership for more than one domain of authority (aka facility). This is applied at first to separate resources and their policy for each of the Marine Observatories from the resources and policy of the Integrated Observatory Network. The system will federate all facilities in the system and their resource catalogs and policies. Policy is enforced across the federated system.
- Managed Interactions: All communication within the system follows described patterns of interaction. Any violation of these patterns can be detected; this helps to increase system stability and reliability. System components development can be simplified in reaction to defined interactions.
Release 3 will strengthen the following Release 1 and 2 foundations and core infrastructure capabilities:
- Service-Orientation: All system capabilities are expressed as services with defined interfaces. Services can be deployed where needed and can be scaled to demand during operation of the system. Service implementations can be replaced and technologies changed without disrupting the system functionality as the system evolves.
- Message-Based: All internal system interactions occur via the exchange of defined messages using a secure, reliable asynchronous messaging infrastructure. Messaging is a highly successful integration strategy for large-scale systems of systems. Messaging helps to reduce the dependencies between the system elements and increases the robustness of the system.
- Data-Oriented: All information in the system, whether communicated via messages or stored in the system's registries is present in objects of described type and composition. This emphasis of data over behavior enables the use of multiple technologies to manipulate these objects, over extensive periods of time.
- Resource-Oriented: All capabilities of the system, from physical (instruments, storage, servers) to computational (virtual machines, services) and virtual (data sets, information objects) are expressed in common form as resources. Resources are registered in resource registries and can be described, annotated, cross-referenced, advertised, published and used in consistent ways across the system.
- Multi-Technology: More than one implementation technology is used to implement the capabilities of the system. Release 3 furthers integration of the most suitable third party packages into the system.
- Elastic Virtualized Deployment: All compute resources are provided in virtualized host environments, whether in OOI operated CyberPoPs or in cloud execution sites. The system manages these resources and can flexibly provide capabilities, such as system services or user processes to match demand. No fixed deployment schemes or hardware incompatibilities limit the extent.
- Multi Site Deployment: The Integrated Observatory Network will operate across multiple distributed deployment sites. Release 3 will span four CyberPoPs and an execution point in the Amazon commercial cloud. Each site has its own compute, storage, network and software infrastructure, all brought into the Integrated Observatory system to the advantage of its end user, such as to keep data close to consumers. The CyberPoPs will host essential system capabilities, while the cloud execution points are used for extended user computations and for test deployments.
- Resource Life Cycle: All resources managed by the Integrated Observatory system follow a defined resource life cycle, which will be more explicitly defined and expanded for some resources in Release 3. The life cycle prescribes mandatory consistency requirements for resource descriptions and state. The life cycle also determines how users can interact with the resources and the policy that applies in any given state. Applying a defined life cycle to all system resources makes managing a system of this scale easier and substantially more scalable.
- Common Science Data Model: All science data is transformed into one encompassing common data model when ingested into the system. From this common data model, any external representation can be generated as needed. Powerful standard operations can be applied to all science data in the system, independent of their original representation. Operations include subsetting, coordinate system transformation, interpolation and aggregation, selective access and more.
The Release 3 Product Description will provide a top-level description of Release 3 extent. The Product Description lists a number of scenarios for the capabilities (functions) that are relevant for the target user audience in Release 3. In addition, it contains use cases for system internal infrastructure capabilities, required to enable the user-visible capabilities. The Product Description use cases do not prescribe the way the capabilities are implemented; this is covered in this architecture documentation.
The Product Description is an agreed-upon and normative definition of Release 3 scope. Project stakeholders, management, user representatives, user experience team, designers and implementers agree on the specifics of this scope during and after the LCO review. The Product Description is subject to (moderate) modification with required approval, for instance to provide clarification or adjust scoping. Lost scope may be added to subsequent releases, so long as overall architectural scope is not reduced through this process.
The scope and emphasis per subsystem for Release 3 is further defined in the overview pages for each subsystem and in the Transition to Operations page.
Release 3 is the first release of the Integrated Observatory intended for exposure to the general public. Release 1 was open to a very limited early adopter audience, and Release 2 also limited its target audience to OOI participants and early adopters. Release 3 will therefore incorporate many features to support, and as appropriate constrain, the activities of the public users of the OOI.
Release 3 will be expanding Release 2 capabilities as explained above. All Release 2 capabilities will be carried forward. Capability implementations, integrated technologies as well as user interfaces may change.
Release 3 will replace Release 2 in place, though some explicit transformations of system resources may be required when the replacement occurs. While backwards compatibility will be maintained for this release in most aspects. Of course, an an as-needed based, some changes or rework may be required.
In addition to the extended set of capabilities, Release 3 will continue to extend its set of science-relevant data resources. Release 3 will add access to external data sets and catalogs from the Integrated Ocean Observing System, NEPTUNE Canada, and the World Meteorologic Organization. In addition to that, Release 3 will provide key OOI data sets through these organizations._