Skip to end of metadata
Go to start of metadata
This page describes the OOI science coverage model.

Overview

The OOI coverage data model is part of the OOI Common Data and Metadata Model. It was designed as an OOI internal representation of science data and other information in any shape and feature type. It was strongly inspired by standards such as GML, SWE and ISO-19123. It's implementation was inspired by projects such as Fenics and MOAB iMesh. The OOI coverage data model can hold data of any dimensionality and structure, store data efficiently and for direct access, and supports distributed appendage and retrieval. The process of Ingestion takes a data stream of Granules (another part of the OOI Common Data and Metadata Model) and inserts them into an OOI coverage.

Definition

Coverages are defined by a coordinate reference system, a domain and a range. The coordinate reference system is the frame to which the objects in the domain are referenced. The domain provides the features and the range specifies the value of those features.

Key Features
Science Data Model

  • The key is the separation of information concerns into:
    • Observational context
    • Scientific feature types characterization
    • Structural (topological and geometric) specification of the spatiotemporal data.
    • Content specification and management of the data records
  • Separation provides an effective formulation of generalized operators for:
    • The metadata mapping
      • Ontological crosswalks and inferencing, Vocabulary mapping
    • Textual and geospatial search and navigation
      • List, Find, Select(overlaps | within | contains | disjoint)
    • Topological and Geometric manipulation
      • Index slicing, Geometric sub-setting, Geometric evaluation, Structure to structure projection
    • Mathematical manipulation of the scalar and vector Fields over the spatiotemporal Structure
      • Linear Algebra thru to Tensor Calculus
    • Representation and format conversion

Temporal dimension

Each data point that is added to an information set augments the set wrt to the temporal dimension. There is no explicit relationship between the information set updates along the temporal dimension

The figure below provides an exemplar for how the Science Data Coverage is applied within ION for a data stream (to persist the granules of the stream in granule order) and for a dataset (to persist the parameters inside the stream granules within the defined structure of the dataset).

Figure 1. Coverage applied to both Data Stream and Dataset (OV-1)

Internal Structure

Coverage Domain Model

Refined details of the Fenics API level:

Error formatting macro: plantuml: java.lang.IllegalStateException

skinparam object {
BackgroundColor LightGray
ArrowColor Black
BorderColor DarkSlateGray
FontSize 15
}
skinparam package {
Style rect
BorderColor Black
}

object "Coverage" as cov {
----ISO 19123----
Coverage[] list()
Entity find()
Entity[] select()
Value[] eval()
Value[] inv_eval()
void insert()
}

object "Domain" as dom{
----ISO 19123----
}
object "Range" as rng{
----ISO 19123----
}
object "Coordinate Reference System" as crs{
----ISO 19111----
}
object "Topology" as topo
object "Geometry" as geom
object "Parameter" as param
object "<<FEniCS SubMesh>>\nMesh Entity Set" as mes
object "Mesh Entity" as me
object "<<FEniCS Mesh>>\nMesh" as mesh
object "<<FEniCS Mesh>>\nMesh Function" as mfunc

object "Value Sequence" as vset

object "Dimension" as dim
object dim_0
object dim_1
object dim_n

object "Shape" as shp

cov o-- "*" cov
cov o-- "*" rng
cov o-- "*" crs
cov o-- "*" dom
crs .left. "*" dom
dom o-- topo
dom o-- "*" geom
geom left|> param
geom .left. topo: over as coord
param .. topo: over
mesh .. "*" mes: related
mesh <|-- mes
mesh -- "" me

mesh -- "" dim
dim <|-- dim_0
dim <|-- dim_1
dim <|-- dim_n

mes o-right- "*" me
me o-- dim
mfunc "*" o-left- mesh
mfunc *-- "1" vset
rng o-- "*" param
param <|-- mfunc
topo o-- "*" shp
topo <|-- mesh

Figure 2. Coverage Model Specification (OV-7)

Note: explore relationship between mesh and mesh entity set (submesh) - what is the relationship? What should it be?

Coverage Context

The Coverage Context is defines the mapping of Value Sets to the domain and range and the meaning of each range.

Note: Chapter 6 of the GML Application Schema describes Domain Set and Range Type which should map to our Coverage Context

Domain Set

The Domain Set is defines the topology (and possibly geometry) referenced by a stream granule

Note: Any part of the domain set describing the topology and geometry may be dynamic such that it is all transmitted in every granule. Only information that is invariant or variant over epochs is passed by reference in the domain set.

Error formatting macro: plantuml: java.lang.IllegalStateException

skinparam object {
BackgroundColor LightGray
ArrowColor Black
BorderColor DarkSlateGray
FontSize 15
}
skinparam package {
Style rect
BorderColor Black
}

object "Domain Set" as ds
object "Geometry" as geom
object "Shape" as shp
object "Mesh" as mesh

ds -- "" geom
ds *-- mesh
ds -- "1:" shp

Figure 3. Domain Set Specification (OV-7)

Note: What are the Geometry, Shape, Mesh: When their content is variant? When their content is invariant?

Taxonomy

The Taxonomy is a mapping of Range Type to an name in the Record Dictionary (GML Range Set) where the name is unique to a Coverage.

Note: This object does not exist in GML. GML Range Type appears to be a tuple which must map by position to the records in the Range Set: More investigation needed!

Error formatting macro: plantuml: java.lang.IllegalStateException

skinparam object {
BackgroundColor LightGray
ArrowColor Black
BorderColor DarkSlateGray
FontSize 15
}
skinparam package {
Style rect
BorderColor Black
}

object "<dict>\n Taxonomy" as tx
object "Map" as m
object "Name Set" as ns
object "Handle (Ordinal)" as hnd
object "Name" as n

object "NickName" as nn
object "RangeType" as rt

tx -- "" m
m *-- ns
m *-- hnd
ns -- "" n
n <|-- nn
n <|-- rt

Figure 4. Taxonomy Specification (OV-7)

There is open discussion on the use of the term ordinal. The usage of taxonomy is to provide a flat namespace for a tree of value sequences in a hierarchical record dictionary which would make it more properly name:path. The term ordinal derives from the use in Fudge where it is an optimization used to demarcate fields where the names are sent separately as a taxonomy so that the long string name need not be sent in every message.

Range Type (Formerly called Parameter Description Resource)

A dictionary which stores GML descriptions (or equivalent) at user defined levels of detail.

From 09-146r1_OGC_GML_Application_Schema_-__Coverages.pdf:

The rangeType component adds a structure description and technical metadata required for
an appropriate (however, application independent) understanding of a coverage. For this
structure description, the SWE Common DataRecord is used.

GML Range Type Example

Further points of interest:

6.5 RangeSet coherence
Both domainSet and rangeType describe the coverage values given in the rangeSet.
Hence, consistency must be enforced between them. The pertaining requirements are listed in
this Subclause.

Requirement 5 /req/gml-coverage/one-range-value-per-position:
For each coordinate position contained in the domain set description of a coverage there shall
exist exactly one range value in the coverage’s range set.
NOTE Both duplicates and values omitted are not allowed. For range values not known for some reason nil values can be used.
Requirement 6 /req/gml-coverage/range-structure-consistency:
All range values contained in the range set of a coverage shall be consistent with the
structure description provided in its range type.

In GML the range set there is a one to one mapping between the values described and the values present - by order.

In our system we use the Taxonomy to identify the structures that are present from those that are defined and may be present in a granule.
We want to relax the requirements listed above!

References:
GIcat mapping - WCS : ISO 19139

Record Set (GML Range Set)

A Record Set is a group of Entity Sets mapped by ordinal which share a common shape.

Coverage Model

The coverage model provides 3 concrete coverages classes that fullfil the AbstractCoverage: ViewCoverage, ComplexCoverage, SimplexCoverage
SimplexCoverage contains a set of AbstractParameterValues which are blocks of data, an implicitly associated set of AbstractParameterTypes which are the type and context for each value block, domain information and a RecordDictionary.

Goal is to keep the typing separate from the values due to the nature of the majority of our data (array's of values all of the same type)

Typing system exists above the concepts of float, int, double, etc - these are Boolean, Count, Quantity, etc. Also a concept of Ranges over those types

RangeDictionary is a 1-to-1 component of the RangeType - it is just associating a bunch of names with the concrete name of the parameter (via AbstractParameterType)

How does a parameter become associated with a coverage? The domain spec of the coverage defines shape of the brick that defines the parameter. Make me a parameter of specification domain shape --> a brick who's shape is equivalent to the shape of the coverage. Associate this parameter with the coverage.

When you ask for a parameter (for now), you get a COPY of the pval & ptype

  • parameter has a mutable flag and when someone makes it go from mutable to mutable, it forces the copy at that point

SpatialDomain is set when Coverage is constructed and is immutable
TemporalDomain is set when Coverage is constructed an is mutable (append)

Error formatting macro: plantuml: java.lang.IllegalStateException

title <b>Coverage Model</b>

skinparam class {
ArrowColor Black
BackgroundColor<<Content>> LightSteelBlue
BorderColor<<Content>> SteelBlue
BackgroundColor<<AbstractIdentifiable>> LightGray
BorderColor<<AbstractIdentifiable>> DarkSlateGray
BackgroundColor LightYellow
BorderColor Black
}

enum "AxisTypeEnum" as atenum {
ENSEMBLE
GEO_X
GEO_Y
GEO_Z
HEIGHT
LAT
LON
PRESSURE
RADIAL_AZIMUTH
RADIAL_DISTANCE
RADIAL_ELEVATION
RUNTIME
TIME
}
note left of atenum: Subject to change, taken from:\nhttp://www.unidata.ucar.edu/software/netcdf-java/v4.2/javadoc/ucar/nc2/constants/AxisType.html

enum "MutabilityEnum" as menum {
IMMUTABLE
EXTENSIBLE
MUTABLE
}

enum "VariabilityEnum" as venum {
TEMPORAL
SPATIAL
BOTH
NONE
}

class "DomainOfApplication" as doa {
+ int topological_dimension
+ slice[] slices
}

abstract "AbstractCoverage" as cov <<AbstractIdentifiable>> {

== classmethods ==
+ AbstractCoverage load(str cov_path)
+ void save(AbstractCoverage cov, str out_path)
}

class "SimplexCoverage" as ecov <<AbstractIdentifiable>> {
+ AbstractDomain temporal_domain
+ AbstractDomain spatial_domain
+ AbstractParameterValue[] _range_value
+ ParameterContext[] _range_dictionary
+ ParameterDictionary parameter_dictionary
+ str info

+ void _init_(RangeDictionary range_dict, AbstractDomain spatial_domain, GridDomain temporal_domain)
+ void append_parameter(ParameterContext parameter_context) @deprecated
+ Parameter get_parameter(str param_name)
+ ParameterContext get_parameter_context(str param_name)
+ ndarray get_parameter_values(str param_name, DomainOfApplication temporal_doa, DomainOfApplication spatial_doa, ndarray return_array)
+ ndarray get_time_values(DomainOfApplication temporal_doa, ndarray return_array)
+ void insert_timesteps(int count, int origin)
+ str[] list_parameters(bool coords_only, bool data_only)
+ int num_timesteps()
+ void set_parameter_values(str param_name, ndarray values, DomainOfApplication temporal_doa, DomainOfApplication spatial_doa)
+ void set_time_values(ndarray time_values, DomainOfApplication temporal_doa)

  • _append_parameter(ParameterContext parameter_context)
    }

class "ComplexCoverage" as ccov <<AbstractIdentifiable>> {
+ AbstractCoverage[] coverages
}

class "ViewCoverage" as vcov <<AbstractIdentifiable>> {
+ handle-to-AbstractCoverage reference_coverage
+ AbstractFilter parameter_filter
+ AbstractFilter structure_filter
}

abstract "AbstractFilter" as filt <<AbstractIdentifiable>>
class "StructureFilter" as sfilt <<AbstractIdentifiable>>
class "ParameterFilter" as pfilt <<AbstractIdentifiable>>

abstract "AbstractDomain" as dom <<AbstractIdentifiable>> {
+ CRS crs
+ AbstractShape[] shape
+ MutabilityEnum mutability

+ void _init_(AbstractShape shape, CRS crs, MutabilityEnum mutability)
+ int get_max_dimension()
+ int get_num_elements(int dimension)
+ void insert_elements(int dimension, AbstractShape shape, DomainOfApplication doa)
+ {} dump()

+

Unknown macro: {static}

AbstractDomain load({} ddict)
}

class "TopologicalDomain" as tdom <<AbstractIdentifiable>>
class "GridDomain" as gdom <<AbstractIdentifiable>> {

}
note bottom of gdom: GridDomain is presumed to always\napply to 0d topological entities

abstract "AbstractShape" as shp <<AbstractIdentifiable>> {
+ str name
+ int[] extents

+ _init_(str name, int[] extents)
+ int rank()

  • {} _dump()

-

AbstractShape _load({} sdict)
}

class "GridShape" as gshp <<AbstractIdentifiable>>

class "CRS" as crs <<AbstractIdentifiable>> {
+ dict Axes


+ _init_([] axes)

  • {} _dump()

== classmethods ==
+ CRS lat_lon()
+ CRS lat_lon_height()
+ CRS standard_temporal()
+ CRS x_y_z()

  • CRS _load({} cdict)
    }

class "ParameterDictionary\n(formerly RangeDictionary; Taxonomy)" as rdict <<AbstractIdentifiable>> {

  • ParameterContext{} _map


+ _init_(ParameterContext[] contexts)
+ void add_context(ParameterContext context)
+ {} dump()
+ ParameterContext get_context(str param_name)
+ str key_from_ord(int ordinal)
+ int ord_from_key(str param_name)

== classmethods ==
+ load()
}

abstract class "AbstractParameterValue" as pval <<Content>>

abstract "AbstractSimplexParameterValue" as asp <<Content>>
abstract "AbstractComplexParameterValue" as acp <<Content>>

pval <|-- asp
pval <|-- acp

class "ParameterContext" as pc <<AbstractIdentifiable>> {
+ AbstractParameterType param_type
+ VariabilityEnum variability
==which of these go to type==
+ AxisTypeEnum reference_frame
+ ValueType[] allowable_values
+ fill_value
+ nil_value
..templated attrs..
. . .


+ boolean is_coordinate()

  • {} _dump()
    == classmethods ==
  • ParameterContext _load({} pcdict)
    }

abstract "AbstractParameterType\n(SWE::AbstractDataComponent)" as ptype <<AbstractIdentifiable>>

abstract "AbstractSimplexParameterType" as aspt <<AbstractIdentifiable>>
abstract "AbstractComplexParameterType" as acpt <<AbstractIdentifiable>>

pc o-- ptype

ptype <|-- aspt
ptype <|-- acpt

vcov o-- cov

cov <|-- ecov
cov <|-- ccov
cov <|-- vcov

ccov o-- "2..*" cov

vcov -- "" filt
filt <|-- sfilt
filt <|-- pfilt

shp <|-- gshp

dom <|-- tdom
dom <|-- gdom
dom o-- crs
dom -- "" shp

ecov -- "" pval
ecov -- "" pc
ecov *-- " 1..2" dom
ecov o-- rdict

Figure 5. Coverage Model Specification (OV-7)

Coverage API

The Coverage API is the object API against which ingestion and retrieve (replay) processes are written. The API is independent of the science data feature type it is describing

The figure below shows a remote container accessing a coverage using a local coverage API that makes remote calls against the coverage persistence layer (see below). Note: Data transforms do not directly access the Coverage API in Release 2. Instead they work against streams of Granules, as produced by real-time or replayed data streams.

Figure 6. Coverage API and external access (OV-1)

Persistence Layer

See CIAD DM SV Coverage Model Persistence for details.

References

See also:

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.