Skip to end of metadata
Go to start of metadata
Data Calibration capabilities are allied automatically for core OOI data products in Release 2, for instruments that don't already apply calibration.

In Release 2, these capabilities will be generalized and extended to enable user defined calibration computations and the flexiblem management of calibration coefficients.

Overview

Data Calibration services are applied to derive higher level data products from unprocessed instrument measurements, for instance to transform unprocessed voltage counts provided by an instrument into engineering units.

The Data Calibration services are based a number of enabling infrastructure services, including:

See Also:

Details

Data Processing with Parametric Values

The term parametric value in this context is not to be confused with the term Parameter in the context of the DM coverage model. Parametric values are eventually stored as parameters of specific type in coverages, but so are science measurements and timestamps, which are not parametric values.

Providing and Parsing Parametric Input

Project scientists and technicians provide parametric lookup tables in spreadsheet form. The basic form of these spreadsheets is a list of rows of unique characteristic, parameter name, parameter value. The unique characteristic might be an instrument model name or id, the serial number of a device, a valid reference designator in OOI or any other information that an associated parser can associated with one or multiple resources within ION.

Procedure:

  • Scientist or technician compose spreadsheet
  • Upload spreadsheet as resource Attachment in ION, e.g. to an InstrumentModel.
  • A parser takes the spreadsheet and places individual rows into the ION object store

Storing Parametric Input

Staging Storage

The ION object store contains a staging store of parametric values to be applied to the various places where these values are needed.

Storage in the Coverage

During distinct times in the lifecycle of ION resource, lookup values are extracted from the above staging storgage location in the ION object store and placed as values of constant or sparse parameter types into the respective coverages.

This affords the ION operations with a time window of collecting and maintaining these parametric values between the definition of the ION resource and its "activation" for operational use. For instance, device DataProduct resources are created during the ION load process in a PLANNED lifecycle state. There is a window of several months preparing for the actual physical deployment of these devices during which the parametric values can be collected and attached. Once the physical deployment is prepared, an ION operator can change the lifecycle state of the device to INTEGRATED, which enforces that parametric values are present.

Applying Parametric Input

During Ingestion

Ingestion worker processes are exclusive writers to one or multiple individual coverages.

TBD. Need for ingestion to know parametric values is unclear.

In Coverage Parameter Functions

Derived parameters defined by parameter functions will use parametric values present in the coverage as input for the parameter functions.

Using Constant Type Parameter to Store Parametric Values

Using this parameter type has the side effect that any derived parameters using the constant value as input will change when the constant value parameter changes. This will apply to historic time steps in the entire coverage. This may not be the intended behavior.

Using Sparse Type Parameter to Store Parametric Values

Using this parameter type has the side effect that any derived parameters using the sparse value as input will only apply the value after the time step that the sparse value is associated with. A change of the sparse value only affects time steps after the change, not before. This may not be the intended behavior.

In Data Stream Processes

A procedure has been implemented to scan Attachments to resources for a DataProcess in a certain order for parametric values. Found values are then made available to transform processes as part of the config argument during process spawn.

This has not been fully implemented and does not filter out the specific values for one specific process. Instead it provides all values to the process and leaves the process with the task of extracting the specific values. Not used in R2

Necessary Parametric Values

ION needs to know which parametric values apply to which coverages at the time the coverage (and its ParameterDictionary) is computed.

The ION load procedure needs access to appropriate lookup information for the following types of resources

Resource Type Parametric Class Number of Parametric Values Value Source Comments
InstrumentModel QC lookup value Up to 10 as defined by QC DPS. Actually applying params named in SAF QC lookup spreadsheet provided by PS The actual parametric values apply a default value by model but leave the possibility for specific values by device. The unique device is represented as a reference designator.
InstrumentModel Calibration coefficients Depending on the device and the device IOS respectively. TBD device spreadsheet  
DataProduct Geospatial 2 (latitude, longitude) TBD manual input Certain L1/L2 parameter functions as defined by the DPS are parameterized by the lat/long of the device deployment. Example is PRACSAL (Salinity) computation. Necessary is a nominal location only not a precise actual location.
         
         

Implementation

See

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.