|Data Calibration capabilities are allied automatically for core OOI data products in Release 2, for instruments that don't already apply calibration.
In Release 2, these capabilities will be generalized and extended to enable user defined calibration computations and the flexiblem management of calibration coefficients.
Data Calibration services are applied to derive higher level data products from unprocessed instrument measurements, for instance to transform unprocessed voltage counts provided by an instrument into engineering units.
The Data Calibration services are based a number of enabling infrastructure services, including:
- Data product generation (SA R2)
- Data processing (SA R2)
- [syseng:CIAD DM OV Data Processing with Parametric Values]
- Data distribution (DM)
- Process management services (CEI)
- CIAD DM OV Common Data and Metadata Model
|The term parametric value in this context is not to be confused with the term Parameter in the context of the DM coverage model. Parametric values are eventually stored as parameters of specific type in coverages, but so are science measurements and timestamps, which are not parametric values.|
Project scientists and technicians provide parametric lookup tables in spreadsheet form. The basic form of these spreadsheets is a list of rows of unique characteristic, parameter name, parameter value. The unique characteristic might be an instrument model name or id, the serial number of a device, a valid reference designator in OOI or any other information that an associated parser can associated with one or multiple resources within ION.
- Scientist or technician compose spreadsheet
- Upload spreadsheet as resource Attachment in ION, e.g. to an InstrumentModel.
- A parser takes the spreadsheet and places individual rows into the ION object store
The ION object store contains a staging store of parametric values to be applied to the various places where these values are needed.
During distinct times in the lifecycle of ION resource, lookup values are extracted from the above staging storgage location in the ION object store and placed as values of constant or sparse parameter types into the respective coverages.
This affords the ION operations with a time window of collecting and maintaining these parametric values between the definition of the ION resource and its "activation" for operational use. For instance, device DataProduct resources are created during the ION load process in a PLANNED lifecycle state. There is a window of several months preparing for the actual physical deployment of these devices during which the parametric values can be collected and attached. Once the physical deployment is prepared, an ION operator can change the lifecycle state of the device to INTEGRATED, which enforces that parametric values are present.
Ingestion worker processes are exclusive writers to one or multiple individual coverages.
TBD. Need for ingestion to know parametric values is unclear.
Derived parameters defined by parameter functions will use parametric values present in the coverage as input for the parameter functions.
Using this parameter type has the side effect that any derived parameters using the constant value as input will change when the constant value parameter changes. This will apply to historic time steps in the entire coverage. This may not be the intended behavior.
Using this parameter type has the side effect that any derived parameters using the sparse value as input will only apply the value after the time step that the sparse value is associated with. A change of the sparse value only affects time steps after the change, not before. This may not be the intended behavior.
A procedure has been implemented to scan Attachments to resources for a DataProcess in a certain order for parametric values. Found values are then made available to transform processes as part of the config argument during process spawn.
|This has not been fully implemented and does not filter out the specific values for one specific process. Instead it provides all values to the process and leaves the process with the task of extracting the specific values. Not used in R2|
ION needs to know which parametric values apply to which coverages at the time the coverage (and its ParameterDictionary) is computed.
The ION load procedure needs access to appropriate lookup information for the following types of resources
|Resource Type||Parametric Class||Number of Parametric Values||Value Source||Comments|
|InstrumentModel||QC lookup value||Up to 10 as defined by QC DPS. Actually applying params named in SAF||QC lookup spreadsheet provided by PS||The actual parametric values apply a default value by model but leave the possibility for specific values by device. The unique device is represented as a reference designator.|
|InstrumentModel||Calibration coefficients||Depending on the device and the device IOS respectively.||TBD device spreadsheet|
|DataProduct||Geospatial||2 (latitude, longitude)||TBD manual input||Certain L1/L2 parameter functions as defined by the DPS are parameterized by the lat/long of the device deployment. Example is PRACSAL (Salinity) computation. Necessary is a nominal location only not a precise actual location.|