Automated Quality Control Functions

These are the generic, automated Quality Control (QC) functions that will be applied to various OOI data products.

In general the QC results use reverse Boolean logic. So, 0 means the value met the test conditions (e.g. for Global Range Test this would mean it exceeds the min/max) and is bad data, and 1 means it did not meet the test conditions and is good data.

Locate the QC Flags in the Data

Navigate to a Data Product and Initiate Download
Ensure QC Variables are Checked and Timeframe is Appropriate
Review QC Flags in Downloaded Data

Understand the Functions

Global Range Test
Spike Test
Stuck Value Test
Polytrend Test
Local Range Test
Gradient Test
Propogate Flags

Locate the QC Flags in the Data

Navigate to a Data Product and Initiate Download

Navigate to the data product of interest, and initiate downloading data.

Ensure QC Variables are Checked and Timeframe is Appropriate

1: Use Slider Bar to Adjust Timeframe of Data - Note that the download can take a significant amount of time because upon download the QC flags are computed, so ensure the slider bar for the time frame is adjusted appropriately for only the data of interest.

2: Combined Data Quality Control Flag - Overview flag for alert purposes.

3: Check and/or Uncheck QC Variables of Interest - The QC variables should be selected on the ERDDAP download page. The example shows (3a) Density and (3b) Practical Salinity test flags.

4: File Type or URL Display Options - Choose the option that best meets the user needs for viewing and downloading the data and QC flags.

5: Submit - Activates the download process, note that the process can take a significant amount of time depending on the variables and timeframe chosen. Also the remaining download process is browser dependent so follow regular browser actions for the rest of the download process.

Review QC Flags in Downloaded Data

The look of the information will be dependent on the browser used and format selected. The example shown here were using a Google Chrome browser and downloading a .csv file format.

Understand the Functions

Global Range Test

falls within a give range

Spike Test

individual data values that deviate significantly from surrounding data values

Stuck Value Test

repeated occurrence of one value in a time series

Polytrend Test

tests a time series for whether the data contain a significant portion of a polynomial

Local Range Test

testing if measurements fall into a user-defined valid range, varies with measurement location

Gradient Test

changes between successive data points fall within a certain range

Propogate Flags

propagates "bad" qc flags to another (derived) datase

Source: https://github.com/ooici/ion-functions/tree/master/ion_functions/qc/README.txt