Skip to end of metadata
Go to start of metadata
This page describes query possibilities supported by the Discovery service

Discovery Query Expressions Formats

There are 3 possible formats for query expressions:

  • Datastore Query Format: A flexible, comprehensive structured format supporting complex, nested queries
  • Discovery Service Query DSL: A textual syntax for simple queries
  • Discovery Service Intermediate Format: Transcoding of the textual DSL syntax for slightly enhanced flexibility

The preferred format for future support is the Datastore query format. The following sections describe the available formats in more detail.

Datastore Query Format

This format is a structured object format, based on a query dict (data structure) with defined keys identifying operators and filters. It is understood by the OOINet datastores as generic query format and can be executed against the resource registry datastore and events datastore. Powerful nested queries and view queries are possible with this format. This format is the suggested choice for maximum future compatibility.

Details for this format are explained in a separate section below.

Examples

Return all instrument devices that have an associated instrument model with attribute manufacturer set to value Sea-Bird.

or as command line curl statement:

Discovery Service Query DSL

Discovery service query DSL is a textual syntax for simple resource and event queries, inspired by SQL. It is parsed by the discovery service into Discovery Service Intermediate Format.

See below for a definition of the query DSL in Backus-Naur form (BNF)

Examples

Discovery Service Intermediate Format

This format is a structured object format, based on a query dict and known keys identifying operators and filters.

Examples

Datastore Query Format

This format is a structured object format, based on a query dict and known keys identifying operators and filters. It is understood by the OOINet datastores as generic query format and can be executed against the resource registry datastore and events datastore.

Basic Query Structure

The query object is a dict data structure with the following keys:

Query Object Key Type Description
QUERYEXP str Identifies this object as query object. Must be set to value "qexp_v1.0"
query_args dict Mandatory dict of additional query arguments. The following rows explain the contents of the query_args dict
query_params dict Optional dict of parameterized values to resolve during query execution
query_args/datastore str Identifies the target datastore. Possible values see below
query_args/profile str Identifies the target datastore profile. Possible values see below
query_args/ds_sub str Optionally identifies a sub-datastore. Possible values see below
query_args/id_only bool If True, returns only IDs of objects
query_args/limit int If not 0, returns a maximum number of results
query_args/skip int If existing, skips a number of results for pagination
where list A filter expression. Can be combined with and, or, not operators and arbitrarily nested. See below for a list of possible operators.
order_by list A result ordering expression. See below for an explanation of result ordering.

Supported datastore/profile combinations

Datastore Profile Sub-Datatore (ds_sub) Description
resources RESOURCES (empty) Resource metadata objects
resources
RESOURCES
assoc
Resource association objects
events EVENTS (empty)
Persisted event metadata objects

Query Operators

The following operators are available. Additional operators may exist.

Operators are expressed in form of a 2-tuple with the operator code as first element and the second element a list of additional elements as defined below.

Operator Arg 1 Arg 2 Arg 3 Arg 4 Arg 5 Description
exp:and op expression op expression (unlimited)
etc etc etc
Intersection (AND) of the argument operator expressions
exp:or op expression op expression (unlimited)
etc
etc
etc
Conjunction (OR) of the argument operator expressions
exp:not op expression         Negation (NOT) of the argument operator expressions
op:eq attribute name comparison value       Value equals
op:neq
attribute name
comparison value
      Value not equals
op:lt
attribute name
comparison value
      Value lower than
op:lte
attribute name
comparison value
      Value lower than or equal
op:gt
attribute name
comparison value
      Value greater than
op:gte
attribute name
comparison value
      Value greater than or equal
op:like
attribute name
comparison value
      Value like pattern (Postgres LIKE format, see here)
op:ilike
attribute name
pattern expression
      Value like pattern, case insensitive (Format, see here)
op:fuzzy
attribute name
comparison value
      Value similar
op:regex
attribute name
regex expression       Value matches regex (POSIX regex, see here)
op:iregex
attribute name
regex expression
      Value matches regex, case insensitive (POSIX regex, see here)
xop:in
attribute name
possible value possible value (unlimited)
etc
etc
Value in list of possible values
xop:between
attribute name
lower bound upper bound     Value in between bounds (inclusive)
xop:allmatch
contained value
        Any first level attribute contains given value
xop:istype
type expression         Find objects with type or base type equal to given value (e.g. events)
gop:overlaps
geospatial attribute name
x1 y1 x2 y2 Find objects with geometry overlapping given bbox
gop:within
geospatial attribute name
x1 y1 x2 y2 Find objects with geometry within given bbox
gop:contains
geospatial attribute name
x1 y1 x2 y2 Find objects with geometry containing given bbox
gop:overlaps_geom
geospatial attribute name
geometry in WKT format buffer (in degrees or meters if ends with m)     Find objects with geometry overlapping given WKT geometry
gop:within_geom
geospatial attribute name
geometry in WKT format
buffer (in degrees or meters if ends with m)
    Find objects with geometry within given WKT geometry
gop:contains_geom
geospatial attribute name
geometry in WKT format
buffer (in degrees or meters if ends with m)
    Find objects with geometry containing given WKT geometry
rop:overlaps
vertical/temp attribute name
lower bound
upper bound
    Find objects with range overlapping given range
rop:within
vertical/temp attribute name
lower bound
upper bound
    Find objects with range containing given range
rop:contains
vertical/temp attribute name
lower bound
upper bound
    Find objects with range containing given range
assop:associated
target resource id target type (None, str or list) predicate (None, str or list) direction expression (see below) target filter op expression Resource is associated with other resources
assop:descend_o
target resource id
target type (None, str or list)
predicate (None, str or list)
direction expression
target filter op expression
Match all descendants of resource (object direction)
assop:descend_s
target resource id
target type (None, str or list)
predicate (None, str or list)
direction expression
target filter op expression
Match all descendants of resource (subject direction)

Standard Attributes

The following attributes are standard attributes that can be queried using the standard "op:" operators. Other attributes can also be queried but are computationally more expensive on the database because they are not indexed.

Datastore Attribute Name Operators Description
resources id Standard (op: prefix) Resource ID (uuid)
resources type_ Standard (op: prefix)
Resource type
resources name Standard (op: prefix)
Resource name
resources lcstate Standard (op: prefix)
Resource lifecycle state
resources availability Standard (op: prefix)
Resource availability state
resources ts_created Standard (op: prefix)
Resource metadata creation timestamp (as string with millis since 1970-01-01)
resources ts_updated
Standard (op: prefix)
Resource metadata last update timestamp (as string with millis since 1970-01-01)
resources
geom Geospatial (gop: prefix) Current resource geospatial center point
resources
geom_loc Geospatial (gop: prefix)
Current resource geospatial 2D bounding box
resources
vertical_range Range (rop: prefix) Current resource vertical range (meters)
resources
temporal_range Range (rop: prefix)
Current resource temporal range (as seconds since 1970-01-01)
resources (assoc) s Standard (op: prefix)
Association subject resource ID
resources (assoc)
st Standard (op: prefix)
Association subject resource type
resources (assoc)
p Standard (op: prefix)
Association predicate
resources (assoc)
o Standard (op: prefix)
Association object resource ID
resources (assoc)
ot Standard (op: prefix)
Association object resource type
events id Standard (op: prefix)
Event ID (uuid)
events type_
Standard (op: prefix)
Event type
events origin Standard (op: prefix)
Event origin value (typically a resource ID causing the event)
events origin_type Standard (op: prefix)
The type of resource producing the event
events sub_type Standard (op: prefix)
A sub-category for the event type, e.g. CREATE
events
ts_created Standard (op: prefix)
Event occurrence timestamp (as string with millis since 1970-01-01)
events
actor_id Standard (op: prefix)
ID of actor causing the event

Datastore Query Examples

Return all resources with attribute "firmware_version existing" and value equal to "A2".

Return all resources owned by the current actor

Return all child sites of given Observatory resource:

Return all child sites of given Observatory resource:

Return all instrument devices that have an associated instrument model with attribute manufacturer set to value SeaBird.

Return all instrument devices that have an associated instrument model with attribute manufacturer set to value SeaBird and are shared in a facility (Org) with name "CSGN Facility".

Query Arguments and Parameters

The following arguments can be provided within the search_args argument:

Argument Type Description
count bool If present the result is a list with one numeric value: the number of matching objects
attribute_filter list If present and id_only==False, discovery filters the resulting objects  A list of top level attribute names for returned discovery objects. Note that attributes _id and type_ are always returned
query_info bool If True, the last element of the result list is a dict with query stats for development purposes

Query Ordering

The query return is a list of objects or objects ids. If no ordering is present, there is no guarantee on result order, although order is typically stable in between calls.

The order_by query key is a list of 2-tuples, with elements (standard attribute name, order direction). See above for a list of permissible standard attribute names. Order direction can either be "asc" or "desc" for ascending respectively descending result order. If more than one ordering tuple is present, ordering is applied following the sequence of the order_by list. 

See below for ordering examples.

Persistent Views and Resource Collections

Persistent Views are named, persisted query expressions that can be evaluated at any given time, potentially yielding different resources as the underlying events, resources and associations change. Resource collections are static lists of resources, such as a personal favorites list or the result of a previous search.

Both persistent views and resource collections can be created on requests and can subsequently be referenced by id and name.

Core Indexes

Core query indexes apply to the query DSL and the query intermediate format (see above).

Index Datastore Description
resources_index resources Query over all resources in the system, visible to the the calling actor's visibility context
data_products_index resources Query over a restricted set of resource types in the system, visible to the the calling actor's visibility context.
Types include: DataProduct, DataProcess, Deployment, InstrumentDevice, InstrumentModel, InstrumentAgentInstance, InstrumentAgent, PlatformDevice, PlatformModel, PlatformAgentInstance, PlatformAgent, PlatformSite, Observatory, UserRole, Org, Attachment, ExternalDatasetAgent, ExternalDatasetAgentInstance
events_index events Query over all persisted event objects in the system

View Resources

Discovery can query persistent views using the query_view operation. A View resource must exist with the given id or name. A View is a resource object containing a discovery query plus additional information about parameterized values and parameter defaults (see below). 

Parameter values can be provided call of query_view, enabling flexible saved searches.

View resource attributes
Attribute Type Description
_id uuid ID of the View
name str Name of the view
view_type str Empty for now. Supports future different types of view definition
view_definition dict A datastore query (see above). Can have parameters in form of $(parameter_name)
view_parameters list List of CustomAttribute objects. Define the names and defaults of parameters. This is optional. Parameters can be used without definition, but definition is suggested to be able to keep a description, type and default.
param_values dict Defines initial values for parameters. These values are applied over the default values provided in the view_parameters list
Example query_view calls

The user interface can store user searches as Views. Such resources will be associated with the user's ID can can later be presented to the user and executed again.

Query Parameters

Views can apply parameterized values. These are values within the view's filter expressions that will be substituted with actual values when the view executes. A flexible persistent view can be provided this way, allowing users to fill in important parts of the view. Query parameters are string expressions of the form $(parameter_name)

The following basic parameters are available:

Parameter Name Type Description
current_actor uuid Will be filled with the actor making the service request

Resource Collections

A resource collection is a statically managed list of resources. The content of the list does not change (in contrast to a saved search). The actual resource metadata attributes may change.

Discovery Query DSL Definition

Backus-Naur Form

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.