View Source

{info}This page describes behavior related to managing data streaming resources and container level objects.{info}

h2. Data Stream Resource Interactions

The figure below demonstrates the activity sequence for the three key processes of the data management subsystem, Register, Subscribe and Publish.

!2910-00020 OV5 DM Register subscribe publish data stream.png!

_Figure 1. Registration, Subscription, Publication Activities (OV-6)_

When the stream is registered, the producer describes the structure of the data to the system at that time. The system responds with an identifier/reference to that structure description.
The producer can update this structure at a later time and may receive an updated reference in return.

When the producer streams data to the system it is a unidirectional conversation. Each data message contains a reference to the structure in the header.
A consumer can request the structure definition of a stream, can request data from that stream in real time, or can request the stream from the beginning of the stream.

h2. Container Stream Object Interactions

The figure below shows the dependencies between capability container level message publishers and subscribers. The following design drivers exist:

* Creation and removal of message broker resources should be symmetrical with clear responsibilities
* Interactions may be a multi-participant interaction, e.g. a service sets up the message queue and the worker process connects to the queue

!https://docs.google.com/drawings/pub?id=18uNogqGbkDv9XPPs_7oKJwqQAM5LsD4WgSSoALnJHLQ&w=890!

_Figure 2. Container stream publisher and subscriber interactions_

Code in Pyon: [endpoint module|https://github.com/ooici/pyon/blob/master/pyon/ion/endpoint.py]