Skip to end of metadata
Go to start of metadata

Overview

The OOI common message format applies to all messages within the ION system. The common message format specification defines message headers and message content characteristics. It allows for arbitrary message content, as long as the content is described properly in headers and complies with ION content characteristics. The COI Capability Container is the ION component implementing this message format.

Message types are defined as part of the service interface definitions. Messages contain objects of the OOI Common Object Model. Message encoding/decoding as well as type checking and stub generation is handled by the infrastructure.

Message Format and Interaction Levels

To reduce the complexity of the ION interactions, we distinguish between several levels of abstraction:

  • At the application (e.g. service-to-service or process-to-process) level, two applications engage into a conversation based on a previously agreed upon conversation protocol, such as RPC.
  • At the integration level, applications execute as a process in a capability container, and each message is routed through defined message interceptors for governance, policy and identity. All processes have well-defined message names in the ION Exchange.
  • At the messaging level, capability containers interact through their messaging clients. The conversation from the integration level becomes a conversation between a messaging client and a previously known broker that contains a representation of the Exchange Point. Messages are encoded into binary form on this level.

The following figure shows how each level for communication adds its own header to the message. All headers are always piped through the Identity Management system (IdM) corresponding to the current layer. The existence of the IdM presupposes a domain of authority.

Figure 1. Different levels of abstraction in message interactions

The diagrams in the behavior section of the COI capability Container page show the interaction pattern within the capability container when an application sends a message.

  • When the message is sent, it goes to the capability container (integration level) that sets some headers and sends the message to the interceptors. The interceptors will update local knowledge base and act - the sender interceptor might block the message from going out, in which case the denial should propagate back to the application. If the agent does not block the message, it goes to the signer, who adds the signature and other IdM headers.
  • Then, the Messaging Abstraction (messaging level) component converts it to AMQP format and sends it to the Broker.

Common Message Headers

Overview

A message transported via a messaging infrastructure (such as an AMQP Message Broker) is a unit of communication that can be transported, understood and processed atomically. The underlying network layer ensures that messages pass through the network unmodified.

The bytes comprising a message can be distinguished into the following parts

  • Messaging level transport header and footer: For AMQP based systems, these are AMQP headers and footers. Also called the "message envelope". Certain headers can be modified while a message is in transit and is routed across multiple hops
  • Integration level headers (immutable): Headers specific to the OOI Exchange system, describing invariant properties about the message content, such as sender and receiver name and specification of the message content in form of encoding, structure, language and ontology and association of the message to a conversation and an interaction specification,
  • Application level content (immutable): The actual payload of a message. Entirely

Application Level Content

Message types are defined as specific objects of the OOI Common Object Model. Message types are defined as part of service interface definitions.

Applications are free to define objects as needed to place any application content into the message.

Science Data in messages is handled and encoded as described in Science Data Transport.

Integration Level Header

Headers are based on the FIPA ACL Message Structure specification.

Header Header Category Required?
Description Implementation Comments
performative
Communicative Act (based on FIPA ACL)
mandatory
The type of communicative act expressed by the message. I.e. the category of a message (e.g. request, response, negotiate, error etc.)
In R1, values included request, accept, reject, inform-result, failure.

In R2 will use in policy interceptor to ignore policy checking on result and failure messages.
op Communicative Act
mandatory for request only
The operation (also known as action, method) for a "request" message.
In particular selects the service/agent operation
sender Participant in communication (based on FIPA ACL) mandatory The exchange name of the sender ION process. Do not use this name to reply; use reply-to for this purpose.
 
sender-service
Application optional
The exchange name of the sender ION service. Do not use this name to reply; use reply-to for this purpose. New in R2
sender-type
Application
optional
The process type of the sender process. Includes service, agent, standalone, stream-process
New in R2
sender-name Participant in communication optional
A human readable name of the sender process, used for logging and debugging purposes.
This name is not unique in the system
receiver Participant in communication (based on FIPA ACL) mandatory The exchange name of the receiver. The Receiver entity with this name may be an ION process, an ION agent, a service, or an Exchange Point.
 
reply-to Participant in communication (based on FIPA ACL) mandatory The exchange name for replies to this message and for error and undeliverable notifications.  
reply-with Control of conversation (based on FIPA ACL) optional
The reply-with parameter provides a token that can be returned in reply to this specific message, if there is potential for ambiguity. It is useful to follow a conversation thread in a situation where multiple dialogs occur simultaneously. not set in R1
in-reply-to Control of conversation (based on FIPA ACL) optional
Contains the token provided in a prior message 'reply-with' header to distinguish multiple dialogs that occur simultaneously. not set in R1
reply-by Control of conversation (based on FIPA ACL) optional
Denotes the latest time by which the sender would like to have received the next message in the protocol flow (this is NOT the latest time by which the entire conversation should terminate). For instance, indicates the timeout of a request message.
not set in R1.
Note: this requires synchronized clocks.
Timestamp is in millis in UNIX epoch (UTC)
ontology Description of Content (based on FIPA ACL) TBD
Ontology identifier that describes the interpretation domain for message. Currently unused.
unused in R1
protocol Control of conversation mandatory Identifies the conversation type (also knows as interaction protocol, interaction pattern) to which this conversation id and this message belongs to.
In R1, included 'rpc' and 'request' and 'generic'
conv-id Control of conversation mandatory Identifies the conversation ID for this instance of the protocol. Note: The initiator of the protocol must assign a unique value to the conversation-id parameter.
All responses to the message within the scope of the same conversation must contain the same value.
 
conv-seq
Control of conversation mandatory Sequence number of conversation messages. Starts with 1 and increases with every response and follow-up.
 
original-conv-id Control of conversation mandatory Identifies the original conversation ID across a sequence of request/response messages.
 
language Description of Content (based on FIPA ACL) TBD
Specification language in which the message format is described in. Allows for future extensibility. Within ION, the language is typically set to a specific value.
In R2, set to "ion-r2"
format Control of conversation (based on FIPA ACL) mandatory Identifies a registered format specification for this message in the given "language". It is typically the type object name for the content of the message. unused in R1
encoding Description of Content mandatory Transport encoding of message content. Examples include msgpack.
In R1, set to TBD. In R2, set to N/N TBD
accept-encoding
Description of Content optional
Possible encodings accepted in response messages.
obsolete in R1 production, obsolete in R2?
ts
  mandatory Timestamp when message was sent.
Timestamp is in millis in UNIX epoch (UTC)
status
  mandatory Status return value set by the application or container in response messages to provide indication of success or failure of a request. HTTP status codes are used
 
msg-rcvd
  optional Timestamp when message was received.
In millis since UNIX epoch (UTC). Used to calculate messaging latency in sFlow statistics.
process-saturation
  optional ION Process saturation
Integer percentage (0-100) of amount of time spent processing requests over total time running. Does not include messaging latency.
resource-id
ION governance
optional - but required for certain governance operations.
The resource id of the resource that is targeted by this message's performative and operation. This is not always set since operations like find and create do not have a resource associated with them. Operations on a resource like read, update, delete would have a specific resource ID and then the header should be set by the caller.
New in R2
Typically will be set by the use of a #@ResourceId decorator on the service operation parameter which contains the resource id
ion-actor-id
ION security
optional OOI id of user (UserIdentity resource id) or acting resource (ResourceIdentity resource id) that originated the current request, or 'anonymous'. This propagates to downstream requests.
 
expiry
ION security optional Timestamp of when the user authentication expires. This propagates to downstream requests.
Timestamp is in millis in UNIX epoch (UTC)
ion-actor-roles ION security optional
For the acting identity, lists the Orgs that the identity is member of and the roles within these Orgs. Note: membership in an Org always implies the "Member" role in that Org.  This will only be passed on downstream if an ion-actor-id is present.
New in R2.
The format of this value is a JSON dict of Org names, mapping lists of role names. Example of header name and value:
ion-actor-tokens
ION security optional
Any previously issues, not yet expired tokens for this acting identity. Can be cached and relayed in subsequent requests to speed up subsequent processing and avoid repeated permission checks. This will always be passed on downstream if an ion-actor-id is present or not, since unset actors are still anonymous actors. Should probably be a dict as a general basket of tokens so that endpoints can create and use tokens as they need to. The messaging layer will just pass these on.
origin-container-id
ION security optional Identifies the node (typically a container or a client) from which this message originated.
 
origin-certificate
ION security optional X.509 certificate for the origin (node or container) that signed this message. Includes chain of trust to a known authority.
 
message-signature
ION security optional A digital signature provided by the originator of this message.
 


Messaging Level Header

ION supports different message transports, including AMQP and ZeroMQ, with potential for additional transports.

The common internal transport relies on AMQP messaging transport (0.8 for Release 1, 0.9.1 for Release 2 LCA). AMQP level headers wrap common ION messages and have the following mandatory and optional fields. Note: these headers are used by the message transport only and are of no relevance to ION processes and business logic.

Parameter Description
durable  
priority  
transmit-time  
ttl  
former-acquirers  
delivery-failures  
format-code  
message-attrs  
delivery-attrs  

AMQP Message properties (immutable).

Parameter Description
message-id Unique AMQP level identifier for the message
user-id Only for AMQP use; not to be confused with the ion-actor-id below
to  
subject  
reply-to  
correlation-id  
content-length  
content-type  
AMQP security headers
Digital signature etc as needed for AMQP message security

Message Context Propagation

Message context is information that carries over between one message and another, as part of a process sending and receiving related messages.

Message Context is propagated in a number of cases:

  1. Downstream message: When a message is received and triggers some action, and the action includes sending further messages
  2. Reply message: When a message is replied to the original sender as a consequence of a causing request message

Downstream Message

The following headers from the original message are propagated to the downstream message:

  • ion-actor-id. In case ion-actor-id was not set, it is set to 'anonymous'
  • expiry
  • ion-actor-roles
  • ion-actor-tokens

Reply message

The following headers from the original message are propagated to the downstream message:

  • protocol
  • conv-id
  • conv-seq. The number is increased by 1.

Message Content Encodings

[Data Type Representations (TV-1)|CIDev:DM Data Type Representations

DM Common Data Model (OV-7)

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.