A main capability of the COI Governance Framework is defining, monitoring and supporting interactions between distributed entities in the system, spread across multiple domains of authority.
OOI CI is designed as a 30 year program. It is expectation based on historic evidence that everything but the most basic architectural fundamentals will change during the life time of the program. The most invariant part of the concrete designs and implementations are the interaction protocols, i.e. the message sequences and formats between distributed (service) processes and other processes. These interactions are much more invariant than their various implementations. Even these interaction interfaces will evolve over time, but much slower and only by adding more comprehensive newer protocols, not by modifying or replacing existing protocols.
The benefits of this approach are in multiple dimensions:
- Architects can design services of the system precisely and formally test them for important properties and for compliance with existing specifications. This generalizes to any form of acting entity besides services, such as agents, users, resources.
- Architects can simplify interactions, group them systematically, reduce cyclic dependencies and tight coupling.
- Architects can document interactions unambiguously
- Software implementers can base their fine designs on interaction specifications, for instance to develop two different implementations of the same service (e.g. Java, Python or version 1, version 1.1)
- Software implementers can take well documented interaction specifications understood as blueprints and code against them with confidence
- Code generators can generate interaction monitors from the specifications
- Code generators can generate client interaction stubs from the specifications
- Code generators can generate object model classes from the specifications
- Code generators can generate object encoders and decoders (for transport or persistence) from the specifications
- Testers can derive test cases from interaction specifications, including unit and integration test cases
- Testers can run regression and compliance tests of software components against interaction specifications
- The governance framework can monitor and intercept ongoing interactions between entities, i.e. send and receipt of atomic messages, and act based on this knowledge.
- Entities in their own domain of authority can apply governance framework mechanisms (part of a capability container) to protect their integrity and policy, when interacting with entities from a different domain of authority
- The governance framework can log interactions for later historic analysis
- The governance framework can perform compliance checking of existing implementations against specifications
- The governance framework can enforce policy and longer term, complex commitments across multiple domains of authority (in an OLTP way)
- The system can create audit traces of interactions in the system
- Developers can analyze interaction logs in order to find and resolve complex bugs
- More complex commitments and violations of contracts can be detected by analyzing past interaction histories (in an OLAP way)
Interaction specifications include various complementing parts, such as:
Interaction Protocol Specification:
- Specify the valid sequences of messages for success and failure cases, and identify the different types of messages, as well as the roles of the different participants in an interaction
- In Release 2, we use Scribble.
- Other candidates include MSC (Message Sequence Chart) based graphical and textual languages and formalisms
- Also knows as interaction pattern, conversation type, session type, protocol etc.
Governance and Commitment Specification:
- As annotation to interaction protocol specifications
- Governance acts (contracts, commitments, policy decisions) are performed during various points of an interaction.
- In complement to protocol specifications. Candidates include Google Protocol Buffers, UML MOF (or XML schema), RDF/semantics
- Message formats (realizing message types) are composite objects
- In realization of object specifications. Encode and decode object content for transport and persistence.
The table below contains a list of currently defined protocol specifications.
|CIAD COI OV Brokered Negotiate Protocol||Brokered Negotiate||Negotiation between a Provider and a Consumer, facilitated by a Broker. The negotiation can originate from the Provider (invitation) or the Consumer (request) and exchanges Service Agreement Proposals until both parties accept or the negotiation is cancelled.||Provider, Consumer, Broker|
|CIAD COI OV Inform Protocol||Inform||Standard protocol to inform a Recipient by the Sender.||Sender, Recipient|
|CIAD COI OV Negotiate Protocol||Negotiate||Negotiation between a Provider and a Consumer. The negotiation can originate from the Provider (invitation) or the Consumer (request) and exchanges Service Agreement Proposals until both parties accept or the negotiation is cancelled.||Provider, Consumer|
|CIAD COI OV Service Request Protocol||Request||Standard protocol to make a request to a service and wait for the response, failure or timeout. Two variants: Simple RPC and Service Request||Requester, Provider|
|CIAD DM OV Publish-Subscribe Protocol||Publish-Subscribe||Two protocols; one to register as a producer and to publish continuously. The other to subscribe to a stream and receive messages to consume.||Publisher, Subscriber, Pubsub Management Service, Exchange Space|
- Common interaction mechanism, COI Exchange: All entities in the system (i.e. processes such as services and agents) communicate exclusively via the COI Exchange interaction mechanisms. This makes sure that all communication is comparable and traceable in terms of governance. Note: external interfaces of the ION system can be in any form, such as Web Services, HTTP, DAP etc.
- Common Message Format: Enables consistent, automated interaction monitoring and historic analysis. Every message in the system is self-described sufficient enough to identify the conversation it belongs to, the exact interaction protocol specification it complies with, sender, recipient, encoding, format and semantic interpretation.
- Common Object Model: Enables consistent management of the system's "business objects" and their transport, persistence and use in the code
- Capability Container: The software infrastructure developed by OOI CI COI that enables all interaction management and governance mechanisms (besides other things). Service and developers have limited exposure to these complexities. The capability container exists in various technological environments (such as Python, Java and more in the future).
- CIAD COI TV Conversations and Session Types#Scribble: Scribble is used as the language to specify conversation protocols. Scribble is based on a theory of session types
Figure 1 shows the current draft implementation strategy for implementing the monitoring of conversation protocols between processes.
Figure 1. Draft implementation strategy for conversation monitoring between processes.
Figure 2 shows an the current draft implementation strategy for implementing the Governance capabilities within the capability container.
Figure 2. Draft implementation strategy for interceptors
R1 and R2 are based on architects defining services and service operations, and sequence diagrams of service calls, and developers implementing service operations, making backend service calls. The capability container provides a robust RPC implementation for service calls, and publish/subscribe/consumer for science data and event messages.
The programming model is strongly distributed but still mostly synchronous in the control flow. At any time in the processing of a request, only one service is actively computing. Of course, many high-level requests can be processed in parallel, but the same request is executed synchronously.
The availability of a protocol (conversation type) specification language such as Scribble will enable the definition of global protocols that can be mapped to the individual participants by a tool chain, making the task of implementing protocols easier for the developer, albeit more complex to understand, because control flow is not mixed anymore with business logic. It basically results in call-back based development, where the framework makes the callbacks within each service based on the global protocol specification.
This will enable more asynchronous control flows, where more than one participant can act at any given time in the processing of a high-level request.
Caveats to be determined:
- Timeouts on each side of a protocol
- Synchronization of distributed state after a timeout/error in adverse network conditions
- Non-deterministic choice in protocol influenced by business logic callback
In ION Release 2, all service interactions are based on a simple RPC-style request/response protocol, mapped to message exchange between two participants. The requester sends a message that is routed to the in-queue of the provider (=service). The response is send back to the RPC-response queue of the requester. The requester blocks execution until either a response or failure arrives, or a timeout occurs.
Most service calls originate by human users interacting with the web UI. The web IO on the browser issues service requests to high-level services. These requests go transparently via the service gateway.
In most cases, the high-level services make subsequent RPC calls to lower level services, e.g. a SA service calls a DM service, which calls COI resource registry and other services. A cascading set of RPCs result that are all encompassed by the originating request from the UI. We call this a global protocol or conversation that is composed of sub-protocols. All security attributes, e.g. the acting user's ID are informed by the global protocol. The figure below shows the cascading RPCs forming larger protocols out of simpler RPC protocols.
Figure 3. Global protocol composed of RPCs