Define, register and update an executable process, based on a source code or binary process definition.
|Actors||System Process Developer, Process Operator (e.g., Data Process Operator)|
|References||UC.R2.18 Visualize Data Product
UC.R2.19 Produce Matlab Visualization
|Uses||UC.R2.48 Schedule Process for Execution
UC.R2.44 Define Service Type During Runtime
UC.R2.51 Define Execution Engine
|Is Used By||UC.R2.45 Instantiate Service Anywhere During Runtime|
|Is Extended By|
|In Acceptance Scenarios||AS.R2.04A Data Product Leads Drive Core Data Product Creation|
|Technical Notes||See components "Source code representation services" and everything else in 18.104.22.168.2.2.|
|Primary Service||Process Catalog & Repository Services|
|UC Status||Mapped + Ready|
This information summarizes the Use Case functionality.
System Process Developer creates a process definition that references a versioned process definition package. The user supplies this definition to ION. The definition or additional user inputs provide information about how the process is to be instantiated; with which inputs it may be executed; which outputs it creates; which execution modes it supports (e.g. stream based, schedule based, maximum frequency, one time); which parameters it supports; which provenance its outputs create; which additional resources are required; which anticipated resource use occurs; which error conditions and notifications are supported. After process definition, the system can schedule this process UC.R2.48 Schedule Process for Execution using the provided information. The appropriate outputs are distributed.
- Processes can be data stream processes and service processes
- Process definitions are available in packaged, versioned form on the software component repository
- Process definitions may be source code or binary, with additional metadata descriptions
- Each process declares which execution engine it supports
- Processes are executed by execution engines
- In Release 2, typical process types are Python (with science extensions) and Matlab
The System Process Developer has a process definition script within a versioned process definition package.
- The System Process Developer selects the Integrated Observatory capability to define a process.
- Via an operator console or an API function
- During operations of the system
- Note: processes may also be bundled with the system during integration time
- The System Process Developer enters or selects basic process metadata
- Producer and ownership information
- Name, version, description, type, use
- Supported execution engines, based on a given selection of available execution engines
- The System Process Developers provides the process definition or a reference to a process definition
- A reference to the versioned process definition package on an available software component repository; this can be the Integrated Observatory software component repository, GitHub, PyPi, or similar
- Alternatively the process developer can directly provide a process definition package in file format
- The system accesses the process definition and places it in a system internal repository
- The system checks basic validity of the process definition against supported execution engines, e.g., can it be installed without errors, does it have an included metadata manifest
- (optional) The System Process Developer declares process resource use metadata
- Which resources are required for process installation: Software packages, services, other processes; Licenses
- Which resources are required before process can be instantiated: Software packages, services, other processes; Operational unit size, network location, security, etc.; Local storage; Licenses
- Which resources are consumed during process execution and how does the process perform: Memory, temporary storage, network bandwidth; Execution time; Average throughput
- The System Process Developer or Process Operator declares process inputs, parameters and outputs
- Done as part of Transform Management Service.
- Provides instantiation mode (One time process, instantiate on use, stream process, scheduled activate [TF: is scheduled something the developer would have a say in?], service process; Singleton, worker pattern, concurrent)
- Declares process inputs: Which types of input data packets are produced
- Declares process parameters: Which parameter values (named, typed key-value pairs) can be provided during process instantiation (e.g. average over x packets)
- Declares process outputs: Which types if output data messages are produced
- (optional) The System Process Developer or Process Operator specifies instantiation and product use restrictions
- Instantiation restrictions (access control): Only in certain execution zone, Only by certain user groups/roles, Only when certain conditions are true
- Restrictions on product use, e.g. only for private consumption
- (optional) The System Process Developer or Process Operator can update process metadata
- If the process definition is changed, the system maintains the old definition and the new definition as separate versions with distinguishable provenance
- Users with appropriate access privileges can search for defined processes by their attributes.
Process is registered in the system resource registry and directory services, process definition is present in the system, and process can be instantiated.
These comments provide additional context (usually quite technical) for editors of the use case.