Skip to end of metadata
Go to start of metadata
This page describes the plan for a subsystem iteration. It contains high level goals for the release, the phase, and the iteration. It should also contain dependencies on other subsystems, plans for integrating functionality and a list of iteration tasks or a link to the subsystem google doc containing the iteration task details. This page should also contain references to the associated architecture pages, construction plan pages, and use cases.

Table of Contents

Vision and Scope

Heart of E1 will be finalizing the design for the EPU and process management services.

Elaboration phase goal: Fully integrated launch using the process dispatcher. Making it fast and making it flexible to multiple launch environments (laptop, CyberPoP). Working with integration team, allow for a system launch for the entire system.

Construction: productize the R2 CEI Scope

High-Level Iteration Tasks

Iteration Tasks

Iteration Result

Task Summary
Adjustments to R1 code to work towards R2 container and goals This time was spent on R2 container research, gevent research and experimentation, and Celery/kombu research
and experimentation (alternate messaging harness).  No actual porting until R2 container tutorial next week.
Availability Zone Support Ready for inclusion in next Nimbus IaaS release.
Detailed design for RM Architecture document: CIAD CEI OV Elastic Computing
Lightweight Provisioner The "LWP" starts local Vagrant VMs and there is a mapped out place to start an R2 container in the local directory.
Concurrency research and prototyping
Researched ZooKeeper, accepted it, and prototyped multi-worker service. See: CEI Zookeeper Evaluation
DM Workshop
This went from in-person to phone+email.  See: CEI+DM Collaboration
Taskable resource framework design Postponed. Patrick used extra time to investigate R2 container and work on task 192.
Initial Chef recipe for new container Postponed.  We did all spend time on trying to install and understand the R2 container.  And spent time documenting
the whole bootstrap flow in anticipation of this task: VM Instance Bootstrap
Begin EPU implementation
Time boxed, made much progress: EPUManagement Refactor
Detailed design for Process Dispatcher Architecture document: CIAD CEI SV Process Dispatch Design
Detailed design of EE agent See: CIAD CEI SV Process Dispatch Design and CIAD CEI SV EE Agent Implementation
Prototype a simple process Blocked on R2 container.  We have all the pieces but not put together and working in an R2 container.
Detailed design of process management library See: CIAD CEI SV PIDantic Process Management Library
Skeleton implementation of Process Dispatcher Working skeleton with some interactions simulated (it interacts with EPUM and EE Agent which are not built).
Prototype process execution
Dropped, decided this will be mostly DM (see task 197 above).  CEI side is covered in tasks 202, 203, 204.

Subsystem Dependencies

Dependencies with COI:

Dependencies with DM:

  • Datastream process design

Integration Points

See R2 Integration Points


  • Distractions
  • Getting the necessary high-level designs done and committed
  • Availability of new COI Container
  • Quality of new COI Container
  • Lock/persistence situation


Most relevant pages from CIAD CEI Common Execution Infrastructure architecture pages:

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.