|This page describes the plan for a subsystem iteration including the high level goals for the iteration. It should also contain dependencies on other subsystems, plans for integrating functionality and a list of iteration tasks or a link to the subsystem google doc containing the iteration task details. This page should also contain references to the associated architecture pages, construction plan pages, and use cases.|
There are two primary goals in E2:
- Port CEI services and functionality away from ION/Twisted. CEI will not immediately use pyon but will instead use AMQP messaging directly. At the end of the iteration, the core EPU services (Provisioner, DTRS, EPU Management, and OUAgent) will be functional and integrated without any dependency on Twisted or R1 ION.
- Produce a working prototype of process dispatching. This will include a functional Process Dispatcher managing and dispatching processes to at least one functional EEAgent. This EEAgent will be running on a VM resource managed by the EPU Management service.
Additionally, integration efforts will continue with a Chef recipe for launching pyon containers.
All relevant CEI code has been ported away from ION and Twisted. The few things that were not ported are being reevaluated for a better strategy in R2. For example, cassandra code was not ported as we will use Zookeeper/CouchDB. Additionally, some of the EPU "scripts" (called via ssh by the launch plan and epumgmt) will be directly integrated with epumgmt. We have many tests and have produced an example launch plan.
We've also made a lot of progress on process dispatching, however the code is still quite soft. Many more higher level tests are needed and the internals of the process dispatcher need to be refactored to make way for ZooKeeper integration. Though I believe we have (or will by the end of the week) reasonably completed all iteration tasks, we are still not where we should be at this point in Elaboration. E3 will be difficult.
- JIRA search: CEI-R2E2-ALL
- Transition of Senior Developer responsibilities
- Porting away from twisted could have unforeseen obstacles
- CEI alternative messaging could be more costly than expected
Most relevant pages from CIAD CEI Common Execution Infrastructure architecture pages:
- CIAD CEI SV Process Dispatch Design
- CIAD CEI OV Elastic Computing
- CIAD CEI OV Logical Regions
- CIAD CEI SV EE Agent Implementation
- CIAD CEI SV PIDantic Process Management Library
- EPUManagement Refactor