The Transition phase of the OOI system life cycle is predominantly focused on testing and defect correction.
Management of Transition activities resides with the System Development Manager and System Engineer. The former manages all day-to-day test and defect correction activities, and coordinates work assignments to the developers and Integration and Test Team. The System Engineer is responsible for delivering a quality and tested system to the PMO at the end of Transition for the acceptance process, and manages all verification testing. The Project Scientist is responsible for validation of the release. The QA Manager is responsible for auditing the processes and products of Transition on an ongoing basis.
Code freeze occurs at the end of the Construction phase (May 13, 2011) for R1. Except for defect correction, code features are not expected to be added after this date.
Defect correction is the core activity of Transition. At the end of Construction and prior to the IOC milestone, the developers are required to enter remaining code bugs into Jira as defects. Defects will be carefully tracked using Jira (i.e., added to or closed) throughout the Transition phase in a dynamic manner. See: Bug Tracking During Transition for details on the process.
Developers, members of the I&T team, and beta testers are all required to report bugs in a formal manner using Jira as they are detected and/or corrected. The System Development Manager will conduct daily scrum meetings and coordinate bug triage with the Senior Developers and Integration and Test team. The System Engineer and System Architect will also participate in these meetings.
The defects in Jira constitute the Defect List that the System Engineer is required to report to the PMO at bi-weekly intervals during Transition, and defects will continue to be tracked once R1 is complete.
At the end of Construction, unit tests provide about 60% coverage of the R1 code base. However, industry best practices require about 80% coverage, and consequently the effort will be made during Transition to expand the unit test base to achieve this level. The System Development Manager will coordinate the activities of developers between defect correction and unit test extension throughout Transition, placing priority on the former but ensuring that developer time is used efficiently to achieve a broader test base. See current code coverage levels at: http://ooici.net/coverage/ioncore-python/
Beta testing by representative users is an important input to the defect detection process. The Project Scientist and selected numerical modelers that constitute the target users for R1 will be engaged at several points during Transition to exercise the system and evaluate its functionality. They will be assisted by the System Development Manager and Senior Developers, who will use the input of beta testers to detect defects and enter them into Jira.
Besides all the functional testing that is performed on the system, non-functional testing will also be performed during transition which includes: Performance, Load, Stress, and Stability testing. The System Development Manager will coordinate the non-functional testing effort with assistance from the I&T lead. As part of this effort additional tests will be created and automated for regression testing purposes. As part of this effort, a tool similar to "Chaos Monkey" will be used or written to randomly kill instances and services within a running system to simulate unexpected outages.
As part of end-to-end testing, live test users will perform ad hoc testing of the end-to-end running system. This type of testing is used to uncover additional defects not normally encountered by automated or manual scripted testing.
Verification is testing against the L3 System and L4 Subsystem requirements, and consitutes the process of asking "was the system built right". There are three verification approaches used in R1:
- Inspection-- comparison of an item to documentation to confirm compliance with requirements. Inspection is typically used to verify properties that can be confirmed by examination. It may also be used to verify an L3 System requirement as a roll-up of all linked L4 and lower level requirements.
- Demonstration--the action of showing that the requirements are fulfilled through direct evidence or observation of the response of an item of software. Demonstration utilizes a set of selected test activities and inputs to show that the response is correct.
- Test-- a process by which the functionality or performance of an item of software is assessed under specified, controlled conditions that may either be simulated or real. This may require a special test harness and post-test analysis.
A procedure using one of these approaches is assigned to each requirement in R1, and is documented in 2160-00003 R1 Verification Procedures that is an IOC deliverable. Verification is carried out by the I&T team and/or Senior Developers, and is the responsibility of the System Engineer, who delivers a Verification Report for the PRR milestone at the end of Transition.
Validation is testing against the L2 User requirements, and constitutes asking the question "was the right system built". For R1, validation will be carried out throught demonstration of the workflows described in 2130-00005 R1 Functional Design Specification that also is the core of the PMO acceptance procedures that will follow Transition. Validation is the responsibility of the Project Scientist, who delivers a Validation Report at the PRR milestone.
Software development teams deliver their components, services and subsystems in the form of software packages and assemblies of packages. Each software package or assembly has an associated Git source code repository. Packages may have dependencies on other delivered packages that must be specified as part of the package. Each package or assembly delivery is uniquely identified by a set of attributes, including package name and version number. At the end of Transition, the release will constitute a set of packages and assemblies that must be available to the R1 user community. The System Development Manager is responsible for this activity with the assistance of the Senior Developers.
At the end of Transition, a set of documentation of the release is a deliverable at the PRR milestone. This consitutes the following elements:
- User documentation of the capabilities, workflows and user interfaces of the release
- Service documentation using Doxygen that is usable by developers
- Operation doumentation for the use of the Operations Team
The System Engineer and System Development Manager will plan for the acceptance process that follows the PRR milestone. This primarily requires ensuring the the required documentation is provided to the PMO for acceptance.
The System Engineer, System Development Manager and Operations Manager must plan for the transition to operations after acceptance of the CI by the PMO. This includes planning for "hot tixes" or patches to the code once R1 goes live. The transition to operations process is described in 2110-00002 Transition to Operations Plan.
The following links provide details of transition activity on a subsystem by subsystem basis:
- CEI Release 1 Transition
- COI Release 1 Transition
- DM Release 1 Transition
- SA Release 1 Transition
- ITV Release 1 Transition
- UX and AIS Release 1 Transition
During Transition, the ITV team drives the testing of the integrated software on the San Diego Testing Environment. This is comprised of the following:
- A Pair of Computers running VMWare and hosting:
- A Cluster of static VMs hosting Cassandra
- A Static VM hosting the rabbitmq broker
- A Static VM hosting Grails server and the User visible Web Application
- A Static VM hosting Thredds server and entry point to datasets
- A Static VM hosting Nimbus "cloud controller"
- A Pair of Computers running KVM and hosting:
- A set of dynamically launched VMs (via Nimbus) running ION CCs based on the R1 Deployment Configuration https://spreadsheets.google.com/spreadsheet/ccc?key=0AqTqkXCx-C2SdG0wZDNSMXZQZmc5SGdsakZJSGtHdXc&hl=en_US#gid=3