|This page is for developers new to OOI CI, and attempts to collect information to get you started and productive. It's a wiki, so please feel free to edit it as you learn.|
- What this page is
- Get Connected
- Get Help
- Set up Git and version control
- Prepare Directory Structure
- Install Code Dependencies
- Set up Python development environment
- Get Project Code
- Working with GitHub/Git
- Using Gerrit for code reviews
- Big Picture
If you have not received a login as part of the sign-up process, send email to helpdesk-at-oceanobservatories.org to get a login. This process will give you access to the Confluence wiki, Jira, FishEye and the other Atlassian tools via an LDAP server. It will also subscribe you to the firstname.lastname@example.org (all CI development related announcements) and email@example.com (CI developer topics) mailing lists and other important mailing lists for your subsystem or team.
We run an XMPP/Jabber chat server (an ejabberd daemon), connected to our LDAP server. You should be able to join as soon as you have a Confluence login. Instructions for clients are at Jabber client setup.
Mac users should use Messages or Adium. On Windows, Pidgin is a very usable client.
We use GoogleDoc for a lot of collaboration between the teams, such as to work on iteration task lists and to draw designs. You need a Google/Gmail login. Send the login to Tim A/Michael and ask to be added to the OOI-CI shared folder. This gives you instantaneous access to many shared documents and any new document will automatically be shared with you. Don't explicitly manage GoogleDoc sharing settings unless you really have to.
We use the Google Calendar to schedule shared meeting times. Once you have a Google ID, ask Paul to be added to the calendars for the relevant teams (CI developers, your subsystem etc).
You can share your personal calendar with others if you so wish to make it easier for them to see your schedule.
For all support issues for underlying tools and computer infrastructure, send email to helpdesk-at-oceanobservatories.org. This will result in a Jira support issue being generated, using your subject as the summary (please make it issue-specific) and the body of your text as the issue. The helpdesk account should be heavily monitored by the Operations support team. Contact the System Development Manager or Operations Manager if you do not see a timely response.
For issues with the actual CI developed software, send email to the ciswdev-at-lists.oceanobservatories.org mailing list, or to the Integration Engineer (jachen-at-ucsd.edu) if you don't want to bother the entire development team with it.
- Create a free GitHub ID, if you don't already have one, at github.com.
- Generate an SSH key on your laptop using
Do not enter a filename, just hit enter when it asks for a file to save the key. GitHub expects the default file name.
- Edit the ~/.gitconfig and change your full name (such as John Doe - please use full name and capitalization) and email address. This file may not exist; create it if it doesn't.
- Upload your SSH public key (located at ~/.ssh/id_dsa.pub) following instructions on GitHub, if you have not done so.
- You are set to fork any repo at OOI-CI source code repositories on GitHub. You can submit a pull request when you have a code delivery.
- In general, the fork and pull request model is preferred so your code can be properly reviewed by your subsystem lead before merged. Your subsystem lead can also grant you direct push access if needs be.
A useful utility is the GIT tab completion script for bash. Install via:
and add the following to your .profile or .bash_profile:
Recommended GIT reading:
- Git is Simpler
- Git from the Bottom Up by John Wiegley. This is a detailed reference book
- Github workflow and pull requests
- Excellent git article on using github workflow plus doing rebase instead of merge
YOUR_DEV_DIR will be ~/Documents on Mac, anything you like on Linux.
Execute the following commands in the terminal:
The directory structure should look like as follows. Modify/refine as you please.
To create all the directories at once, execute the following command from YOUR_DEV_DIR:
When building or developing software for the OOI Integrated Observatory Network, a number of basic dependencies need to be installed. Follow the steps on this page on Installing dependencies. In general, if you are using a Mac, install git first and then use homebrew for all your package management. You can even reinstall git using homebrew after you have done the initial setup.
Specific installation instructions:
- See here for detailed installation instructions on Ububtu Linux.
- See here for installation tips for Mac OX 10.8 Mountain Lion.
- See here for Installation on Mac OS X 10.9 Mavericks
|See Python and Virtualenv Mac Troubleshooting in case of installation troubles|
On Ubuntu and Linux:
Add the following to your shell profile; the profile you need to edit will vary depending on your OS:
- Mac: ~/.profile
- Ubuntu: ~/.bashrc
- Linux (general): ~/.bash_profile
Add the following lines:
Changes will be automatically applied at the start of your next shell session, but for now, update your current shell with the changes:
Here are some useful Bash Profile settings dot_profile
All of the following editors are currently in use, and are all acceptable:
- Pycharm, which has VIM/Emacs modes for those who are so inclined. The group now has an open-source license for PyCharm, so it's free to use.
- Vim (tutorial 1, tutorial 2)
- Komodo. Note there is also a free editor only version with Komodo. Use whatever makes you happy.
- Emacs (Python Tutorial)
- Netbeans (Python Tutorial)
- Eclipse (see Eclipse as a Python IDE by BrianF)
- Geany (X11, GTK2, cross-platform).
- Wing IDE is pretty clean, snappy, X11-based, and cross-platform with a decent debugger...but is python only.
The OOI Network system's code is primarily located in the coi-services repository on GitHub. It pulls in several GIT submodules automatically. Please follow the README instructions in here:
Please refer to the following pages for additional details:
- ION R2 Documentation - main page for OOI Network Release 2 documentations. Contains some of the pages linked below.
- R2 Container Use Guide
- R2 Service Implementation Guide
- Code Conventions
- R2 Resource Development Guide
- R2 Service Testing Guide
- Pyon README
- ION Architecture and Design
Git clone the https://github.com/ooici/ion-ux repository into your Dev directory.
Please refer to the installation notes in README.txt on Github.
If you have any installation or execution troubles, please see here for potential help. Please add any solutions to problems you have solved to the documentation and make sure to make the integration team aware of any documentation issues:
OOI-CI is hosting source code in GitHub code repositories. Here's the original migration steps sent to the developer when we migrated our in-house Gitosis repository late 2011. Not everything is relevant to a new developer, but in it you will find useful git commands and collaboration workflows as you work day-to-day with GitHub/Git.
|OOI repositories rely on GIT submodules. A submodule defines a dependency between two code repositories. In OOI, the container and its services depend on definition YAML files which define objects and services. Pyon and coi_services each have submodule links to the ion-definitions repository, which which must be kept up to date in each of the parent repositories.|
In the bash shell, add the following to your .profile to get a git command prompt such as "user@host ~/code/coi-services (master)$". Modify as you like.
A git hook is a script that executes during various points of using git. Some simple hooks have been written to help automate dealing with submodules for most people. See the steps here:http://blog.chaitanyagupta.com/2009/08/couple-of-hooks-to-make-life-easy-with.html
They do require an initial setup. Simple instructions:
Clone this repository:
Use the provided install script:
The install script does the following (you can also do it manually):
Now, when checking out a branch, pulling, merging etc, git will prompt you to automatically update
if it notices a change to the commit that your supermodule points to.
The pre-commit script is so you don't forget to push changes to the submodule BEFORE you push changes
to the supermodule.
If you have more than one remote defined for git you will see messages like this where your git hook fails because it does not know how to fetch the latest ion-definitions:
Here are some customized git hooks that fetch from all submodule remotes when you run checkout or merge:
Customized git hooks
Please see Setting up and Using GerritHub for Code Reviews for instructions on setting up and using GerritHub for code reviews.
Please see the Overview for New Developers for more information on the project, the CI technology choices, and the engineering processes we use. The CIAD AV Glossary may also be useful as you get up to speed.
Page: EC2 Development Setup
Page: General Development Best Practices
Page: Git Best Practices for OOI-CI
Page: Gotchas with inlineCallbacks, yield and returnValue
Page: Gotchas with messaging
Page: Gotchas with Python development
Page: How to do a code review using Crucible
Page: Installation on Mac OS X 10.8 Mountain Lion
Page: Installation on Mac OS X 10.9 Mavericks
Page: Installing dependencies
Page: Installing dependencies on Ubuntu Linux
Page: OOICI Java Projects
Page: Procedures to Edit the Architecture and Design Document
Page: Python and Virtualenv Mac Troubleshooting
Page: Setting up and Using GerritHub for Code Reviews
Page: Troubleshooting OOINet and the developer environment
Page: VirtualBox image
Page: Working with Pycharm