Thursday Aug 12, 2010
- Review Figure 1 from the original CI DMAC
- Place the discussion at Rutgers relative to that diagram
- what we discussed in detail
- how does Chris' document relate
- Concerns about IOOS' team role in this first year of integration
- Chris to walk the team through the EOI Release 1 Elaboration Plan
- Discuss the EOI LCO Review deliverable
- Original goal was to use CI infrastructure of release 1 to acquire data streams coming from different sources that were needed by some modelers to carry out their processes.
- Diagram 1 in section 2 of DMAC document illustrates this with notion of NOAA port, NDBC, etc. through IDD and THREDDS format. Then in researcher's environment, provide way to integrate into this system.
- The concept resonates. The labels on the arrows of the figure don't resonate---the specific mechanisms that are now available from IOOS data providers into CI.
- At a conceptual level, diagram shows the relations between sources, modelers, and CI. The actual sources and mechanisms were all illustrative to make it concrete. We were not intending to be limited to this particular set.
- Chris' document shoudl reflect 2 major things out of Rutger's meeting (context: 3-5 modelers chosen in year 1 to work out this pipeline, to check it out before going to larger community (in year 2); focused on John W as primary, stretching to 3 then 5 to control the confusion as we find pace. Rutgers meeting represented chance to find the process and products with the primary customer. Chri's picture represents the detailed work there to understand that world.
- Explicit map of where to execute, and agreement with John's team on the process.
- All the products that John is using, by type and measurement, then by source. Data Inputs as Currently Implemented in Rutgers (ROMS ESPRESSO) Workflow Scripts: http://www.oceanobservatories.org/spaces/download/attachments/22220584/Data+Sources+-+Rutgers.docx?version=1
- Meeting notes are at http://etherpad.ooici.org/ioosooi, all materials will be linked from Confluence page (at TBD)
- Tom Auer from UX development team (Applied Science Associates) recorded the meeting.
- IOOS Team role (concerns)
- Jeff is looking for way to bring sources through favored IOOS mechanisms. CI will give NetCDF/CF common data model. Original idea was to use everything as is. Goal is not to impact publisher; given a choice, we'll balance political and engineering capacity.
- IOOS has established gridded data services, and SOS services. Gridded data good to go. If there is in situ data available both ways, welcome to connect directly to DAP. But for NDBC buoy type data, most up to date is SOS. Note task ID 4.2 in Chris' document.
- Agreement these are opportunities we are looking for. These sources represent John W's current approach. Classes of preparation: (1) Provenance analysis (Task 2). (2) Pursue opportunity to ingest new copies of those sources (Task 3), perhaps leading to new sources.
- Suppose there are new sources coming into existence. OOI CI could provide notification mechanism as they come into existence. (Listening to data catalogs, not data sources themselves) Uses IOOS catalog as community catalog, envisions catalogs talking to each other.
- Decision-maker as to correct source lies with John W, correct provider decision lies with IOOS. Thought is to use OOI Ci as lever, but CI could get it through IOOS. Rich leads process as John's agent, done with IOOS to figure out right places. CI has weaker say, but worth listening to for economy's sake. Therre may be sources Wilkins wants ... but for year 1, we should focus on existing sources. Given available time, we'll start on sources of interest. No issues with (broader? IOOS-centric?) approach as long as timeliness requirements for data are met.
- OOI is leveraging Rich and Chris, but IOOS represents a key role (to reinforce within your community of sources your strategy). CI doesn't want to go at this one by one, CI wants to reinforce IOOS interests. OOI CI is a consumer of IOOS. We're also a distributor or channel for IOOS. Also working on architectural elements that IOOS is not -- deeper part of relationship. IOOS is working with community on distribution of measurement, OOI CI focused on distribution mechanism. OOI CI is one method of distribution.
- This conversation merits deeper discussion to ensure understanding of distinctions, which plays in iteration 2. More modest objectives in IOOS reflecting more modest endowment. But IOOS contribution is much greater than just the development of specific tools, it includes a huge community engagement process that IOOS is driving.
- How to get involved? Much more what is strategic approach to the tasks. Can we chat briefly about this? Let's plan on doing so through the process. For now, need to focus on the materials that will be presented at LCA.
- Description of Elaboration Plan
- overview goals
- hope to provide them eventually the ability to use OOI CI resources to follow up when some data sets not available, in prioritized way.
- Where to start with LCO presentation? Not sure if time now.
- Data sifting - qualifying producers and limiting to those within our "tolerance" levels. Rating of producers.
- Data sources that are valuable to the community as a whole should take precedence over those that are particular to John W's workflows.
- How do we deal with duplicate data coming over different "streams"/providers.