compared with
Current by Emily Hahn
on May 16, 2014 12:03.

Key
This line was removed.
This word was removed. This word was added.
This line was added.

Changes (15)

View Page History
h2. Multiple Harvester / Parser Dataset Drivers

There may be some instances where multiple harvesters / parsers are needed for a single driver.  In this case a different harvester configuration is needed to keep track of the different harvesters.  Each harvester / parser pair is given a data source ID to link it to the corresponding harvester and parser combination, where each sub-dictionary identified by the ID will match the single directory harvester and parser configuration.  The harvester and parser configs are each indexed separately by the data source key due to different columns in the pre-load spreadsheet.

{code}
config = {

data_source_key_telemetered: 'harvester': {


'harvester': data_source_key_telemetered: {

'directory': '/tmp/path1,

'pattern': '*.type1.log',
}


'parser': {} data_source_key_recovered: {
}

data_source_key_recovered: {


'harvester': {

'directory': '/tmp/path2',

'pattern': '*.type2.log', }
}


'parser': {}
}

'parser': {
data_source_key_telemetered: {},
data_source_key_recovered: {}
}
}