Consumers - peter-mount/opendatabridge GitHub Wiki

Each source topic or queue contains one or more consumers. When a message is received from the source then each consumer receives that message in the order that they are defined under:

For example:

    <topic topic="darwin.pushport-v16">
      <log label="record nre.darwin.pushport-v16"/>
      <graphite metric="feeds.nre.darwin.pushport-v16" metricNameInBody="true"/>
      <bridge routingKey="nre.darwin.pushport-v16" gzip="true"/>
    </topic>

here we have three consumers defined:

  • log will write a message count to the console once a minute
  • graphite will submit a message count to graphite/carbon once every 10 seconds
  • bridge submits the message, with some processing, to a rabbitMQ exchange

For the main configuration see the Configuration page.

bridge

This is obviously the most common consumer as it performs the actual bridge between ActiveMQ and RabbitMQ.

Attribute Required Default Description
exchange optional amq.topic The exchange to publish to
routingKey yes The routingKey to add to every message
gzip optional false If true then all binary messages will be decompressed

graphite

graphite will keep a count of the messages received over a period of time and submits them to graphite over the same RabbitMQ connection.

Note:

  1. This requires that you have an instance of carbon configured to connect to the same RabbitMQ broker.
  2. If carbon.conf has AMQP_METRIC_NAME_IN_BODY set to true then you need to include the metricNameInBody="true" attribute so that the metric name is included in the message body, e.g. "metric value timestamp".

The default period for submitting to graphite/carbon is 10 seconds.

Attribute Required Default Description
metric yes the metric name within Graphite/Carbon
exchange optional graphite the exchange to publish to
metricNameInBody optional false See notes above
period optional 10 The period in seconds between each count being submitted
initialDelay optional period The initial period in seconds before the first count is submitted.

log

log will keep a count of the messages received over a period of time and write those values to the console. The default period for writing to the log is 60 seconds.

This was originally used to submit metrics to Amazon Cloud Watch as that can extract metrics from an application's logs.

Attribute Required Default Description
label yes Text to prefix the message count in the log
min optional 1 The minimum value this monitor will log events. The default is 1 so we don't spam the logs.
period optional 60 The period in seconds between each count being logged
initialDelay optional period The initial period in seconds before the first count is logged.
⚠️ **GitHub.com Fallback** ⚠️