Loupe Data Flow

Publisher

The Publisher sits between everything attempting to record information with Loupe and all outputs. It maintains the primary queue of messages waiting to be written (in the form of IPackets). It owns the collection of data sinks (called Messengers, similar to Appenders in other log systems but at a lower level to support arbitrary typed data streams). It also owns the current running configuration.

A publisher's lifecycle determines the boundaries of a single Loupe Session - from when it is created until it exits. Two publisher instances can't write to the same session and a single publisher only ever writes to a single session.

Since the publisher is the central object within Loupe it's possible to create multiple Log objects and even create other custom objects that directly work with the publisher for advanced behaviors without causing data issues within Loupe.

Messengers

A messenger is a single sink for data in Loupe. All Loupe data is presented to each messenger and the messenger is responsible for determining if it should write out some or all of the data. Since the publisher itself is asynchronous a messenger can determine if it should be synchronous or async. The API to the publisher anticipates that publishers are largely synchronous but may need to periodically do maintenance or flush data.

When a Loupe client requests a message be written synchronously the publisher will wait for all messengers to confirm the message is written before continuing to the next message, presuming the messenger itself will flush the data if necessary.

The messenger API does not require it but all built-in messengers use their own dedicated thread for work and have a their own independent queue to allow them to proceed at a different pace than the publisher. In most cases they will degrade from being non-blocking to blocking behavior if their queue is full. To simplify implementation it's recommended that new messengers inherit from MessengerBase instead of creating a new implementation from the IMessenger interface. The advantage of doing this is MessengerBase abstracts all of the issues related to multithreading, allowing inheritors to be coded as if they are single threaded.

Messengers can choose to not write out all packet types. For example, the live stream messenger only writes out log message data and skips metrics since they aren't displayed by the live log viewer.

Messengers can chose to not serialize to the Loupe binary format if desirable.

Listeners

Listeners are started by the Publisher and are run in the background to collect information from other sources. This may be based on events (like registering for .NET Framework events), streams (like console), or more active polling (such as performance counters).

The Publisher starts configured listeners asynchronously after the publisher is initialized. They are run until the Publisher is put in shutdown mode.