Alo, AloFlux, and AloDecorator - atleon/atleon GitHub Wiki

Under the hood of AloFlux, the key abstraction that enables Atleon to receive, process, and observe data is Alo.

Users do not typically need to interact with Alo directly, as AloFlux enables defining pipelines purely in terms of the type of data referenced by received Alo elements. Therefore, this page is largely educational in purpose.

In typical use cases, it is required that a message is not abstractly marked "fully processed" at the source from which it came until it is known that the message has been fully processed by whatever pipeline we have defined for it. In Atleon, the concept of marking a message as "fully and successfully processed" is called "positive" acknowledgement, and when used as a verb, Atleon "acknowledges" the message. In contrast, we typically want some functionality in place for when a message is "unable to be successfully processed" due to some fatal error. In Atleon, the termination of a message's processing due to a fatal error is called "negative" acknowledgement, and when used as a verb, Atleon "nacknowledges" the message.

Propagating mechanisms for acknowledgement can be cumbersome in reactive pipelines. Explicitly doing so can make such pipelines less readable and takes away from developers' and readers' abilities to focus on what's being done to the actual data flowing through the pipeline. Atleon provides precisely the abstraction to propagate acknowledgement as data items are transformed in the form of Alo.

Each received message of type T is wrapped as an Alo Functor of type T. That Alo implementation, in addition to referencing T is responsible for propagating what's necessary to positively or negatively acknowledge the message. Alo exposes callbacks for positive vs. negative acknowledgement via getAcknowledger() and getNacknowledger().

In the case of Kafka, for example, Atleon keeps track of the offset for any given received ConsumerRecord. When a message is positively acknowledged, its corresponding offset is marked complete; However, that offset will not be committed until received ConsumerRecord instances from that same partition with earlier offsets have been positively acknowledged as well. Negatively acknowledging any offset, by default, results in emitting the error into the stream, such that at least the corresponding ConsumerRecord will be re-received in the future.

The following animation may help to visualize how acknowledgement of Kafka Records is managed in Atleon to accomplish the At Least Once behavior described above:

As another example, in the case of RabbitMQ, single message acknowledgement is natively supported, so whenever a Delivery is positively acknowledged, it is immediately ack'd back to the RabbitMQ broker(s). Negative acknowledgement for RabbitMQ is configurable, but by default results in emitting the error into the stream. This can be configured to instead "nack" the Delivery, resulting in it either being requeued, dropped entirely, or dead-lettered.

Atleon makes most of the handling of Alo implicit for developers through AloFlux. Much like Project Reactor's Flux, AloFlux allows developers to define logic purely in terms of data items while wrapping those transformations with appropriate operations on Alo.

Alo items can be decorated with extra context in order to support decorative functionalities. This is accomplished through AloDecorator. This interface is how cross-cutting concerns, like metrics and tracing, are implemented in Atleon.

Atleon Receivers allow explicitly providing decorators through AloDecoratorConfig#DECORATOR_TYPES_CONFIG in their ConfigSource. However, if decorators are not explicitly defined, then applicable decorators on the class path are applied using Java's ServiceLoader SPI.