Creating Event Driven Microservices why,what,how by Andrew

https://youtu.be/ksRCq0BJef8

https://ibm-cloud-architecture.github.io/refarch-eda/introduction/overview/

Event-driven architecture (EDA) is an architecture pattern that promotes the production, detection, consumption of, and reaction to events. This architectural pattern can be applied to the systems that transmit events among loosely coupled software components and services.

The business value of adopting this architecture is that you can easily extend EDA with new components that are ready to produce or consume events that are already present in the overall system. While events are more visible, new business capabilities are addressed, like real time insights. EDA helps also to improve continuous availability of a microservice architecture.

1. Microservices can produce and consume events using publish / subscribe events. ( Have a kafka topic and several producers and consumers. Producer example bulk reader / batch reader / online).

Flow : reader -> router ( uses derivation logic jar determines blue or green route and puts message to adapter microservices which convert message to canonical model and puts message to processor for further processing. In case of any exception life cycle api is called which puts message to a topic and database which is processed by other service. Each service publish events to event source microservices topic. In case of any failure with Event source we can rebuild current state of the system.

To enable file processing from FTP microservices send file path, server details via light weight Json message to subsequent microservices.

Batch or bulk reader subscribes to channel topic and receives message checks size and sends it router via topic, router uses derivation microservices to figure out whether blue or green channel to use and routes accordingly.

2. events are handled by event backbone. Example : Kafka, solace etc.

Event backbone is collect of messaging systems used to receive events. A new microservice can ingest events to other microservice by publishing event to specific microservices topic especially to router which routes messages to either blue or green channel.

Alerts microservices can also be part where it can subscribe and listen process message from single microservice and sent it out email, sms etc.

Analytical microservices can receive message from multiple microservices and process it and proceed.

Unique transaction reference binds all these events together. Possible events wud have been defined and it’s order as well and which microservices consume which event need to be defined.

Say a router can only handle route event, adapter adapter event, processor processor event etc.

Event backbone building block can be event source, stream processing, event archive.

3. data is eventually consistent.

How to start building event driven arch

With a team of few people discuss below and find events, microservices, commands and data.

1. Identify actors, events, data, commands.

Example:

Actors - atm operator, mobile and portal user.

Events - external system event like cancelled, insufficient fund etc or process this transaction. Figure out all events and microservices that will handle this.

To achieve loose coupling, data consistency, efficient queries we need Patterns like - saga, event sourcing, cqrs, db per service.

4. Centralised logging achieved by allowing microservices to publish log directly to Kafka - kibana as logstash is removed to avoid latency.

5. Log compaction for data replication.

event driven architectures for spring developers using Kafka

https://youtu.be/BFcoFu-udQA

Generate events - store events - process events. Use Kafka connect to integrate diverse systems to read data from s3, Mongo etc Kafka streams to process data

Capture and publish db change events to Kafka using maxwells and debezium (https://debezium.io/) its built on top of Kafka. https://maxwells-daemon.io/