Real-Time Slurm-Kubernetes Integration

Overview

Change Data Control (CDC) is an approach that enables real-time Slurm-HPC integration via the Debezium MySqlConnector which leverages the MySQL binlog that is normally used for replication and recovery. The Debezium MySQL connector reads the binlog, produces change events for row-level INSERT, UPDATE, and DELETE operations, and emits the change events to Kafka topics.

Downstream applications can execute a variety of use cases on the Kafka topics containing messages written by the MySqlConnector.