Demonstration for detecting MySQL Slave Lag - ksgh/dkrb-slave-check GitHub Wiki

It's a fairly common practice with RDBMS's to dedicate writes to a master instance, and read from any number of secondaries or "slaves". It's also fairly common to consume & aggregate data in the background, which could be adding more writes to your data store thus potentially compounding a replication lag issue. If any part of your application is sensitive to eventually consistent data on these slaves, having a mechanism that you can tap into may be beneficial.

Now, asking the database in realtime "how up2date are you" is more overhead and latency (not to mention you'd have to ask this question in each concerned process). If we poll for this information in the background in another "service" and write it to more appropriate key-store we've separated concerns as well as provided a means for any other service (or application) in your stack to access this information if/when its needed.

For instance. If you have a customer-facing front end reading data from secondary (slave) instances, and also at the same time have background processing chewing on data it's certainly possible to cause slave lag to some degree. At this point it might be wise (depending on your business case) to pause background processing until slave lag is back to an acceptable level.

With MySQL, Seconds_Behind_Master is never really accurate, but in most cases it's probably good enough for use in decisions of sorts. If you need more accurate replication status there are other means of going about that which are out of scope here.

I'm going to walk through an approach to this leveraging Docker Swarm on 3 separate t2.micro instances (CentOS) in AWS which should make it easy to follow along and implement yourself if you like. This can also be done on one instance (say in virtualbox) too with just some minor tweaks.

Next

Let's get MySQL setup and Replicating