Correlate Metrics and Logs - monicasarbu/talks GitHub Wiki

In the world of containers and microservices, where your infrastructure consists of thousands of containers that are changing continuously, monitoring your infrastructure becomes a challenge.

At the beginning, you collect the logs from all your servers to help you understand when there is a failure in your system, but logs are not always available, think of when the service is down. To prevent a failure, you would need to monitor the status of your services, and the health of the server where the service is running. Ideally, you would like to use the same system for collecting logs and metrics in order to correlate them together, and if the number of your servers grows 10 times, 1000 times, the system should be easily scalable.

This talk shows you how to use open source tools and in particular the Elastic Beats to offer a broad visibility into your network by collecting different kinds of operational data from all your services into a central point in Elasticsearch, and then build Dashboards with Kibana.