Analytics - avito-tech/Emcee GitHub Wiki
You can configure analytics for global events and also per-job events. Global configuration is set via queue configuration, and job configuration is set via test arg file.
Refer to Graphite page to read more about events being sent.
There is a detailed explanation about how analytics work, as well as what events being sent, in Emcee v12 "Grown Logs Man" release page.
No Analytics - No Problems
If you don't want any analytics, provide an empty JSON object:
{}
Graphite
This is a JSON object. It configures analytics services that Emcee can use to report various events.
To enable reporting to graphite, add graphiteConfiguration
as below:
{
"graphiteConfiguration": {
"socketAddress": "graphite.example.com:2111",
"metricPrefix": "your.own.emcee.graphite.prefix"
}
}
Replace graphite.example.com:2111
with your graphite address and port, and set your own graphite metric prefix instead of your.own.emcee.graphite.prefix
. Read more about Graphite events here.
Statsd
Some metrics can only make sense when they are logged into Statsd due to its aggregation features. Configuration is very similar to Graphite though:
{
"statsdConfiguration": {
"socketAddress": "statsd.example.com:1234",
"metricPrefix": "your.own.emcee.graphite.prefix"
}
}
Kibana
To enable logging to Kibana, add kibanaConfiguration
as below:
{
"kibanaConfiguration": {
"endpoints": [
"endpoint01.kibana.example.com",
"endpoint02.kibana.example.com"
],
"indexPattern": "your-index-pattern"
}
}
indexPattern
usually includes date. E.g., in Python you can generate it like so:
# e.g. 'ios-emcee-test-2021-03-09',
'ios-emcee-test-' + date.today().strftime('%Y-%m-%d'),
Complete Example
Below is an example of a global analytics configuration (defined via queue configuration), but it can be applied for job analytics configuration (defined via test arg file) by simply renaming globalAnalyticsConfiguration
to analyticsConfiguration
.
{
"globalAnalyticsConfiguration": {
"kibanaConfiguration": {
"endpoints": [
"endpoint01.kibana.example.com",
"endpoint02.kibana.example.com"
],
"indexPattern": "ios-emcee-test-2021-03-09"
},
"statsdConfiguration": {
"socketAddress": "statsd.example.com:1234",
"metricPrefix": "your.own.emcee.graphite.prefix"
},
"graphiteConfiguration": {
"socketAddress": "graphite.example.com:2111",
"metricPrefix": "your.own.emcee.graphite.prefix"
}
}
}
Grafana Dashboard Example
Here you can find interactive Grafana dashboard example with Queue metrics.
You can import this example by by json file. Note that graphiteConfiguration.metricPrefix
here is apps.mobile.metrics.awesomeapp
.
Also in this example you need to configure $machine
variable in your Grafana dashboard. Or just replace $machine
by *
.