Invalid Version In Content Type of OpenMetrics Endpoints - promregator/promregator GitHub Wiki


With the advent of the OpenMetrics format, a new Content Type for scraping metrics has been introduced to the Prometheus world. In old times, there was only the Content Type

text/plain; version=0.0.4; charset=utf-8

whereas the new OpenMetrics format has the Content Type

application/openmetrics-text; version=1.0.0; charset=utf-8

Note the version number here reads 1.0.0. The OpenMetrics Specification does not foresee any other version identifier so far - in particular there is no such version which has a major version number of zero (0).

The Problem

Apparently, due to an initial bug at Prometheus, there are some clients out there, which think that they should return a content type which contains the version identifier of 0.0.1. That is wrong.

Reaction of Promregator

Promregator tries to remain fault-tolerant in this direction: It will write a warning into its logs reading along the lines of

The implementation at https://example.bogus and instance 5 returned an invalid content type 
by specifying an invalid version identifier 0.0.42 of the OpenMetric format. Promregator 
is guessing that you mean version 1.0.0. Please fix your Prometheus client library!

It will assume that the client has meant to send version identifier 1.0.0. This may cause issues in the future, if a new version of the specification is provided.

What You Should Do

  1. Check the implementation of the Prometheus' client library the scraping target uses. Perhaps there is already a newer version available that addresses this issue. If that is the case, upgrade your client library to ensure that you are compliant to the specification.
  2. In case that there is no fixed version available, check the bug tracker / issue management tool of your Prometheus' client library. Perhaps the issue is already known but not fixed yet. Report there that you are also affected from this issue.
  3. In case the issue is not known to the Prometheus' client library yet, create a new bug report for them. Refer to this page to help you explaining what is wrong. Ask them to fix it on next occasion.

How to Prevent Log Flooding

Until the issue is fixed at your Prometheus' client library and you were able bump to that version, you may disable the warning in your Promregator's log files by setting the log level of logger named org.cloudfoundry.promregator.fetcher.CFMetricsFetcher.wrongVersion to ERROR.

Fixes in Client Libraries

Further References