Observability - OpenIDC/mod_auth_openidc GitHub Wiki

Since version 2.4.15 mod_auth_openidc allows for collection and retrieval of a number of metrics that are designed to be used in systems monitoring and alerting tools such as Prometheus. Version 2.4.15.1 reworked that implementation and fixed an overlap in the Prometheus output, so 2.4.15.1 is the recommended minimum.

Metrics

The OIDCMetricsData primitive configures the top level classes of metrics that will be collected.

The second column of each listing below is the version of mod_auth_openidc in which that class, timer or counter was introduced.

Classes

authtype        2.4.15     Request counter, overall and per AuthType: openid-connect, oauth20 and auth-openidc.
authn           2.4.15     Authentication request creation and response processing.
authz           2.4.15     Authorization errors per OIDCUnAutzAction (per Require statement, not overall).
require.claim   2.4.15     Match/failure count of Require claim directives (per Require statement, not overall).
claim           2.4.16.8   Claims per value.
provider        2.4.15     Requests to the provider [token, userinfo, metadata] endpoints.
session         2.4.15     Existing session processing.
cache           2.4.15     Cache read/write timings and errors.
logout          2.4.20.1   Front- and back-channel logout processing.
redirect_uri    2.4.15     Requests to the Redirect URI, per type.
content         2.4.15     Requests to the content handler, per type of request: info, metrics, jwks, etc.

Metrics are recorded and published per defined virtual host in Apache. Two types of metrics exists:

  • timers: recorded as a histogram: count, sum and 11 buckets (0.1, 0.5, 1, 5, 10, 50, 100, 500, 1000 and 5000 ms, plus +Inf)
  • counters: recorded as a cumulative total

Timers

authtype.handler     2.4.15     Histogram of the overall authn+authz processing time.
authn.request        2.4.15     Histogram of authentication requests.
authn.response       2.4.15     Histogram of authentication responses.
session.valid        2.4.15     Histogram of successfully validated existing sessions.
provider.metadata    2.4.15     Histogram of provider discovery document requests.
provider.token       2.4.15     Histogram of provider token requests.
provider.refresh     2.4.15     Histogram of provider refresh token requests.
provider.userinfo    2.4.15     Histogram of provider userinfo requests.
provider.jwks        2.4.20.1   Histogram of provider JWKs document requests.
provider.par         2.4.20.1   Histogram of provider pushed authorization requests.
cache.read           2.4.15     Histogram of cache read requests.
cache.write          2.4.15     Histogram of cache write requests.

Counters

authtype.mod_auth_openidc              2.4.15     Nr of requests handled by mod_auth_openidc.
authtype.openid-connect                2.4.15     Nr of requests handled by AuthType openid-connect.
authtype.oauth20                       2.4.15     Nr of requests handled by AuthType oauth20.
authtype.auth-openidc                  2.4.15     Nr of requests handled by AuthType auth-openidc.
authtype.declined                      2.4.15     Nr of requests not handled by mod_auth_openidc.
authn.request.error.url                2.4.15     Nr of errors matching the incoming request URL against the configuration.
authn.response.error.state-mismatch    2.4.15     Nr of state mismatch errors in authentication responses.
authn.response.error.state-expired     2.4.15     Nr of state expired errors in authentication responses.
authn.response.error.provider          2.4.15     Nr of errors returned by the provider in authentication responses.
authn.response.error.protocol          2.4.15     Nr of protocol errors handling authentication responses.
authn.response.error.remote-user       2.4.15     Nr of errors identifying the remote user based on provided claims.
authz.action.auth                      2.4.15     Nr of step-up authentication requests.
authz.action.401                       2.4.15     Nr of 401 authorization errors.
authz.action.403                       2.4.15     Nr of 403 authorization errors.
authz.action.302                       2.4.15     Nr of 302 authorization errors.
authz.error.oauth20                    2.4.15     Nr of AuthType oauth20 (401) authorization errors.
require.claim.match.<claim-value>      2.4.15     Nr of (per-) Require claim authorization matches.
require.claim.error.<claim-value>      2.4.15     Nr of (per-) Require claim authorization errors.
claim.id_token.<claim-name>            2.4.16.8   Nr of claim values in the ID Token.
claim.userinfo.<claim-name>            2.4.16.8   Nr of claim values returned from the Userinfo Endpoint.
provider.metadata.error                2.4.15     Nr of errors retrieving a provider discovery document.
provider.token.error                   2.4.15     Nr of errors making a token request to a provider.
provider.refresh.error                 2.4.15     Nr of errors refreshing the access token at the token endpoint.
provider.userinfo.error                2.4.15     Nr of errors calling a provider userinfo endpoint.
provider.http.connect.error.<string>   2.4.15     Nr of (libcurl) provider/network connectivity errors.
provider.http.response.code.<code>     2.4.15     Nr of HTTP response code calling a provider endpoint.
provider.jwks.error                    2.4.20.1   Nr of errors retrieving a provider JWKs document.
provider.par.error                     2.4.20.1   Nr of errors making a pushed authorization request to a provider.
provider.registration.error            2.4.20.1   Nr of errors registering a client dynamically at a provider.
provider.revocation.error              2.4.20.1   Nr of errors revoking a token at a provider revocation endpoint.
provider.dpop.retry                    2.4.20.1   Nr of provider calls retried with a fresh DPoP nonce.
session.error.cookie-domain            2.4.15     Nr of cookie domain validation errors for existing sessions.
session.error.expired                  2.4.15     Nr of sessions that exceeded the maximum duration.
session.error.refresh-access-token     2.4.15     Nr of errors refreshing the access token before expiry in existing sessions.
session.error.refresh-user-info        2.4.15     Nr of errors refreshing claims from the userinfo endpoint in existing sessions.
session.error.general                  2.4.15     Nr of existing sessions that failed validation.
session.fallback-cookie                2.4.20.1   Nr of sessions stored in a browser cookie after a session cache write failure.
cache.cache.error                      2.4.15     Nr of cache read/write errors.
cache.cache.retry                      2.4.20.1   Nr of cache operations retried after a backend failure.
logout.backchannel                     2.4.20.1   Nr of back-channel logout requests processed successfully.
logout.backchannel.error               2.4.20.1   Nr of back-channel logout requests rejected.
redirect_uri.authn.response.redirect   2.4.15     Nr of authentication responses received in a redirect.
redirect_uri.authn.response.post       2.4.15     Nr of authentication responses received in a HTTP POST.
redirect_uri.authn.response.implicit   2.4.15     Nr of (presumed) implicit authentication responses to the redirect URI.
redirect_uri.discovery.response        2.4.15     Nr of discovery responses to the redirect URI.
redirect_uri.request.logout            2.4.15     Nr of logout requests to the redirect URI.
redirect_uri.request.jwks              2.4.15     Nr of JWKs retrieval requests to the redirect URI.
redirect_uri.request.session           2.4.15     Nr of session management requests to the redirect URI.
redirect_uri.request.refresh           2.4.15     Nr of refresh access token requests to the redirect URI.
redirect_uri.request.request_uri       2.4.15     Nr of Request URI calls to the redirect URI.
redirect_uri.request.remove_at_cache   2.4.15     Nr of access token cache removal requests to the redirect URI.
redirect_uri.request.revoke_session    2.4.15     Nr of revoke session requests to the redirect URI.
redirect_uri.request.info              2.4.15     Nr of info hook requests to the redirect URI.
redirect_uri.request.dpop              2.4.16     Nr of DPoP proof requests to the redirect URI.
redirect_uri.error.provider            2.4.15     Nr of provider authentication response errors received at the redirect URI.
redirect_uri.error.invalid             2.4.15     Nr of invalid requests to the redirect URI.
content.request.declined               2.4.15     Nr of requests declined by the content handler.
content.request.info                   2.4.15     Nr of info hook requests to the content handler.
content.request.dpop                   2.4.16     Nr of DPoP proof requests to the content handler.
content.request.jwks                   2.4.15     Nr of JWKs requests to the content handler.
content.request.discovery              2.4.15     Nr of discovery requests to the content handler.
content.request.post-preserve          2.4.15     Nr of HTTP POST preservation requests to the content handler.
content.request.authn-post             2.4.16.11  Nr of HTTP POST authentication requests to the content handler.
content.request.unknown                2.4.15     Nr of unknown requests to the content handler.

Collection

The measurements are recorded per virtual host in a local in-memory hash table for performance reasons. The local data is flushed to shared memory in a dedicated thread that runs every 5 seconds per virtual host. Notice that any data consumed on OIDCMetricsPublish is therefore max 5 seconds behind. The shared memory is shared between all of the virtual hosts configured on the same Apache instance. Retrieval of the data on OIDCMetricsPublish returns and resets the counters/timings for all virtual hosts.

Retrieval

The OIDCMetricsPublish primitive configures where the data is published. This URL may contain sensitive data and can be protected by any available Apache mechanism (including e.g. AuthType oauth20). This can be done as part of the virtual host that runs mod_auth_openidc but one may also define a separate virtual host in the same Apache server to publish the data, e.g.:

<VirtualHost 192.168.0.200:8080>
  OIDCMetricsPublish  /metrics
  Require ip 192.168
</VirtualHost>

Data can be retrieved in a number of formats (see below), using the format query parameter. The format-specific default setting whether or not counters/timings are reset on retrieval can be overruled using the parameter reset with values true|false.

Formats

The URL configured in OIDCMetricsPublish returns the collected metrics data in a specified format, passed in the format=<value> query parameter. The following formats are supported, listed with their reset setting default:

  • format=prometheus&reset=false (default)
    the Prometheus text-based exposition format
  • format=json&reset=true
    JSON a custom extended JSON format with descriptions and labels
  • format=status&reset=false
    Status a short text response containing a single OK line, meant to be using in monitoring/uptime tools such as Nagios, optionally extended to OK: <counter-value> by retrieving a specific counter in the provided server_name=<vhost>&counter=<metric-name>[&value=<value>][&name=<name>] parameters.
  • format=internal&reset=false
    Internal a terse internal JSON-based representation of the data that is stored in shared memory, without textual labels or descriptions, only used for debugging
Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️