Service: HLS Stream Monitor - EyevinnOSC/community GitHub Wiki

HLS Stream Monitor

HLS Stream Monitor (serviceId eyevinn-hls-monitor) monitors one or more HLS streams for manifest errors and inconsistencies. It is a long-running service instance. License: MIT.

What it can do

  • Detect media sequence counter issues.
  • Detect discontinuity sequence counter issues.
  • Detect stale manifests. The default stale limit is at least 6000 ms and can be configured per monitor.
  • Expose current monitoring status as metrics in OpenMetrics format on GET /metrics, for example for Prometheus and Grafana.
  • Monitor several streams in one monitor, created through the instance's REST API.

Creating an instance

Create the instance with create-service-instance using serviceId: "eyevinn-hls-monitor". The instance name must be alphanumeric and at most 20 characters. It has no dependencies on other OSC services.

{ "name": "mymonitor" }

Wait for the instance with wait-for-service-ready and get its URL with describe-service-instance. The instance serves an API with interactive API docs.

Options

Option Required Type Meaning
name Yes string Name of the instance. Alphanumeric, at most 20 characters.

The service has no other options. The streams to monitor are configured through the API after the instance has started.

Using the instance

Add a monitor and a stream

Create a monitor with POST /monitor on the instance, for example for a live stream from a FAST Channel Engine:

{
  "streams": [
    "https://eyevinn.ce.prod.osaas.io/channels/www/master.m3u8"
  ],
  "stale_limit": 8000,
  "monitor_interval": 3000
}

streams is the list of stream URLs to monitor, stale_limit is the time in milliseconds after which a manifest is considered stale, and monitor_interval is the monitoring interval in milliseconds. The monitor then runs continuously.

Check metrics

GET /metrics returns the current metrics in OpenMetrics format.

Visualize with Prometheus and Grafana

The metrics endpoint can be scraped by Prometheus and shown in Grafana. Instance URLs are behind the OSC authentication gate, so this example puts a Basic Auth adapter in front of the metrics, running in a Web Runner:

% osc create eyevinn-web-runner myadapter \
  -o SourceUrl=https://github.com/EyevinnOSC/basic-auth-adapter \
  -o GitHubToken="{{secrets.ghtoken}}" \
  -o OscAccessToken="{{secrets.osctoken}}"

Create prometheus.yml for a local Prometheus. metrics_path is /<instance-name>/metrics, where the instance name is the name of the HLS Stream Monitor instance; the target is the hostname of the Web Runner running the adapter:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'hls-monitor'
    metrics_path: '/mymonitor/metrics'
    basic_auth:
      username: 'myuser'
      password: 'changeme'
    static_configs:
      - targets: ['<tenant>-myadapter.eyevinn-web-runner.auto.prod-se.osaas.io']

And a docker-compose.yml for Prometheus and Grafana:

version: '3'
services:
  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  grafana:
    image: grafana/grafana
    ports:
      - "3001:3000"
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_INSTALL_PLUGINS=grafana-piechart-panel
    volumes:
      - ./grafana/provisioning:/etc/grafana/provisioning
    depends_on:
      - prometheus

Start with docker-compose up -d, then:

  1. In Prometheus (http://localhost:9090), under Status, Targets, check that the hls-monitor target is UP. If not, check that the adapter's metrics URL (https://<tenant>-myadapter.eyevinn-web-runner.auto.prod-se.osaas.io/mymonitor/metrics) is reachable and that prometheus.yml matches.
  2. In Grafana (http://localhost:3001), add a Prometheus data source with URL http://prometheus:9090.
  3. Import the example dashboard via Dashboards, Import, and select the Prometheus data source. It shows monitor status, error counts and distributions, and stream health metrics.

Resources

⚠️ **GitHub.com Fallback** ⚠️