Telegraf Operator

Overview
Prerequisites
Set Up
1. Installation
2. Configuration
Installation
Configuration
Troubleshooting
Additional References

Overview

Telegraf Operator is an application designed to create and manage individual Telegraf instances in Kubernetes clusters. Use it to collect, process and aggregate metrics from applications in your Kubernetes cluster and send them to Coralogix.

Once installed, Telegraf Operator watches for pods deployed with a specific set of pod annotations. The advantage of using Telegraf Operator is that you only have to define the input plugin configuration for Telegraf when creating the pod annotations. Telegraf Operator then sets the configuration for the entire cluster, avoiding the need to configure a metrics destination when deploying applications.

This guide shows you how to run Telegraf Operator in Kubernetes to export your data to Coralogix.

Prerequisites

Sign up for a Coralogix account and access your private key.
Install an open-source text editor such as Visual Studio Code, Atom, Notepad++ or Sublime Text.
Install Kubernetes. This should include installation of the command-line tool kubectl, designed to operate on your Kubernetes cluster.
Install and configure Helm. We suggest you use this guide to familiarize yourself with the basics of using Helm to manage packages on your Kubernetes cluster.

Set Up

Installation

1. Add the Influxdata Helm Chart Repository

Using your text editor, open a new terminal and install the Influxdata Helm chart repository:

helm repo add influxdata https://helm.influxdata.com/

helm repo update

2. Install Telegraf Operator

Update the Helm chart from the added repository:

helm upgrade --install telegraf-operator influxdata/telegraf-operator -n <namespace>

Configuration

1. Create YAML-Formatted Override File

Telegraf Operator allows you to configure how Telegraf sidecars send data using classes. You can create or modify existing classes using Helm values file, as shown in the example below.

In order to send your data to Coralogix, you are required to declare particular variables into your template:

service_address: In order to send metrics to Coralogix, you will need to include your Coralogix endpoint, as displayed in the table below, which derives from the cluster URL associated with your Coralogix account.

Region **Metrics Endpoint **

USA1 otel-metrics.coralogix.us:443

APAC1 otel-metrics.coralogix.in:443

APAC2 otel-metrics.coralogixsg.com:443

EUROPE1 otel-metrics.coralogix.com:443

EUROPE2 otel-metrics.eu2.coralogix.com:443

Region	Metrics Endpoint
USA1	otel-metrics.coralogix.us:443
APAC1	otel-metrics.coralogix.in:443
APAC2	otel-metrics.coralogixsg.com:443
EUROPE1	otel-metrics.coralogix.com:443
EUROPE2	otel-metrics.eu2.coralogix.com:443

private_key: Access your Coralogix private key.
application and subsystem: Choose the Application and Subsystem names that suit you or keep the default values provided in the example. Find out more about the importance of organizing your data with Coralogix using Application and Subsystem names.

image:
 # 
  sidecarImage: "docker.io/library/telegraf:1.24.2"
classes:
  secretName: "telegraf-operator-classes"
  default: "infra"
  data:
    infra: |
      [[outputs.opentelemetry]]
        service_address = "<yourclusterURL>:443"
        compression = "gzip"
        [outputs.opentelemetry.coralogix]
          private_key = "<private_key>"
          application = "$NAMESPACE"
          subsystem = "$HOSTNAME"
      [global_tags]
        env = "ci"
        hostname = "$HOSTNAME"
        nodename = "$NODENAME"
        namespace = "$NAMESPACE"
        type = "infra"

Save this file.

2. Apply the Configuration

Apply the configuration by running the following command:

helm upgrade --install telegraf-operator influxdata/telegraf-operator --values values.yaml -n <namespace>

Validation

To test your configuration, deploy a Redis instance with Telegraf Operator annotations.

Create a pod configuration file redis.yml with the following data:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  selector:
    matchLabels:
      app: redis
  serviceName: redis
  template:
    metadata:
      labels:
        app: redis
      annotations:
        telegraf.influxdata.com/inputs: |+
          [[inputs.redis]]
            servers = ["tcp://localhost:6379"]
        telegraf.influxdata.com/class: infra
        telegraf.influxdata.com/env-fieldref-NAMESPACE: metadata.namespace
    spec:
      containers:
      - name: redis
        image: redis:alpine3.16

Run the pod by executing:

kubectl apply -f redis.yml -n <namespace>

The pod should contain 2 containers: redis and telegraf. Verify this by executing:

% kubectl describe pod -n coralogix redis-0

Here is an output example:

Name:             redis-0
Namespace:        coralogix
Priority:         0
Service Account:  default
Node:             minikube/192.168.49.2
Start Time:       Thu, 27 Oct 2022 08:48:10 +0200
Labels:           app=redis
                  controller-revision-hash=redis-75fd658465
                  statefulset.kubernetes.io/pod-name=redis-0
Annotations:      telegraf.influxdata.com/class: infra
                  telegraf.influxdata.com/env-fieldref-NAMESPACE: metadata.namespace
                  telegraf.influxdata.com/inputs:
                    [[inputs.redis]]
                      servers = ["tcp://localhost:6379"]
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    StatefulSet/redis
Containers:
  redis:
    Container ID:   
    Image:          redis:alpine3.16
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z5kkv (ro)
  telegraf:
    Container ID:  
    Image:         docker.io/library/telegraf:1.24.2
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Command:
      telegraf
      --config
      /etc/telegraf/telegraf.conf
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  200Mi
    Requests:
      cpu:     10m
      memory:  10Mi
    Environment:
      NODENAME:    (v1:spec.nodeName)
      NAMESPACE:  coralogix (v1:metadata.namespace)
    Mounts:
      /etc/telegraf from telegraf-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z5kkv (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  kube-api-access-z5kkv:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
  telegraf-config:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  telegraf-config-redis-0
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  4s    default-scheduler  Successfully assigned coralogix/redis-0 to minikube
  Normal  Pulling    4s    kubelet            Pulling image "redis:alpine3.16"
  Normal  Pulled     2s    kubelet            Successfully pulled image "redis:alpine3.16" in 2.263405407s
  Normal  Created    2s    kubelet            Created container redis
  Normal  Started    2s    kubelet            Started container redis
  Normal  Pulling    2s    kubelet            Pulling image "docker.io/library/telegraf:1.24.2"

Verify that the pod is running:

% kubectl describe pod -n coralogix redis-0

Example of expected output:

Name:             redis-0
Namespace:        coralogix
Priority:         0
Service Account:  default
Node:             minikube/192.168.49.2
Start Time:       Thu, 27 Oct 2022 08:48:10 +0200
Labels:           app=redis
                  controller-revision-hash=redis-75fd658465
                  statefulset.kubernetes.io/pod-name=redis-0
Annotations:      telegraf.influxdata.com/class: infra
                  telegraf.influxdata.com/env-fieldref-NAMESPACE: metadata.namespace
                  telegraf.influxdata.com/inputs:
                    [[inputs.redis]]
                      servers = ["tcp://localhost:6379"]
Status:           Running
IP:               172.17.0.9
IPs:
  IP:           172.17.0.9
Controlled By:  StatefulSet/redis
Containers:
  redis:
    Container ID:   docker://3f2318a0fde95f12f71b37744ed8d7dd83e1415026d57f72cbd9f046f06c857b
    Image:          redis:alpine3.16
    Image ID:       docker-pullable://redis@sha256:40b02b7a48829317e973114d07968d28eaaf75ec6b80ddef20f3999238aad7c8
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 27 Oct 2022 08:48:13 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z5kkv (ro)
  telegraf:
    Container ID:  docker://e23923a3bfa8a63898e05310a454265b83efbcc9d53552315aa10a86342fab6e
    Image:         docker.io/library/telegraf:1.24.2
    Image ID:      docker-pullable://telegraf@sha256:eb1a6c7cfb61e6deb727830ccedbba4844fdc3d2f6544f08a66bafc70423ff70
    Port:          <none>
    Host Port:     <none>
    Command:
      telegraf
      --config
      /etc/telegraf/telegraf.conf
    State:          Running
      Started:      Thu, 27 Oct 2022 08:48:53 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  200Mi
    Requests:
      cpu:     10m
      memory:  10Mi
    Environment:
      NODENAME:    (v1:spec.nodeName)
      NAMESPACE:  coralogix (v1:metadata.namespace)
    Mounts:
      /etc/telegraf from telegraf-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z5kkv (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-z5kkv:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
  telegraf-config:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  telegraf-config-redis-0
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  5m44s  default-scheduler  Successfully assigned coralogix/redis-0 to minikube
  Normal  Pulling    5m44s  kubelet            Pulling image "redis:alpine3.16"
  Normal  Pulled     5m42s  kubelet            Successfully pulled image "redis:alpine3.16" in 2.263405407s
  Normal  Created    5m42s  kubelet            Created container redis
  Normal  Started    5m42s  kubelet            Started container redis
  Normal  Pulling    5m42s  kubelet            Pulling image "docker.io/library/telegraf:1.24.2"
  Normal  Pulled     5m3s   kubelet            Successfully pulled image "docker.io/library/telegraf:1.24.2" in 39.115955049s
  Normal  Created    5m3s   kubelet            Created container telegraf
  Normal  Started    5m2s   kubelet            Started container telegraf

Access your Coralogix - Grafana dashboard to query the monitored targets using PromQL. Explore > Metric Browser > Target Info > Run Query.

[Explore > Metric Browser > Target Info > Run Query. Need visual here]

Query the monitored targets using PromQL in your Coralogix - Grafana dashboard:

target_info

The query above should return metrics similar to the following:

target_info{__meta_applicationname="<redis-namespace>", __meta_subsystem="redis-0", cx_tenant_id="32680"}

To discover all Redis metrics, use the following PromQL:

{__meta_applicationname="<redis-namespace>", __meta_subsystem="redis-0"}

This should result in:

...
redis_total_connections_received{__meta_applicationname="<redis-namespace>, __meta_subsystem="redis-0", cx_tenant_id="32680", env="ci", host="redis-0", hostname="redis-0", namespace="<redis-namespace>", nodename="rock64-1", port="6379", replication_role="master", server="localhost", type="infra"}

Here is a display of the expected result:

Additional References

Github: Official Telegraf-Operator Documentation

Support

Need help? Contact Coralogix support.

If you have any questions, feel free to reach out to us 24/7 via our in-app chat!

Telegraf Operator - MLKEREN/SOFTWARE-DOC-DEV-MIYA-KEREN GitHub Wiki

Telegraf Operator

Table of Contents

Overview

Prerequisites

Set Up

Installation

Configuration

Validation

Additional References

Support

Support

⚠️ GitHub.com Fallback ⚠️

Telegraf Operator - MLKEREN/SOFTWARE-DOC-DEV-MIYA-KEREN GitHub Wiki

Telegraf Operator

Table of Contents

Overview

Prerequisites

Set Up

Installation

Configuration

Validation

Additional References

Support

Support

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️