Collect StatsD metrics

StatsD is a network daemon (originally written by Etsy) that listens for metrics (such as counters, timers, and gauges) sent over User Datagram Protocol (UDP). Its primary job is to aggregate these metrics periodically and then pass them along to a backend service for storage and visualization.

📘

Note

The following instructions are based on Agent Chart version 0.57 or later. If you're currently using Agent Chart versions older than 0.57, follow the legacy guide: Collect StatsD metrics (legacy). To view all available Agent Chart versions, run: helm search repo observe --versions | grep observe/agent.

  1. After the Observe Agent is up and running, create a statsd-values.yaml file.
    agent:
      config:
        forwarder:
          receivers:
            statsd:
              endpoint: ${env:MY_POD_IP}:8125
              aggregation_interval: 60s 
              transport: "udp"
          processors:
            attributes/debug_source_statsd_metrics:
              actions:
              - action: insert
                key: debug_source
                value: statsd
          service:
            pipelines:
              metrics/custom-statsd:
                receivers: [statsd]
                processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_statsd_metrics]
                exporters: [otlphttp/observe/otel_metrics]
    
    forwarder:
      ports:
        statsd:
          enabled: true
          containerPort: 8125
          servicePort: 8125
          protocol: UDP
  2. Run the following command to redeploy the Observe Agent in the observe namespace:
    helm upgrade --reuse-values observe-agent observe/agent -n observe --values statsd-values.yaml
  3. Run the following commands to restart the pods with the updated configuration:
    kubectl rollout restart deployment -n observe
    kubectl rollout restart daemonset -n observe
  4. Run the following command to make sure the Observe Agent has been redeployed successfully:
    kubectl get pods -o wide -n observe
  5. Once the Observe Agent is up and running on a Kubernetes cluster, you can configure your application running on the same Kubernetes cluster to send statsd metrics to the Observe Agent.
    STATSD_HOST=observe-agent-forwarder.observe.svc.cluster.local
    STATSD_PORT=8125
    STATSD_PREFIX=myapp
  6. Explore your StatsD metrics using the Metrics Explorer.