Collect Statsd Metrics

StatsD is a network daemon (originally written by Etsy) that listens for metrics (such as counters, timers, and gauges) sent over UDP (User Datagram Protocol). Its primary job is to aggregate these metrics periodically and then pass them along to a backend service for storage and visualization.

  1. After the Observe Agent is up and running, create a statsd-values.yaml file.

agent:
  config:
    nodeLogsMetrics:
      receivers:
        statsd:
          endpoint: ${env:MY_POD_IP}:8125
          aggregation_interval: 60s 
          transport: "udp"
      processors:
        attributes/debug_source_statsd_metrics:
          actions:
          - action: insert
            key: debug_source
            value: statsd
      service:
        pipelines:
          metrics/custom:
            receivers: [statsd]
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_statsd_metrics]
            exporters: [prometheusremotewrite]

node-logs-metrics:
  service:
    enabled: true
    type: ClusterIP
  networkPolicy:
    enabled: true
  ports:
    statsd:
      enabled: true
      containerPort: 8125
      servicePort: 8125
      protocol: UDP

  1. Redeploy the Observe Agent

Run the following command to redeploy the Observe Agent in the observe namespace with the prometheusScrape configuration.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values statsd-values.yaml --version 0.37.0

  1. Restart pods

Run the following commands to restart the pods with the updated configuration.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe

Run the following command to make sure the Observe Agent has been redeployed successfully.

kubectl get pods -o wide -n observe

  1. Once the Observe Agent is up and running on a Kubernetes cluster, you can configure your application running on the same Kubernetes cluster to send statsd metrics to the Observe Agent.

STATSD_HOST=observe-agent-node-logs-metrics.observe.svc.cluster.local
STATSD_PORT=8125
STATSD_PREFIX=myapp

  1. Explore your statsd metrics using the Metrics Explorer