The Observe Agent can turn GenAI spans into GenAI metrics, such as model call duration and token usage, as the spans pass through it. The metrics follow the OpenTelemetry GenAI semantic conventions, so they read the same way as the metrics a GenAI SDK records itself.

The Observe GenAI SDKs for JavaScript and Python already record GenAI metrics, so apps that use them do not need this. Use it when an app uses some other instrumentation that writes GenAI spans but no metrics, and you still want those metrics in Observe by Snowflake.

Before you start

Verify the following conditions are met before you start.

• The spans must follow the GenAI semantic conventions. The agent only reads spans that have gen_ai.operation.name, and takes token counts from the gen_ai.usage.* attributes. It ignores every other span.
• The spans must go through the Observe Agent. Point the app's OTLP exporter at the agent, not straight at Observe by Snowflake. The endpoint for each setup is given below.
• Each call must be counted only once. If the instrumentation also records GenAI metrics, turn those off. If some services use the Observe GenAI SDK and others don't, use include (see Options) to build metrics only for the ones that don't.
• Versions. Kubernetes needs the agent chart from observeinc/helm-charts, version 0.97.0 or later. A host install needs Observe Agent 2.19.0 or later.

Turn it on in Kubernetes

Add this to your Helm values:

application:
  GenAIMetrics:
    enabled: true

Then upgrade the release, keeping the values you already set:

helm upgrade observe-agent observe/agent -n observe --reuse-values -f genai-metrics-values.yaml

Send the app's spans to the agent's forwarder service. With a release named observe-agent in the observe namespace, that is:

OTEL_EXPORTER_OTLP_ENDPOINT=http://observe-agent-forwarder.observe.svc.cluster.local:4318

The metrics are built on the gateway when gatewayDeployment.enabled is true, and on the forwarder otherwise. They carry k8s.namespace.name and k8s.pod.name as well. Spans longer than node.forwarder.traces.maxSpanDuration (1 hour by default) are dropped, so they are not counted.

Turn it on for a host

Add this to the Observe Agent's configuration file. On Linux, that is /etc/observe-agent/observe-agent.yaml:

application:
  genai_metrics:
    enabled: true

Then restart the Observe Agent:

sudo systemctl restart observe-agent

Send the app's spans to the Observe Agent on the same host:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Options

The Helm chart and the Observe Agent's configuration file take the same options, spelled differently. In Helm they go under application.GenAIMetrics (see the chart's values reference); in the configuration file, under application.genai_metrics.

Each include entry sets a service name, an instrumentation scope name, or both. A span matches an entry when it has every field the entry sets. For example, to build metrics only for support-bot, and for the spans of one instrumentation in search-api:

# Helm
application:
  GenAIMetrics:
    enabled: true
    include:
      - serviceName: support-bot
      - serviceName: search-api
        instrumentationScopeName: <instrumentation scope name>

# Config file
application:
  genai_metrics:
    enabled: true
    include:
      - service_name: support-bot
      - service_name: search-api
        instrumentation_scope_name: <instrumentation scope name>

Metrics it builds

This table summarizes the metrics built from GenAI spans:

Model call metrics are broken down by gen_ai.operation.name, provider, requested and returned model, server address and port, agent name, and, on the duration and per-call metrics, error.type. Every metric also carries service.name, service.namespace, service.version, and deployment.environment.name.

A model call span with no token counts adds 0 to the token metrics. Spans don't split token counts by input type, such as text or image, so gen_ai.token.modality is always unknown.