AI agent manual instrumentation

📘

Private preview feature (Invite only)

This feature is enabled per tenant by Observe by Snowflake and is not generally available. To participate, contact your Observe by Snowflake representative. You may access and use this preview feature solely for internal evaluation purposes. Previews are provided as-is. Your use of this preview is subject to the Snowflake Preview Terms located at https://www.snowflake.com/en/legal.

This page describes how to manually instrument your LLM/agent applications with OpenTelemetry and populate the traces and conversations views in Observe by Snowflake's AI Agent Explorer. This page lists the gen_ai.* properties we read, the conversation key that threads a multi-turn session together, the message shape we expect, and the span structures that produce correct trace trees.

To emit metrics, you must also use the Observe Agent to turn GenAI spans into GenAI metrics. See Build GenAI metrics from spans using the Observe Agent.

Here is a summary of the steps and requirements:

  • Spans are only discovered if they have the gen_ai.operation.name attribute present. Make sure to set gen_ai.operation.name on model calls and model/provider/token/message attributes.
  • Put a gen_ai.conversation.id on every span in a session — this is what threads a multi-turn conversation together in the Conversations view.
  • Encode content as gen_ai.input.messages / gen_ai.output.messages using the {role, parts:[{type, content}]} shape.
  • Give each model call, tool call, and agent wrapper its own span; children point parent_span_id at their parent and share one trace_id.

Get your spans discovered

A span only appears in the Agent Explorer if the gen_ai.operation.name attribute is set.

Span envelope (standard OTel)

Every span needs a valid envelope or it can't be navigated:

FieldRequirementWhy
trace_idRequiredAll spans in one interaction/conversation must share it. Opens Trace Inspector.
span_idRequiredUnique per span; forms the (trace_id, span_id) key.
parent_span_idRequired (non-root)Must equal the direct parent's span_id. Omit only on the root.
start_time / end_timeRequiredOrdering, duration, waterfall. Children must fit inside their parent's interval.
span nameRecommendedLow-cardinality label, such as chat gpt-4.1. No prompts/IDs/secrets in the name.
span kindRecommendedCLIENT for provider calls; INTERNAL for agent/tool calls
span statusRecommended on failuresSet OTel status ERROR (code 2) on failed model/tool calls. This drives the Status/Error column.
resource.service.nameRecommendedPowers the Service / App column and grouping.
resource.deployment.environment.nameRecommendedPowers the Environment column/filter.

The gen_ai.* properties

These are the attributes that populate AI Agent Explorer views. Put model/provider/token/content fields on each model-call span, not only the root wrapper.

AttributeRequirementPowers
gen_ai.conversation.idRequired for Conversations viewThreads all turns of a session into one Conversations row. See Conversation threading.
gen_ai.operation.nameRequiredType badge and text: chat, text_completion, embeddings, execute_tool, invoke_agent, create_agent, invoke_workflow, retrieval.
gen_ai.agent.nameRecommendedagent column in the Conversation view. Metric tags.
gen_ai.request.modelRecommended on model callsrequest_model column in the Conversation view.
gen_ai.response.modelRecommended on model callsresponse_model column in the Conversation view.
gen_ai.provider.nameRecommended on model callsprovider column in the Conversation view.
gen_ai.usage.input_tokensRecommendedInput-token column and per-conversation totals.
gen_ai.usage.output_tokensRecommendedOutput-token column and per-conversation totals.
gen_ai.input.messagesRecommendedConversation preview; populating the conversation chat views. See Message content shape.
gen_ai.output.messagesRecommendedPopulating conversation chat views. See Message content shape.
gen_ai.tool.nameRequired on execute_tool spansTraces tab tool spans, tools in the conversation chat views
gen_ai.tool.call.idRecommendedCorrelates the assistant tool_call, the execute_tool span, and the tool response.
gen_ai.tool.call.arguments / .resultOptional (sensitive)Tool arguments/result content in the inspector. Redact per policy.
gen_ai.tool.definitionsOptionalTool-definition cards. Array of {name, description, parameters}.

Conversation threading

The Conversations view groups by gen_ai.conversation.id. To make several turns (which may span multiple traces) appear as one conversation:

  • Put the same gen_ai.conversation.id on every span belonging to a single session.

    Set it on the invoke_agent span along with all of the relevant child spans such as chat and tool, grouping, and per-conversation rollups (models, providers, tools, token totals, duration, trace list) read from every span carrying the ID.

  • A span with no gen_ai.conversation.id does not appear in the Conversations view. It will still appear in the Traces view.

Message content shape

Encode prompts and completions as gen_ai.input.messages and gen_ai.output.messages. Each is either a decoded array or a JSON string encoding that array.

Input message:

[
  {
    "role": "user",
    "parts": [
      { "type": "text", "content": "What is the weather in SF?" }
    ]
  }
]

Output message, with a finish_reason added:

[
  {
    "role": "assistant",
    "parts": [
      { "type": "text", "content": "It is sunny." }
    ],
    "finish_reason": "stop"
  }
]
FieldRequirementUse
roleRecommendedRole badge; System/User/Assistant/Tool filtering.
partsRequired for readable contentArray of content parts.
parts[].typeRequiredtext, reasoning, tool_call, tool_call_response (unknown/multimodal tolerated).
parts[].contentRequired for text/reasoningRendered, searchable text.
finish_reasonOptionalFinish-reason pill.

Tool call inside an assistant message:

{ "type": "tool_call", "id": "call_123", "name": "get_weather", "arguments": { "city": "San Francisco" } }

Tool call inside a tool message:

{ "type": "tool_call_response", "id": "call_123", "response": { "temperature": 18 } }

Warning

Input messages, output messages, tool arguments/results, and reasoning can carry customer data or credentials. Capture only under your content-capture and redaction policy. Never put this content in span names.

Span structures

The Agent Explorer doesn't reject other trees, but these shapes produce correct loops, navigation, and per-operation metrics. Every span shares one trace_id, carries the same gen_ai.conversation.id, and points parent_span_id at its parent.

Single turn (one model call)

invoke_agent support-agent          INTERNAL   gen_ai.operation.name=invoke_agent, gen_ai.agent.name, gen_ai.conversation.id
└── chat gpt-4.1                    CLIENT     gen_ai.operation.name=chat, model/provider/tokens/messages

Single-agent loop (model → tool → model)

invoke_agent support-agent          INTERNAL
├── chat gpt-4.1                    CLIENT     model decision (emits assistant tool_call)
├── execute_tool lookup_order       INTERNAL   gen_ai.tool.name, gen_ai.tool.call.id
└── chat gpt-4.1                    CLIENT     final synthesis
  • The root's children are siblings, ordered by start time.
  • Each chat span carries its own model, provider, tokens, content, status, duration.
  • Tool-call IDs match across the assistant message, the execute_tool span, and the tool-response message.

Multi-turn conversation (threaded by gen_ai.conversation.id)

Each user turn is typically its own trace. Same gen_ai.conversation.id on every span makes them one Conversations row:

Trace A  (gen_ai.conversation.id = conv-123)
└── invoke_agent support-agent
    ├── chat gpt-4.1                turn 1 decision
    ├── execute_tool lookup_order
    └── chat gpt-4.1                turn 1 answer

Trace B  (gen_ai.conversation.id = conv-123)
└── invoke_agent support-agent
    └── chat gpt-4.1                turn 2 answer

LLM-backed tool / delegated sub-agent

Nest only when the tool actually invokes a model or delegates:

invoke_agent coordinator
└── execute_tool delegate_to_specialist
    └── invoke_agent specialist
        ├── chat gpt-4.1
        └── execute_tool specialist-tool

Retrieval-augmented generation

invoke_agent support-agent
├── embeddings text-embedding-3     optional query embedding
├── retrieval vector-store          retrieval operation
└── chat gpt-4.1                    synthesis

Minimal end-to-end example

Conceptual span set (resource/scope wrappers and timestamps omitted for readability). Every span shares trace_id and gen_ai.conversation.id.

{
  "trace_id": "0123456789abcdef0123456789abcdef",
  "spans": [
    {
      "span_id": "0000000000000001",
      "parent_span_id": null,
      "name": "invoke_agent support-agent",
      "kind": "INTERNAL",
      "attributes": {
        "gen_ai.operation.name": "invoke_agent",
        "gen_ai.agent.name": "support-agent",
        "gen_ai.conversation.id": "conv-123",
        "gen_ai.input.messages": "[{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"Where is order 42?\"}]}]"
      }
    },
    {
      "span_id": "0000000000000002",
      "parent_span_id": "0000000000000001",
      "name": "chat gpt-4.1",
      "kind": "CLIENT",
      "attributes": {
        "gen_ai.operation.name": "chat",
        "gen_ai.conversation.id": "conv-123",
        "gen_ai.provider.name": "openai",
        "gen_ai.request.model": "gpt-4.1",
        "gen_ai.response.model": "gpt-4.1",
        "gen_ai.input.messages": "[{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"Where is order 42?\"}]}]",
        "gen_ai.output.messages": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"tool_call\",\"id\":\"call-1\",\"name\":\"lookup_order\",\"arguments\":{\"order_id\":42}}],\"finish_reason\":\"tool_calls\"}]",
        "gen_ai.usage.input_tokens": 120,
        "gen_ai.usage.output_tokens": 20
      }
    },
    {
      "span_id": "0000000000000003",
      "parent_span_id": "0000000000000001",
      "name": "execute_tool lookup_order",
      "kind": "INTERNAL",
      "attributes": {
        "gen_ai.operation.name": "execute_tool",
        "gen_ai.conversation.id": "conv-123",
        "gen_ai.tool.name": "lookup_order",
        "gen_ai.tool.call.id": "call-1",
        "gen_ai.tool.call.arguments": "{\"order_id\":42}",
        "gen_ai.tool.call.result": "{\"status\":\"shipped\"}"
      }
    },
    {
      "span_id": "0000000000000004",
      "parent_span_id": "0000000000000001",
      "name": "chat gpt-4.1",
      "kind": "CLIENT",
      "attributes": {
        "gen_ai.operation.name": "chat",
        "gen_ai.conversation.id": "conv-123",
        "gen_ai.provider.name": "openai",
        "gen_ai.request.model": "gpt-4.1",
        "gen_ai.response.model": "gpt-4.1",
        "gen_ai.input.messages": "[{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"Where is order 42?\"}]},{\"role\":\"assistant\",\"parts\":[{\"type\":\"tool_call\",\"id\":\"call-1\",\"name\":\"lookup_order\",\"arguments\":{\"order_id\":42}}]},{\"role\":\"tool\",\"parts\":[{\"type\":\"tool_call_response\",\"id\":\"call-1\",\"response\":{\"status\":\"shipped\"}}]}]",
        "gen_ai.output.messages": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"Order 42 has shipped.\"}],\"finish_reason\":\"stop\"}]",
        "gen_ai.usage.input_tokens": 180,
        "gen_ai.usage.output_tokens": 30
      }
    }
  ]
}











Did this page help you?