LLM Application Instrumentation¶

Instrument with OpenLLMetry¶

To use LLM Observability, you will need to instrument your application code with OpenLLMetry. OpenLLMetry is an open source SDK which leverages the power of OpenTelemetry auto-instrumentation.

Step 1: install the Observe agent in your environment.

Step 2: Follow the relevant instrumentation guide:

python.html

nodejs.html

For other languages, please refer to this documentation.

Frequently Asked Questions¶

Dropped Spans or Missing Attributes¶

This may occur when there are too many prompts / completions in a given span. Increase the default attribute limit (128) through the environment variable OTEL_ATTRIBUTE_COUNT_LIMIT. Note however, that this can still result in dropped spans or attributes if the limit is exceeded, or the Observe ingest payload limit is breached.

“Missing span” warning in Observe trace UI¶

This may happen due to incorrect instrumentations, or gen_ai.is_llm_root not being set under mixed instrumentation scenarios such as using OpenTelemetry auto-instrumentation with OpenLLMetry for LLM observability. In the latter case, you may solve it by manually setting the attribute, or by using a span processor. The following example (Python) marks the root span based on the presence of a context key injected by OpenLLMetry:

from typing import Any, Optional
from opentelemetry import context
from opentelemetry.context import Context
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

class LLMExplorerSpanProcessor(BatchSpanProcessor):
    """Processor to mark root spans for LLM Explorer."""

    def on_start(self, span: Span, parent_context: Optional[Context] = None) -> None:
        """Triggered during every span start."""
        association_properties = context.get_value("association_properties", parent_context)
        parent_is_llm_span = context.get_value("IS_LLM_SPAN", parent_context)
        is_llm_span = "False"
        if association_properties is not None:
            is_llm_span = "True"
            if not parent_is_llm_span or (
                isinstance(parent_is_llm_span, str) and parent_is_llm_span.lower() == "false"
            ):
                span.set_attribute("gen_ai.is_llm_root", True)
        else:
            is_llm_span = "False"
        context.attach(context.set_value("IS_LLM_SPAN", is_llm_span, parent_context))


# Later, specify the processor in the traceloop-sdk or the OpenTelemetry tracer provider
Traceloop.init(
		...
    processor=LLMExplorerSpanProcessor(OTLPSpanExporter()),
    metrics_exporter=OTLPMetricExporter(),
    ...
)