Node.js (server) instrumentation for LLM observability¶

Supported Frameworks and Libraries¶

OpenLLMetry automatically instruments the following provider APIs and libraries:

OpenAI
Azure OpenAI
Anthropic
Cohere
Bedrock (AWS)
Vertex AI (GCP)

Supported Vector Databases¶

Chroma
Pinecone
Qdrant

Supported Frameworks¶

LangChain
LlamaIndex

Install & configure the Node.js instrumentation¶

Add traceloop SDK as a dependency npm install --save @traceloop/node-server-sdk

Initialize the SDK

import * as traceloop from "@traceloop/node-server-sdk";

traceloop.initialize(
  appName: "<YOUR_SERVICE_NAME>",
);

Set the following environment variables:¶

Environment variable	Example Values	Description	Optional?
`TRACELOOP_BASE_URL`	http://<YOUR_OBSERVE_AGENT_HOSTNAME>:4318	OTLP endpoint	No
`TRACELOOP_TRACE_CONTENT`	true / false	Enables or disables extraction of inputs and outputs from LLM calls	Yes
`OTEL_RESOURCE_ATTRIBUTES`	deployment.environment=dev,service.namespace=inference	A list of key=value resource attributes you wish to add to your spans	Yes

Add attributes at runtime¶

To annotate your span data with custom attributes like customer_id, user_id, and so on, we recommend using OpenLLMetry functions. Depending on your use-case, you may choose one of the following approaches:

For class methods

import * as traceloop from "@traceloop/node-server-sdk"

/* This example wraps a workflow span as denoted by the "workflow" decorator. To wrap spans of other types you may use the task/agent/tool decorators as desired
*/
class MyClass {
  @traceloop.workflow({ associationProperties: { userId: "user123" })
  myMethod() {
    // Your code here
  }
}

Within attributes per workflow / task / agent / tool call spans

import * as traceloop from "@traceloop/node-server-sdk";

// Wraps workflows
// Replace with withTask / withAgent / withTool as needed
traceloop.withWorkflow(
  {
    name: "workflow_name",
    associationProperties: { userId: "user12345", chatId: "chat12345" },
  },
  () => {
    // Your code here
    // (function can be made async if needed)
  }
);

Directly wrapping with global attributes

import * as traceloop from "@traceloop/node-server-sdk"

traceloop.withAssociationProperties(
  {
    userId: "user12345",
    chatId: "chat12345",
  },
  () => {
    // Your code here
    // (can be async or sync)
  }
);