Installing the Observe OpenTelemetry App#
Install the OpenTelemetry App for Observe from the App section in Workspace Setting.
Figure 1 - The OpenTelemetry app card
The default configuration handles many common needs, but you may also specify options appropriate for your environment:
Datastream - the datastream receiving your OpenTelemetry data.
Extract Attributes - List of attributes to be extracted as columns for all spans.
Extract Resource Attributes - List of resource attributes to be extracted as columns for all spans.
Freshness Default - Default dataset freshness. It may be overridden with freshness input.
Kubernetes App - If you also use the Kubernetes App, automatically link your Kubernetes resources to your OpenTelemetry datasets to enable GraphLink.
Max Expiry - Expiry time for OpenTelemetry traces.
Max Time Diff - Maximum time difference for the processing time window.
Name Format - The location of your OpenTelemetry resources is the
OpenTelemetry
folder by default.
Observe also has support for OpenTelemetry Logs and Metrics, however the bundled datasets are disabled by default. If you are sending Logs and Metrics to Observe via the OTEL ingestion endpoints, we recommend enabling these datasets. You can enable them via configuration page under the “Enable Logs” and “Enable Metrics” options.
Figure 2 - The OpenTelemetry Logs & Metrics dataset configuration
Creating a data connection#
To allow the OpenTelemetry app to accept incoming data, create a data connection and a token. If you configured the app to use an existing datastream, this is the datastream token associated with the specified datastream. If not, create a new connection:
From the Apps tab, click Manage on the OpenTelemetry card to open the App settings page.
Click the Connections tab.
In the Data connection requirements section, click Create connection.
Enter a name and description for the new connection token and click Continue.
Copy the newly created token and store it in a safe location. You need it later to configure OpenTelemetry data ingestion.
Click Continue to open the OpenTelemetry collector instructions.
Note
You may also access the Collector instructions by clicking View instructions in the Connections tab.
Configuring the OpenTelemetry collector#
The Observe OpenTelemetry collector is a gateway process that collects span data and sends it to Observe.
To install the collector, you need the following:
An Observe Customer collection endpoint.
The OpenTelemetry connection token you created in the previous section.
OpenTelemetry for Kubernetes#
For Kubernetes environments, deploy the collector with the Observe traces Helm chart using the following steps.
Note
While Helm is the preferred installation method, you can find alternate instructions using Kustomize.
Helm can be used to install the OpenTelemetry collector to an existing namespace. If it does not already exist, you must create an observe
namespace with a human readable cluster name annotation. This can be done with your infrastructure management tool of choice (e.g. Terraform) or with the following command:
CLUSTER_NAME="My Cluster"
kubectl create namespace observe && \
kubectl annotate namespace observe observeinc.com/cluster-name="$CLUSTER_NAME"
See https://github.com/observeinc/helm-charts#namespace for more information on the namespace requirement.
To add the Observe Helm repository, run the following commands:
helm repo add observe https://observeinc.github.io/helm-charts
helm repo update
Then create a release for the traces
chart in the observe
namespace, using helm install
:
helm install --namespace=observe observe-traces observe/traces \
--set global.observe.collectionEndpoint="${OBSERVE_COLLECTION_ENDPOINT?}" \
--set observe.token.value="${OBSERVE_TOKEN?}"
Save the values to a file, and keep it somewhere safe for further configuration and upgrades:
helm -n observe get values observe-traces -o yaml > observe-traces-values.yaml
Configuration#
See Helm chart configuration for more details, including setting custom resource limits for the agent.
Testing OpenTelemetry Ingestion with Docker#
You may also test your OpenTelemetry instrumentation in a development environment with Docker Desktop. To install the collector for testing, you need the following items:
Two command line shell sessions on macOS or Windows with WSL.
An Observe customer ID and datastream token.
The OpenTelemetry app and a data connection token.
Note
This example is intended for testing; it is not suitable for production use.
In the first shell, use the following steps:
Create a new directory for testing.
mkdir otel-test
Create a
config.yaml
configuration file with the following contents:Click to expand
exporters: logging: loglevel: "${OTEL_LOG_LEVEL}" otlphttp: endpoint: "https://${OBSERVE_CUSTOMER}.${OBSERVE_COLLECTOR_HOST}:${OBSERVE_COLLECTOR_PORT}/v2/otel" headers: authorization: "Bearer ${OBSERVE_TOKEN}" extensions: health_check: {} zpages: {} receivers: otlp: protocols: grpc: http: service: extensions: [health_check, zpages] pipelines: traces: receivers: [otlp] exporters: [otlphttp] metrics: receivers: [otlp] exporters: [otlphttp] logs: receivers: [otlp] exporters: [otlphttp, logging] telemetry: metrics: address: 0.0.0.0:8888 logs: level: "${OTEL_LOG_LEVEL}"
Start the Docker process, configured to accept data on ports 4317 (gRPC) and 4318 (POST).
export OBSERVE_TOKEN ; export OBSERVE_CUSTOMER docker run -p 4317:4317 -p 4318:4318 \ --env OBSERVE_TOKEN="${OBSERVE_TOKEN}" \ --env OBSERVE_CUSTOMER="${OBSERVE_CUSTOMER}" \ --env OBSERVE_COLLECTOR_HOST="collect.observeinc.com" \ --env OBSERVE_COLLECTOR_PORT="443" \ --env OTEL_LOG_LEVEL="debug" \ -v $(pwd -P)/config.yaml:/etc/otel-collector-config.yaml \ otel/opentelemetry-collector-contrib:0.62.1 \ --config=/etc/otel-collector-config.yaml
Leave the Docker process running and open a second shell.
In the second shell, perform the following steps:
Create sample span data in
test.json
using the payload below. You may use the sameotel-test
directory created previously.Click to expand
{ "resourceSpans": [ { "resource": { "attributes": [ { "key": "service.name", "value": { "stringValue": "test-with-curl" } } ] }, "scopeSpans": [ { "instrumentationLibrary": { "name": "manual-test" }, "spans": [ { "traceId": "71699b6fe85982c7c8995ea3d9c95df2", "spanId": "3c191d03fa8be065", "name": "spanitron", "kind": 2, "droppedAttributesCount": 0, "events": [], "droppedEventsCount": 0, "status": { "code": 1 } } ] } ] } ] }
Send the test data to the collector with
curl
.
curl -i http://localhost:4318/v1/traces -X POST -H "Content-Type: application/json" -d @test.json
Confirm the collector accepted the test data.
$ curl -i http://localhost:4318/v1/traces -X POST -H "Content-Type: application/json" -d @test.json
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 14 Nov 2022 22:48:06 GMT
Content-Length: 2
{}
You can now view the ingested data:
In the Apps tab, locate the OpenTelemetry app card in the list of installed apps.
Click Manage and then Connections.
Under Existing connections, click on your token to view details.
Click Open dataset to view the data associated with this token.