Installing the Observe OpenTelemetry App

Install the OpenTelemetry App for Observe from the App section in Workspace Setting.

Figure 1 - The OpenTelemetry app card

The default configuration handles many common needs, but you may also specify options appropriate for your environment:

• Datastream - the datastream receiving your OpenTelemetry data.

• Extract Attributes - List of attributes to be extracted as columns for all spans.

• Extract Resource Attributes - List of resource attributes to be extracted as columns for all spans.

• Freshness Default - Default dataset freshness. It may be overridden with freshness input.

• Kubernetes App - If you also use the Kubernetes App, automatically link your Kubernetes resources to your OpenTelemetry datasets to enable GraphLink.

• Max Expiry - Expiry time for OpenTelemetry traces.

• Max Time Diff - Maximum time difference for the processing time window.

• Name Format - The location of your OpenTelemetry resources is the OpenTelemetry folder, by default.

Creating a data connection

To allow the OpenTelemetry app to accept incoming data, create a data connection and a token. If you configured the app to use an existing datastream, this is the datastream token associated with the specified datastream. If not, create a new connection:

1. From the Apps tab, click Manage on the OpenTelemetry card to open the App settings page.

2. Click the Connections tab.

3. In the Data connection requirements section, click Create connection.

4. Enter a name and description for the new connection token and click Continue.

5. Copy the newly created token and store it in a safe location. You need it later to configure OpenTelemetry data ingestion.

6. Click Continue to open the OpenTelemetry collector instructions.

Note

You may also access the Collector instructions by clicking View instructions in the Connections tab.

Configuring the OpenTelemetry collector

The Observe OpenTelemetry collector is a gateway process that collects span data and sends it to Observe.

To install the collector, you need the following:

• An Observe Customer ID.

• The OpenTelemetry connection token you created in the previous section.

OpenTelemetry for Kubernetes

For Kubernetes environments, deploy the collector with the Observe “kustomize manifest” using the following steps:

1. Create a secret for the OpenTelemetry collector.

   OBSERVE_CUSTOMER='customer_id'
   OBSERVE_TOKEN='connection_token_for_otel_app'
   kubectl -n observe create secret generic otel-credentials \
           --from-literal=OBSERVE_CUSTOMER=${OBSERVE_CUSTOMER?} \
           --from-literal=OBSERVE_TOKEN=${OBSERVE_TOKEN?}
   

2. Install the collector kustomize stack using kubectl.

   kubectl apply -k github.com/observeinc/manifests/stack/otel
   

   Specify a larger deployment size for larger environments (100+ Kubernetes nodes).

   kubectl apply -k github.com/observeinc/manifests/stack/otel/l
   

For full details, see the manifest README in GitHub.

Testing OpenTelemetry Ingestion with Docker

You may also test your OpenTelemetry instrumentation in a development environment with Docker Desktop. To install the collector for testing, you need the following items:

• Two command line shell sessions on macOS or Windows with WSL.

• Docker Desktop

• An Observe customer ID and datastream token.

• The OpenTelemetry app and a data connection token.

Note

This example is intended for testing; it is not suitable for production use.

In the first shell, use the following steps:

1. Create a new directory for testing.

   mkdir otel-test
   

2. Create a config.yaml configuration file with the following contents:

   Click to expand

   # Test OpenTelemetry collector configuration. Not for production use.
   exporters:
     logging:
       loglevel: "${OTEL_LOG_LEVEL}"
     otlphttp:
       endpoint: "https://${OBSERVE_CUSTOMER}.${OBSERVE_COLLECTOR_HOST}:${OBSERVE_COLLECTOR_PORT}/v1/otel"
       headers:
         authorization: "Bearer ${OBSERVE_TOKEN}"
   extensions:
     health_check: {}
     zpages: {}
   processors:
     probabilistic_sampler:
       hash_seed: 22
       sampling_percentage: 100
     batch:
     memory_limiter:
       # 80% of maximum memory up to 2G
       limit_mib: 2048
       # 25% of limit up to 2G
       spike_limit_mib: 512
       check_interval: "2s"
   receivers:
     zipkin:
     otlp:
       protocols:
         grpc:
         http:
   service:
     telemetry:
       logs:
         level: "${OTEL_LOG_LEVEL}"
     extensions: [health_check, zpages]
     pipelines:
       traces:
         receivers: [otlp]
         processors: [probabilistic_sampler, memory_limiter, batch]
         exporters: [logging, otlphttp]
       metrics:
         receivers: [otlp]
         processors: [memory_limiter, batch]
         exporters: [logging]
   

3. Start the Docker process, configured to accept data on ports 4317 (gRPC) and 4318 (POST).

   export OBSERVE_TOKEN ; export OBSERVE_CUSTOMER
   docker run -p 4317:4317 -p 4318:4318 \
           --env OBSERVE_TOKEN="${OBSERVE_TOKEN}" \
           --env OBSERVE_CUSTOMER="${OBSERVE_CUSTOMER}" \
           --env OBSERVE_COLLECTOR_HOST="collect.observeinc.com" \
           --env OBSERVE_COLLECTOR_PORT="443" \
           --env OTEL_LOG_LEVEL="debug" \
       -v $(pwd -P)/config.yaml:/etc/otel-collector-config.yaml \
       otel/opentelemetry-collector-contrib:0.62.1 \
       --config=/etc/otel-collector-config.yaml
   

4. Leave the Docker process running and open a second shell.

In the second shell, perform the following steps:

1. Create sample span data in test.json using the payload below. You may use the same otel-test directory created previously.

   Click to expand

   {
    "resourceSpans": [
      {
        "resource": {
          "attributes": [
            {
              "key": "service.name",
              "value": {
                "stringValue": "test-with-curl"
              }
            }
          ]
        },
        "scopeSpans": [
          {
            "instrumentationLibrary": {
              "name": "manual-test"
            },
            "spans": [
              {
                "traceId": "71699b6fe85982c7c8995ea3d9c95df2",
                "spanId": "3c191d03fa8be065",
                "name": "spanitron",
                "kind": 2,
                "droppedAttributesCount": 0,
                "events": [],
                "droppedEventsCount": 0,
                "status": {
                  "code": 1
                }
              }
            ]
          }
        ]
      }
    ]
   }
   

2. Send the test data to the collector with curl.

 curl -i http://localhost:4318/v1/traces -X POST -H "Content-Type: application/json" -d @test.json

1. Confirm the collector accepted the test data.

 $ curl -i http://localhost:4318/v1/traces -X POST -H "Content-Type: application/json" -d @test.json
 HTTP/1.1 200 OK
 Content-Type: application/json
 Date: Mon, 14 Nov 2022 22:48:06 GMT
 Content-Length: 2
 
 {}

You can now view the ingested data:

1. In the Apps tab, locate the OpenTelemetry app card in the list of installed apps.

2. Click Manage and then Connections.

3. Under Existing connections, click on your token to view details.

4. Click Open dataset to view the data associated with this token.