Install Observe Agent on Linux

Install the Observe Agent

Install the observe-agent package. You’ll need to first add the Observe debian repository to your trusted depositories in your sources.list.d file.

echo 'deb [trusted=yes] https://repo.observeinc.com/apt/ /' | sudo tee /etc/apt/sources.list.d/observeinc.list
sudo apt update
sudo apt install observe-agent

To validate that the agent is installed correctly, you can run the version command.

observe-agent version

Install the observe-agent package. You’ll need to first add the Observe yum repository to your trusted repositories in your yum.repos.d folder.

echo '[fury]
name=Gemfury Private Repo
baseurl=https://yum.fury.io/observeinc/
enabled=1
gpgcheck=0' | sudo tee /etc/yum.repos.d/fury.repo

sudo yum install observe-agent

To validate that the agent is installed correctly, you can run the version command.

observe-agent version

Download and install the observe-agent from our official bash script.

curl -Ol "https://raw.githubusercontent.com/observeinc/observe-agent/main/scripts/install_linux.sh"
bash install_linux.sh --token=<YOUR OBSERVE TOKEN> --observe_url=<YOUR OBSERVE COLLECTION ENDPOINT> --logs_enabled=true --metrics_enabled=true

This will install the agent locally and initialize the configuration. If your installation uses systemd, the agent will be added as a service and started automatically.

To validate that the agent is installed correctly, you can run the version command.

observe-agent version

Configure the Observe Agent

Automatic configuration

The easiest way to initialize the agent configuration is to use the observe-agent init-config command as follows:

Replace the contents of the agent configuration file with the config file below. Replace ${OBSERVE_TOKEN} (your instance’s token) and ${OBSERVE_COLLECTION_ENDPOINT} (your instance’s collection endpoint) on each host. For more information on configuring the agent, see Configuration.

sudo observe-agent init-config \
--token ${OBSERVE_TOKEN} \
--observe_url https://${OBSERVE_CUSTOMER_ID}.collect.observeinc.com/ \
--self_monitoring::enabled=true \
--host_monitoring::enabled=true \
--host_monitoring::logs::enabled=true \
--host_monitoring::metrics::host::enabled=true \
--host_monitoring::metrics::process::enabled=false

This is only recommended for initial configuration. For subsequent changes, see below.

Manual configuration

For initial configuration, we suggest using observe-agent init-config. For changes to the defaults, see below.

Open the agent config file /etc/observe-agent/observe-agent.yaml with superuser permissions which will allow you to edit and save changes to the file.

sudo vim /etc/observe-agent/observe-agent.yaml

Replace the contents of the agent configuration file with the config file below. Replace ${OBSERVE_TOKEN} (your instance’s token) and ${OBSERVE_COLLECTION_ENDPOINT} (your instance’s collection endpoint) on each host. For more information on configuring the agent, see Configuration.

Note

The ${OBSERVE_COLLECTION_ENDPOINT} is composed from https://${OBSERVE_CUSTOMER_ID}.collect.${OBSERVE_INSTANCE}. For example, if you typically login to https://123456789012.observeinc.com, your ${OBSERVE_COLLECTION_ENDPOINT} is https://123456789012.collect.observeinc.com.

Note

Some Observe instances may optionally use a name instead of Customer ID; if this is the case for your instance, contact your Observe Data Engineer to discuss implementation. A stem name will work as is, but a DNS redirect name may require client configuration.

# Observe data token (ex: a1b2c3d4e5f6g7h8i9k0:l1m2n3o4p5q6r7s8t9u0v1w2x3y4z5a6)
token: "${OBSERVE_TOKEN}"

# Target Observe collection url (ex: https://123456789012.collect.observeinc.com/)
observe_url: "${OBSERVE_COLLECTION_ENDPOINT}"

# Debug mode - Sets agent log level to debug
debug: false

self_monitoring:
  enabled: true

host_monitoring:
  enabled: true
  logs: 
    enabled: true
    include:
      - /var/log/**/*.log
      - /var/log/syslog
  metrics:
    host: 
      enabled: true
    process:
      enabled: false

# otel_config_overrides:
#   exporters:
#     # This is a net new exporter
#     debug:
#       verbosity: detailed
#       sampling_initial: 5
#       sampling_thereafter: 200
#   service:
#     pipelines:
#       # This will override the existing metrics/host_monitoring pipeline and output to stdout debug instead
#       metrics/host_monitoring_host:
#         receivers: [hostmetrics/host-monitoring-host]
#         processors: [memory_limiter]
#         exporters: [debug]
#       # This is a net new pipeline
#       logs/new-pipeline:
#         receivers: [filelog/host_monitoring]
#         processors: [memory_limiter]
#         exporters: [debug]

Open the agent config file /etc/observe-agent/observe-agent.yaml with superuser permissions which will allow you to edit and save changes to the file.

sudo vi /etc/observe-agent/observe-agent.yaml

Replace the contents of the agent configuration file with the config file below. Replace ${OBSERVE_TOKEN} (your instance’s token) and ${OBSERVE_COLLECTION_ENDPOINT} (your instance’s collection endpoint) on each host. For more information on configuring the agent, see Configuration.

# Observe data token (ex: a1b2c3d4e5f6g7h8i9k0:l1m2n3o4p5q6r7s8t9u0v1w2x3y4z5a6)
token: "${OBSERVE_TOKEN}"

# Target Observe collection url (ex: https://123456789012.collect.observeinc.com/)
observe_url: "${OBSERVE_COLLECTION_ENDPOINT}"

# Debug mode - Sets agent log level to debug
debug: false

self_monitoring:
  enabled: true

host_monitoring:
  enabled: true
  logs: 
    enabled: true
    include:
      - /var/log/**/*.log
      - /var/log/syslog
  metrics:
    host: 
      enabled: true
    process:
      enabled: false

# otel_config_overrides:
#   exporters:
#     # This is a net new exporter
#     debug:
#       verbosity: detailed
#       sampling_initial: 5
#       sampling_thereafter: 200
#   service:
#     pipelines:
#       # This will override the existing metrics/host_monitoring pipeline and output to stdout debug instead
#       metrics/host_monitoring_host:
#         receivers: [hostmetrics/host-monitoring-host]
#         processors: [memory_limiter]
#         exporters: [debug]
#       # This is a net new pipeline
#       logs/new-pipeline:
#         receivers: [filelog/host_monitoring]
#         processors: [memory_limiter]
#         exporters: [debug]

Start the Observe Agent

Now that the configuration is in place, you can start the agent with the following command

sudo systemctl enable --now observe-agent

You can then run the status command to see statistics and information about the agent.

observe-agent status

Configure application instrumentation

Follow these steps to direct your application’s trace data to Observe:

  1. Install the OpenTelemetry App if you have not already done so.

  2. Create a token associated with OpenTelemetry App’s datastream.

  3. Open the agent config file /etc/observe-agent/observe-agent.yaml with superuser permissions which will allow you to edit and save changes to the file.

  4. Use the otel_config_overrides section to configure an additional OTLP/HTTP Exporter. Replace <YOUR ANOTHER OBSERVE TOKEN> (your instance’s token) and <YOUR OBSERVE COLLECTION ENDPOINT> (your instance’s collection endpoint) on each host.

# this should be added to the existing observe-agent.yaml
otel_config_overrides:
  exporters:
    otlphttp/observe-traces:
      # (ex: https://123456789012.collect.observeinc.com/v2/otel)
      endpoint: "<YOUR OBSERVE COLLECTION ENDPOINT>/v2/otel"
      headers:
        # (ex: Bearer a1b2c3d4e5f6g7h8i9k0:l1m2n3o4p5q6r7s8t9u0v1w2x3y4z5a6)
        authorization: "Bearer <YOUR ANOTHER OBSERVE TOKEN>"
      sending_queue:
        num_consumers: 4
        queue_size: 100
      retry_on_failure:
        enabled: true
      compression: zstd
  service:
    pipelines:
      traces/forward:
        receivers: [otlp]
        processors: [resourcedetection, resourcedetection/cloud]
        exporters: [otlphttp/observe-traces]

  1. Restart the Observe Agent

sudo systemctl restart observe-agent

  1. Once the Observe Agent is up and running on a Linux host, you can configure your application running on the same Linux host to send telemetry data to the Observe Agent using one of the following addresses:

  • OTLP/HTTP endpoint: http://localhost:4318

  • OTLP/grpc endpoint: http://localhost:4317

  1. Explore your trace data using the Trace Explorer and the Service Explorer

Uninstall the Observe Agent

You can uninstall the agent via apt after stopping the service.

sudo systemctl stop observe-agent
sudo apt-get purge observe-agent

You can uninstall the agent via yum after stopping the service.

sudo systemctl stop observe-agent
sudo yum erase observe-agent -y

Upgrade the Observe Agent

Warning

Observe Agent v1.0.0 includes breaking changes. Learn more about these changes in Upgrade to Observe Agent v1.0.0

Upgrade the package version. The install process will ask if you want to replace your existing config file. Generally the default answer of N (no) is correct.

sudo apt update
sudo apt install --only-upgrade observe-agent

To apply the new configs and start the agent at the new version you can run the following command.

sudo systemctl restart observe-agent

Upgrade the package version. The install process will ask if you want to replace your existing config file. Generally the default answer of N (no) is correct.

sudo yum update observe-agent
sudo yum install observe-agent

To apply the new configs and start the agent at the new version you can run the following command.

sudo systemctl restart observe-agent