Install Observe Agent on Amazon ECS (Fargate)

Note

These instructions apply to tenants created on or after June 6, 2025. Interested in upgrading to the new experience? Open Docs & Support → Contact Support in the product and let us know.

These steps guide you through building a custom Docker image of the Observe Agent with a configuration file, pushing that image to Amazon ECR, creating an ECS task definition that uses it, and finally running it as a service to collect metrics and traces from your ECS Cluster.

Installation

Prerequisites

  • AWS CLI installed and configured

  • Access to an ECS cluster (Fargate)

  • Permissions to create and manage Amazon ECR repositories and CloudWatch log groups

  • Proper IAM roles for ECS tasks and execution (including permissions for CloudWatch Logs and ECR)

Prepare the Observe Agent configuration

Create the observe-agent.yaml configuration file

# Observe data token (ex: a1b2c3d4e5f6g7h8i9k0:l1m2n3o4p5q6r7s8t9u0v1w2x3y4z5a6)
token: "${TOKEN}"

# Target Observe collection url (ex: https://123456789012.collect.observeinc.com/)
observe_url: "${OBSERVE_URL}"

self_monitoring:
  enabled: true

host_monitoring:
  enabled: false
  logs:
    enabled: false
    include:
  metrics:
    host:
      enabled: false
    process:
      enabled: false

forwarding:
    enabled: true
    metrics:
        output_format: otel
        
otel_config_overrides:
  receivers:
    otlp:
      protocols:
        grpc:
          endpoint: 0.0.0.0:4317
        http:
          endpoint: 0.0.0.0:4318
    awsecscontainermetrics:
      collection_interval: 20s
  service:
    pipelines:
      metrics/ecs_fargate:
        receivers: [awsecscontainermetrics]
        processors: [memory_limiter, resourcedetection, resourcedetection/cloud, batch]
        exporters: [otlphttp/observemetrics]

Build and Push the docker image to Amazon ECR

  1. Create the Dockerfile

Note

To use a specific version, please visit https://github.com/observeinc/observe-agent/releases.

FROM observeinc/observe-agent:latest
COPY observe-agent.yaml /etc/observe-agent/observe-agent.yaml

  1. Build the docker image

docker buildx build --platform linux/amd64 --no-cache -t observe-agent:latest .

  1. Create an ECR repository In the AWS Management Console or via the CLI, create a repository named observe/observe-agent-ecs-fargate. The repository URI will look like this:

<your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe/observe-agent-ecs-fargate

  1. Push the image to ECR

aws ecr get-login-password --region <your_region> | docker login --username AWS --password-stdin <your_account_id>.dkr.ecr.<your_region>.amazonaws.com
docker tag observe-agent:latest <your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe/observe-agent-ecs-fargate:latest
docker push <your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe/observe-agent-ecs-fargate:latest

Create a CloudWatch log group for the Observe Agent

Before running the task, ensure a CloudWatch log group exists for your agent logs:

aws logs create-log-group --log-group-name /aws/ecs-fargate/observe/observe-agent --region <your_region>

Create the ECS task definition

Update the fields below with your values:

  • <your_account_id>: Your AWS account ID

  • <your_region>: The AWS region (e.g., ca-central-1)

  • <YOUR_INGEST_TOKEN>: Your instance’s ingest token you create from the Add Data for Linux page (ex: a1b2c3d4e5f6g7h8i9k0:l1m2n3o4p5q6r7s8t9u0v1w2x3y4z5a6)

  • <YOUR_OBSERVE_COLLECTION_ENDPOINT>: Your Observe collection endpoint URL (e.g., https://123456789012.collect.observeinc.com/)

  • <your_ecs_task_role>: An IAM role ARN granting necessary permissions to the task

  • <your_ecs_execution_role>: An IAM role ARN with permissions to read from ECR and write logs to CloudWatch

Note

Some Observe instances may optionally use a name instead of Customer ID; if this is the case for your instance, contact your Observe Data Engineer to discuss implementation. A stem name will work as is, but a DNS redirect name may require client configuration.

Make sure the execution role has logs:CreateLogStream and logs:PutLogEvents permissions.

{
  "family": "observe-ecs-fargate-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "taskRoleArn": "arn:aws:iam::<your_account_id>:role/<your_ecs_task_role>",
  "executionRoleArn": "arn:aws:iam::<your_account_id>:role/<your_ecs_execution_role>",
   
  "containerDefinitions": [
    {
      "name": "observe-agent",
      "image": "<your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe-agent-ecs-fargate:latest",
      "essential": true,

      "environment": [
        {"name": "TOKEN", "value": "<YOUR_INGEST_TOKEN>"},
        {"name": "OBSERVE_URL", "value": "<YOUR_OBSERVE_COLLECTION_ENDPOINT>"}
      ],

      "portMappings": [
        { "containerPort": 4317, "protocol": "tcp" },
        { "containerPort": 4318, "protocol": "tcp" }
      ],

      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/aws/ecs-fargate/observe/observe-agent",
          "awslogs-region": "<your_region>",
          "awslogs-stream-prefix":"observe-agent"
        }
      }
    }
  ]
}

Register this task definition using the AWS CLI:

aws ecs register-task-definition --cli-input-json file://observe-agent-ecs-fargate-task-definition.json --region <your_region>

Create an ECS service

  • SUBNET_A: Subnet ID #1 in the same VPC as your cluster. Usually one of your public subnets (e.g., subnet-047b48528c1c7d5ab in us-west-2b). Every Fargate task gets an ENI (elastic network interface) inside one subnet. Providing at least two subnets in different AZs lets ECS balance tasks across zones for high availability.

  • SUBNET_B: Subnet ID #2 in a different AZ (e.g., subnet-09b14d2367d0dc917 in us-west-2c). If AZ-A has an outage, ECS can still launch the task in AZ-B without needing a new service deployment (Cross-AZ redundancy)

  • SG_APP: Security group ID that the task ENI will use (e.g., sg-0123456789abcdef0). The security group is the EC2-level firewall for the task ENI. It defines exactly which ports the Observe Agent (4317/4318) can receive, and allows egress to the Observe SaaS endpoint (TLS/443). Without it, the task would default to the VPC’s “default” SG, often too open or too closed. Typical rules: Inbound TCP 5000 (demo-app), Inbound TCP 4317/4318 (optional), Outbound TCP 443 (HTTPS to Observe).

  • assignPublicIp=ENABLED gives each task its own public IP so it can send data straight to Observe without a NAT. If you deploy inside private subnets instead, flip the flag to DISABLED and rely on NAT or VPC endpoints.

Note

If you want private-only networking, point SUBNET_A/B at private subnets and set assignPublicIp=DISABLED. Make sure those subnets have a NAT or VPC-endpoint that can reach *.collect.observeinc.com.

# us-west-2b (public)
# SUBNET_A=subnet-047b48528c1c7d5ab
# us-west-2c (public)   
# SUBNET_B=subnet-09b14d2367d0dc917
# allows inbound 5000/tcp, outbound 443/tcp   
# SG_APP=sg-0123456789abcdef0         

aws ecs create-service \
  --cluster <your_ecs_cluster_name> \
  --service-name observe-agent-ecs-fargate \
  --task-definition observe-ecs-fargate-task \
  --desired-count 1 \
  --launch-type FARGATE \
  --network-configuration \
    "awsvpcConfiguration={subnets=[$SUBNET_A,$SUBNET_B],securityGroups=[$SG_APP],assignPublicIp=ENABLED}" \
  --region <your_region>

Configure application instrumentation

Once the Observe Agent is deployed, configure your application instrumentation or set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable to one of the following addresses to send application telemetry including traces to the Observe Agent.

Note

When setting up the endpoint to send traces, make sure you use the path that your OTLP library requires. Some libraries need traces to go to /v1/traces, while others expect them at the root path /.

  • OTLP/HTTP endpoint: http://localhost:4318

  • OTLP/grpc endpoint: http://localhost:4317

Learn more about how to instrument your app

If your application is not able to reach the OTLP endpoints above, register the observe-agent-ecs-fargate ECS service in AWS Cloud Map

Next steps

Use the Metric Explorer to monitor your systems. To analyze your trace data, explore both the Trace Explorer and the Service Explorer