These steps guide you through building a custom Docker image of the Observe Agent with a configuration file, pushing that image to Amazon ECR, creating an ECS task definition that uses it, and finally running it as a service to collect metrics and traces from your ECS Cluster.

Prerequisites

Before you proceed, verify that the following requirements are met:

• You have AWS CLI installed and configured.
• You have access to an ECS cluster (Fargate).
• You have permissions to create and manage Amazon ECR repositories and CloudWatch log groups.
• Your account has the proper IAM roles for ECS tasks and execution, including permissions for CloudWatch logs and ECR.

Prepare the Observe Agent configuration

Create the observe-agent.yaml configuration file:

# Observe data token (ex: a1b2c3d4e5f6g7h8i9k0:l1m2n3o4p5q6r7s8t9u0v1w2x3y4z5a6)
token: "${TOKEN}"

# Target Observe collection url (ex: https://123456789012.collect.observeinc.com/)
observe_url: "${OBSERVE_URL}"

self_monitoring:
  enabled: true

host_monitoring:
  enabled: false
  logs:
    enabled: false
    include:
  metrics:
    host:
      enabled: false
    process:
      enabled: false

forwarding:
    enabled: true
    metrics:
        output_format: otel
        
otel_config_overrides:
  receivers:
    otlp:
      protocols:
        grpc:
          endpoint: 0.0.0.0:4317
        http:
          endpoint: 0.0.0.0:4318
    awsecscontainermetrics:
      collection_interval: 20s
  service:
    pipelines:
      metrics/ecs_fargate:
        receivers: [awsecscontainermetrics]
        processors: [memory_limiter, resourcedetection, resourcedetection/cloud, batch]
        exporters: [otlphttp/observemetrics]

Build and Push the docker image to Amazon ECR

📘

Note

To use a specific version, visit https://github.com/observeinc/observe-agent/releases.

1. Create the Dockerfile:

FROM observeinc/observe-agent:latest
COPY observe-agent.yaml /etc/observe-agent/observe-agent.yaml

2. Build the Docker image:

docker buildx build --platform linux/amd64 --no-cache -t observe-agent:latest .

3. Create an ECR repository. In the AWS Management Console or via the CLI, create a repository named observe/observe-agent-ecs-fargate. The repository URI will look like this:

<your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe/observe-agent-ecs-fargate

4. Push the image to ECR:

aws ecr get-login-password --region <your_region> | docker login --username AWS --password-stdin <your_account_id>.dkr.ecr.<your_region>.amazonaws.com
docker tag observe-agent:latest <your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe/observe-agent-ecs-fargate:latest
docker push <your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe/observe-agent-ecs-fargate:latest

Create a CloudWatch log group for the Observe Agent

Before running the task, ensure a CloudWatch log group exists for your agent logs:

aws logs create-log-group --log-group-name /aws/ecs-fargate/observe/observe-agent --region <your_region>

Create the ECS task definition

Update the fields below with your values:

📘

Note

Some Observe instances may optionally use a name instead of Customer ID; if this is the case for your instance, contact your Observe Data Engineer to discuss implementation. A stem name will work as is, but a DNS redirect name may require client configuration.

Make sure the execution role has logs:CreateLogStream and logs:PutLogEvents permissions.

{
  "family": "observe-ecs-fargate-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "taskRoleArn": "arn:aws:iam::<your_account_id>:role/<your_ecs_task_role>",
  "executionRoleArn": "arn:aws:iam::<your_account_id>:role/<your_ecs_execution_role>",
   
  "containerDefinitions": [
    {
      "name": "observe-agent",
      "image": "<your_account_id>.dkr.ecr.<your_region>.amazonaws.com/observe-agent-ecs-fargate:latest",
      "essential": true,

      "environment": [
        {"name": "TOKEN", "value": "<YOUR_INGEST_TOKEN>"},
        {"name": "OBSERVE_URL", "value": "<YOUR_OBSERVE_COLLECTION_ENDPOINT>"}
      ],

      "portMappings": [
        { "containerPort": 4317, "protocol": "tcp" },
        { "containerPort": 4318, "protocol": "tcp" }
      ],

      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/aws/ecs-fargate/observe/observe-agent",
          "awslogs-region": "<your_region>",
          "awslogs-stream-prefix":"observe-agent"
        }
      }
    }
  ]
}

Register this task definition using the AWS CLI:

aws ecs register-task-definition --cli-input-json file://observe-agent-ecs-fargate-task-definition.json --region <your_region>

Create an ECS service

• SUBNET_A: Subnet ID #1 in the same VPC as your cluster. Usually one of your public subnets (e.g., subnet-047b48528c1c7d5ab in us-west-2b). Every Fargate task gets an ENI (elastic network interface) inside one subnet. Providing at least two subnets in different AZs lets ECS balance tasks across zones for high availability.
• SUBNET_B: Subnet ID #2 in a different AZ (e.g., subnet-09b14d2367d0dc917 in us-west-2c). If AZ-A has an outage, ECS can still launch the task in AZ-B without needing a new service deployment (Cross-AZ redundancy)
• SG_APP: Security group ID that the task ENI will use (e.g., sg-0123456789abcdef0). The security group is the EC2-level firewall for the task ENI. It defines exactly which ports the Observe Agent (4317/4318) can receive, and allows egress to the Observe SaaS endpoint (TLS/443). Without it, the task would default to the VPC’s “default” SG, often too open or too closed. Typical rules: Inbound TCP 5000 (demo-app), Inbound TCP 4317/4318 (optional), Outbound TCP 443 (HTTPS to Observe).
• assignPublicIp=ENABLED gives each task its own public IP so it can send
  data straight to Observe without a NAT. If you deploy inside private subnets instead, flip the flag to DISABLED and rely on NAT or VPC endpoints.

📘

Note

If you want private-only networking, point SUBNET_A/B at private subnets and set assignPublicIp=DISABLED. Make sure those subnets have a NAT or VPC-endpoint that can reach *.collect.observeinc.com.

# us-west-2b (public)
# SUBNET_A=subnet-047b48528c1c7d5ab
# us-west-2c (public)   
# SUBNET_B=subnet-09b14d2367d0dc917
# allows inbound 5000/tcp, outbound 443/tcp   
# SG_APP=sg-0123456789abcdef0         

aws ecs create-service \
  --cluster <your_ecs_cluster_name> \
  --service-name observe-agent-ecs-fargate \
  --task-definition observe-ecs-fargate-task \
  --desired-count 1 \
  --launch-type FARGATE \
  --network-configuration \
    "awsvpcConfiguration={subnets=[$SUBNET_A,$SUBNET_B],securityGroups=[$SG_APP],assignPublicIp=ENABLED}" \
  --region <your_region>

Send application data to Observe

Once the Observe Agent is deployed, configure your application instrumentation or set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable to one of the following addresses to send application telemetry including traces to the Observe Agent.

📘

Note

When setting up the endpoint to send traces, make sure you use the path that your OTLP library requires. Some libraries need traces to go to /v1/traces, while others expect them at the root path /.

• OTLP/HTTP endpoint: http://localhost:4318
• OTLP/grpc endpoint: http://localhost:4317

See APM instrumentation for more information about how to instrument your app for observability.

If your application is not able to reach the OTLP endpoints above, register the observe-agent-ecs-fargate ECS service in AWS Cloud Map