Trace Tail Sampling

Trace tail sampling allows you to reduce the span data sent to Observe while having full control of which traces are dropped.

To enable trace tail sampling, configure the following in your values.yaml file:

# Enable the gateway deployment and trace sampling
gatewayDeployment:
  enabled: true
  traceSampling:
    enabled: true
    # † Uncomment and edit the following section to override the default
    # tail sampling policies.
    #
    # config:
    #   policies:
    #     # Send all spans with status_code error to Observe
    #     - name: retain-all-errors
    #       type: status_code
    #       status_code: {status_codes: [ERROR]}
    #     # Send 10% of all other spans to Observe
    #     - name: sample-probabilistically-10-percent
    #       type: probabilistic
    #       probabilistic: {sampling_percentage: 10}

# Enable RED metrics generation in order to get accurate counts while sampling
application:
  REDMetrics:
    enabled: true

# To scale the gateway, increase the replicaCount
gateway:
  replicaCount: 3

# ‡ Decrease the resources required for the forwarder daemonset since the
# gateway now handles most of the work.
forwarder:
  resources:
    requests:
      cpu: 250m
      memory: 256Mi
    limits:
      memory: 256Mi

†: For more information on the configuration format, see the tail sampling processor documentation.

‡: For more information on resource requirements for the forwarder daemonset, see Tune service resource requests and limits.