Add and delete attributes

Add attributes

You can add additional attributes to your telemetry data by leveraging the Kubernetes Attributes Processor. For example, if you want to insert an attribute named env with the value latest across entity, logs, metrics, and traces, follow the steps below.

📘
Note
To view all available Agent Chart versions, run: helm search repo observe --versions | grep observe/agent. If you’re currently using Agent Chart versions 0.38 through 0.40, upgrade to version 0.41 or later.

Create a file named add-attribute-values.yaml with the following contents:

agent:
  config:
    # 1) Define an anchor for the repeated configuration (must appear before references to it).
    attributes_custom_base: &attributes_custom_base
      actions:
        - key: 'env'
          action: insert
          value: 'latest'

    nodeLogsMetrics:
      processors:
        # 2) Use the alias instead of repeating the config
        attributes/custom: *attributes_custom_base
      service:
        pipelines:
          # node-logs-metrics
          logs:
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_pod_logs, attributes/custom]
          metrics/hostmetrics:
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_hostmetrics, attributes/custom]
          metrics/kubeletstats:
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_kubeletstats_metrics, attributes/custom]

    clusterEvents:
      processors:
        attributes/custom: *attributes_custom_base
      service:
        pipelines:
          logs/cluster:
            processors: [memory_limiter, batch, resource/observe_common, filter/cluster, transform/cluster, attributes/custom]
          logs/objects:
            processors: [memory_limiter, batch, resource/observe_common, transform/unify, observek8sattributes, transform/object, attributes/custom]
         
    clusterMetrics:
      processors:
        attributes/custom: *attributes_custom_base
      service:
        pipelines:
          metrics/pod_metrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_pod_metrics, attributes/custom]
          metrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_cluster_metrics, attributes/custom]
 
    monitor:
      processors:
        attributes/custom: *attributes_custom_base
      service:
        pipelines:
          metrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_agent_monitor, attributes/custom]

    forwarder:
      processors:
        attributes/custom: *attributes_custom_base
      service:
        pipelines:
          # observe-forward
          traces/observe-forward:
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_app_traces, attributes/custom]
          logs/observe-forward:
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_app_logs, attributes/custom]
          metrics/observe-forward:
            processors: [memory_limiter, k8sattributes, batch, resourcedetection/cloud, resource/observe_common, attributes/debug_source_app_metrics, attributes/custom]

Redeploy the Observe Agent with the updated config.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values add-attribute-values.yaml

Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe

Delete attributes

You can also delete attributes by leveraging the Attributes Processor. For example, if you want to remove kubernetes_namespace and kubernetes_pod_name attributes (under both attributes and resource.attributes), follow these steps:

Create a file named delete-attribute-values.yaml with the following contents:

agent:
  config:
    # 1) Define anchors for common blocks
    attributes_metrics_base: &attributes_metrics_base
      actions:
        - key: 'kubernetes_namespace'
          action: delete
        - key: 'kubernetes_pod_name'
          action: delete

    resource_metrics_base: &resource_metrics_base
      attributes:
        - key: 'kubernetes_namespace'
          action: delete
        - key: 'kubernetes_pod_name'
          action: delete

    nodeLogsMetrics:
      processors:
        # 2) Use aliases instead of duplicating configuration
        attributes/metrics: *attributes_metrics_base
        resource/metrics: *resource_metrics_base
      service:
        pipelines:
          logs:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_pod_logs]
          metrics/hostmetrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_hostmetrics, attributes/metrics, resource/metrics]
          metrics/kubeletstats:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_kubeletstats_metrics, attributes/metrics, resource/metrics]

    clusterEvents:
      processors:
        attributes/metrics: *attributes_metrics_base
        resource/metrics: *resource_metrics_base
      service:
        pipelines:
          logs/cluster:
            processors: [memory_limiter, batch, resource/observe_common, filter/cluster, transform/cluster]
          logs/objects:
            processors: [memory_limiter, batch, resource/observe_common, transform/unify, observek8sattributes, transform/object]

    clusterMetrics:
      processors:
        attributes/metrics: *attributes_metrics_base
        resource/metrics: *resource_metrics_base
      service:
        pipelines:
          metrics/pod_metrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_pod_metrics, attributes/metrics, resource/metrics]
          metrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_cluster_metrics, attributes/metrics, resource/metrics]

    monitor:
      processors:
        attributes/metrics: *attributes_metrics_base
        resource/metrics: *resource_metrics_base
      service:
        pipelines:
          metrics:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_agent_monitor, attributes/metrics, resource/metrics]

    forwarder:
      processors:
        attributes/metrics: *attributes_metrics_base
        resource/metrics: *resource_metrics_base
      service:
        pipelines:
          traces/observe-forward:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_app_traces]
          logs/observe-forward:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_app_logs]
          metrics/observe-forward:
            processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_app_metrics, attributes/metrics, resource/metrics]

Redeploy the Observe Agent with the updated config.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values delete-attribute-values.yaml

Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe

Updated about 2 months ago