Mask Sensitive Data

You can scrub personally identifiable information (PII) from your logs by leveraging the Transform Processor.

1. Create a file named mask-sensitive-data-values.yaml with the following contents:

agent:
  config:
    # ─── Shared anchors ────────────────────────────────────────────────────
    # Full PII-mask processor definition
    pii_mask_def: &pii_mask_def
      transform/pii_mask:
        error_mode: ignore
        log_statements:
          - context: log
            statements:
              # Passwords
              - 'replace_pattern(body, "password=\\S+", "password=********")'
              # Credit-card numbers
              - 'replace_pattern(body, "creditcard=\\d{4}-\\d{4}-\\d{4}-\\d{4}", "creditcard=XXXX-XXXX-XXXX-XXXX")'
              # U.S. SSNs
              - 'replace_pattern(body, "ssn=\\d{3}-\\d{2}-\\d{4}", "ssn=XXX-XX-XXXX")'
              # Bearer / JWT tokens
              - 'replace_pattern(body, "bearer=[A-Za-z0-9\\-_.]+", "bearer=<redacted>")'
              # Email addresses
              - 'replace_pattern(body, "email=[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}", "email=<redacted>")'
              # U.S. phone numbers
              - 'replace_pattern(body, "phone=\\d{3}-\\d{3}-\\d{4}", "phone=XXX-XXX-XXXX")'
              # IPv4 addresses
              - 'replace_pattern(body, "ip=\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}", "ip=X.X.X.X")'
              # Simple “First Last” name
              - 'replace_pattern(body, "name=[A-Za-z]+\\s[A-Za-z]+", "name=REDACTED")'

    nodeLogsMetrics:
      processors:
        <<: *pii_mask_def
      service:
        pipelines:
          logs:
            processors: [memory_limiter, k8sattributes, resourcedetection/cloud, resource/observe_common, attributes/debug_source_pod_logs, transform/pii_mask, batch]

    forwarder:
      processors:
        <<: *pii_mask_def
      service:
        pipelines:
          logs/observe-forward:
            processors: [memory_limiter, k8sattributes, resourcedetection/cloud, resource/observe_common, attributes/debug_source_app_logs, transform/pii_mask, batch]
          

2. Redeploy the Observe Agent.

Run the following command to redeploy the Observe Agent in the observe namespace.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values mask-sensitive-data-values.yaml

3. Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe