Mask sensitive data

You can scrub personally identifiable information (PII) from your logs by leveraging the Transform Processor.

  1. Create a file named mask-sensitive-data-values.yaml with the following contents:
    agent:
      config:
        # ─── Shared anchors ────────────────────────────────────────────────────
        # Full PII-mask processor definition
        pii_mask_def: &pii_mask_def
          transform/pii_mask:
            error_mode: ignore
            log_statements:
              - context: log
                statements:
                  # Passwords
                  - 'replace_pattern(body, "password=\\S+", "password=********")'
                  # Credit-card numbers
                  - 'replace_pattern(body, "creditcard=\\d{4}-\\d{4}-\\d{4}-\\d{4}", "creditcard=XXXX-XXXX-XXXX-XXXX")'
                  # U.S. SSNs
                  - 'replace_pattern(body, "ssn=\\d{3}-\\d{2}-\\d{4}", "ssn=XXX-XX-XXXX")'
                  # Bearer / JWT tokens
                  - 'replace_pattern(body, "bearer=[A-Za-z0-9\\-_.]+", "bearer=<redacted>")'
                  # Email addresses
                  - 'replace_pattern(body, "email=[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}", "email=<redacted>")'
                  # U.S. phone numbers
                  - 'replace_pattern(body, "phone=\\d{3}-\\d{3}-\\d{4}", "phone=XXX-XXX-XXXX")'
                  # IPv4 addresses
                  - 'replace_pattern(body, "ip=\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}", "ip=X.X.X.X")'
                  # Simple “First Last” name
                  - 'replace_pattern(body, "name=[A-Za-z]+\\s[A-Za-z]+", "name=REDACTED")'
    
        nodeLogsMetrics:
          processors:
            <<: *pii_mask_def
          service:
            pipelines:
              logs:
                processors: [memory_limiter, k8sattributes, resourcedetection/cloud, resource/observe_common, attributes/debug_source_pod_logs, transform/pii_mask, batch]
    
        forwarder:
          processors:
            <<: *pii_mask_def
          service:
            pipelines:
              logs/observe-forward:
                processors: [memory_limiter, k8sattributes, resourcedetection/cloud, resource/observe_common, attributes/debug_source_app_logs, transform/pii_mask, batch]
  2. Run the following command to redeploy the Observe Agent in the observe namespace:
    helm upgrade --reuse-values observe-agent observe/agent -n observe --values mask-sensitive-data-values.yaml
  3. Run the following commands to restart the pods:
    kubectl rollout restart deployment -n observe
    kubectl rollout restart daemonset -n observe