Handle multiline log records

❗️
Important
Enable either autoMultilineDetection or a Recombine operator using is_first_entry—never both at the same time.

📘
Note
The File Log Receiver flushes every 5 seconds by default. The 5 seconds is measured from when a log line matching either the default auto-detection regex or the provided is_first_entry regex is detected. Lines emitted outside that window are stored as separate records.

Automatic detection

Agent Chart 0.62+

When autoMultilineDetection is enabled, the agent groups related log lines into a single entry whenever the first line starts with one of these timestamp formats:

YYYY-MM-DD HH:MM:SS → 2025-06-16 13:45:30
YYYY-MM-DDTHH:MM:SS.sssZ → 2025-06-16T14:33:53.123456Z
Mon DD HH:MM:SS → Jun 16 20:16:01
YYYY/MM/DD HH:MM:SS → 2025/06/16 20:46:15

Create a file named auto-multiline-detection-values.yaml with the following contents:

node:
  containers:
    logs:
      autoMultilineDetection: true
      enabled: true

Redeploy the Observe Agent.

Run the following command to redeploy the Observe Agent in the observe namespace.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values auto-multiline-detection-values.yaml

Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe

Custom pattern with the recombine operator

Agent Chart 0.62+

autoMultilineDetection covers the most common timestamp formats, but some applications use different markers to signal the start of a log entry.

When that happens, you can switch to the Recombine Operator and define your own rule with the is_first_entry setting. The operator tells the Observe Agent to begin a new record whenever a line matches the pattern you provide.

Suppose your log lines start with [:

[2025-06-18T18:52:24.089689Z] INFO: User login successful. User ID: 92065
[2025-06-18T18:52:24.561449Z] INFO: User login successful. User ID: 65749
[2025-06-18T18:52:25.309899Z] DEBUG: Starting backup process.
Directory: /data/backup
Estimated files: 556
[2025-06-18T18:52:26.226822Z] DEBUG: Starting backup process.
Directory: /data/backup
Estimated files: 446
[2025-06-18T18:52:27.538841Z] ERROR: Failed to load configuration file.
File path: /etc/app/config.yaml
Cause: FileNotFoundError
Stack trace:
File "/app/main.py", line 23, in load_config
config = open(config_path, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/etc/app/config.yaml'
[2025-06-18T18:52:29.004533Z] ERROR: Failed to load configuration file.
File path: /etc/app/config.yaml
Cause: FileNotFoundError
Stack trace:
File "/app/main.py", line 23, in load_config
config = open(config_path, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/etc/app/config.yaml'
[2025-06-18T18:52:30.190177Z] INFO: User login successful. User ID: 30051

Create a file named multiline-detection-values.yaml with the following contents:

node:
  containers:
    logs:
      autoMultilineDetection: false
      enabled: true

agent:
  config:
    nodeLogsMetrics:
      receivers:
        filelog:
          operators:
            - id: container-parser
              max_log_size: 102400
              type: container
            # Recombine lines until the next one that starts with "["
            - id: multiline-recombine
              type: recombine
              combine_field: body
              is_first_entry: body matches "^\\["

Redeploy the Observe Agent.

Run the following command to redeploy the Observe Agent in the observe namespace.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values multiline-detection-values.yaml

Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe

Updated about 2 months ago