merge_events#

Type of operation: Aggregate, Metadata

Description#

Merge consecutive events into new events based on merge conditions. The ‘matchFirstRow’ condition must be true for any event that should start a new event rather than getting merged with the previous event.

If groupby is not specified, the default grouping will be used. The default grouping for merge_events is the set of primary key columns.

If order_by is specified, it will be used to sort the events before merging them. If order_by is not specified, the default ordering will be used. The default ordering for merge_events is the timestamp of the event.

Rows that cannot either start a group or be merged with an existing group are dropped by default. Set the passthrough option to true to keep those rows in the resulting output. The value of the aggregate column for each passed-through row is set to the result of applying the aggregate function to the singleton row.

Usage#

merge_events matchFirstRow, [ options ], aggregateExpression_1, aggregateExpression_2, ..., [ order_by ], [ groupby ]

Argument

Type

Optional

Repeatable

Restrictions

matchFirstRow

expression

no

no

none

options

options

yes

no

constant

aggregateExpression

expression

no

yes

none

order_by

ordering

yes

no

constant

groupby

grouping

yes

no

constant

Options#

Option

Type

Meaning

max_size

int64

Maximum number of events merged into one event

max_interval

duration

Maximum interval between events to be considered for merging

passthrough

bool

Turn on passthrough for rows that cannot either start a merge group or be merged with an existing group. Default is false.

Accelerable#

merge_events is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.

Examples#

merge_events match_regex(message, /^[^\s]/),  options(max_size: 10, max_interval: 10m),  message:string_agg(message, "\n"), order_by(timestamp), group_by(component)

Merges log messages into groups where the first message of each group must start with a non-whitespace character and the size of each group cannot exceed 10 messages or 10 minutes in time.

merge_events match_regex(message, /^[^\s]/), options(passthrough: true), message:string_agg(message, "\n"), order_by(timestamp), group_by(component)

This statement merges log messages where the first message of each group must start with a non-whitespace character, passing through rows that cannot either start a new group or be merged with an existing group. The value of the aggregate column (message) for each passed-through row is set to the result of the aggregate function (string_agg) applied to the singleton row.