merge_events¶

Description¶

Merge consecutive events into new events based on merge conditions. The ‘matchFirstRow’ condition must be true for any event that should start a new event rather than getting merged with the previous event.

If groupby is not specified, the default grouping will be used. The default grouping for merge_events is the set of primary key columns.

If order_by is specified, it will be used to sort the events before merging them. If order_by is not specified, the default ordering will be used. The default ordering for merge_events is the timestamp of the event.

Rows that cannot either start a group or be merged with an existing group are dropped by default. Set the passthrough option to true to keep those rows in the resulting output. The value of the aggregate column for each passed-through row is set to the result of applying the aggregate function to the singleton row.

Usage¶

merge_events matchFirstRow, [ options ], aggregateExpression_1, aggregateExpression_2, ..., [ order_by ], [ groupby ]

Argument	Type	Optional	Repeatable	Restrictions
matchFirstRow	expression	no	no	none
options	options	yes	no	constant
aggregateExpression	expression	no	yes	none
order_by	ordering	yes	no	constant
groupby	grouping	yes	no	constant

Options¶

Option	Type	Meaning
max_size	int64	Maximum number of events merged into one event
max_interval	duration	Maximum interval between events to be considered for merging
passthrough	bool	Turn on passthrough for rows that cannot either start a merge group or be merged with an existing group. Default is false.

Accelerable¶

merge_events is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.

Examples¶

merge_events match_regex(message, /^[^\s]/),  options(max_size: 10, max_interval: 10m),  message:string_agg(message, "\n"), order_by(timestamp), group_by(component)

Merges log messages into groups where the first message of each group must start with a non-whitespace character and the size of each group cannot exceed 10 messages or 10 minutes in time.

merge_events match_regex(message, /^[^\s]/), options(passthrough: true), message:string_agg(message, "\n"), order_by(timestamp), group_by(component)

This statement merges log messages where the first message of each group must start with a non-whitespace character, passing through rows that cannot either start a new group or be merged with an existing group. The value of the aggregate column (message) for each passed-through row is set to the result of the aggregate function (string_agg) applied to the singleton row.