filter

Type of operation: Filter

Description

Exclude rows from the input dataset that do not match the given predicate expression.

Types of accepted expressions:

• Boolean: returns True if the row matches or False if not. May include OPAL functions that return bool

• Space-delimited string, enclosed in <>: full text search of “searchable text”, meaning any columns of type string. (Case-insensitive). Optional ~ to specify a single field to search.

• Regular expression, with ~: regex matching must specify a field containing text (including JSON).

Usage

filter predicate

Argument	Type	Required	Multiple
predicate	bool	Required	Only one

Accelerable

filter is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs, can be accelerated, making queries on the dataset respond faster.

Examples

filter string(status_code) ~ /^5.*/

Keep only rows where the status_code column, converted to string, starts with “5”.

filter not method="POST"

Keep only rows where the method column is not equal to the string “POST”

filter string(json_payload.name) ~ /^TEST/

Keep only rows where property name of the JSON field json_payload matches the specified regular expression (begins with “TEST”).

filter count >= 5 and count <= 100

Keep only rows where the count column is between 5 and 100, inclusive.

filter contains(log, "ERROR")

Keep only rows where the log column contains the string “ERROR”. Note that the contains() function is case-sensitive.

filter <foo bar baz>

Keep only rows where any “searchable text” (all columns of type string) contains foo, bar, and baz (case-insensitive).

filter <foo or bar or baz>

Keep only rows where any “searchable text” (all columns of type string) contains any of the following values: foo, bar, or baz (case-insensitive).

filter <foo "or" bar>

Keep only rows where any “searchable text” (all columns of type string) contain foo, bar, and “or” (case-insensitive.) Use this form when you need to match the word “or.”

filter log ~ <error or warning>

Keep only rows where column log contains either “error” or “warning” (case-insensitive).

filter log ~ /^DEBUG/

Keep only rows where the field log matches the specified regular expression (begins with “DEBUG”).

filter json_payload.status = <success>

Keep only rows where property status of the JSON field json_payload contains the string “success” (case-insensitive).