OPAL Verbs
Verbs process sets of rows. This page lists every OPAL verb grouped by category.
Aggregate Verbs
Aggregate verbs establish an aggregate context: multiple input rows generate fewer, summarizing, output rows.
| Verb | Description |
|---|---|
| aggregate | Combine aggregate functions over metric columns while collapsing rows to the grouping keys you choose. |
| align | Turns a raw metric-interface dataset into interval rows on a time grid by evaluating aggregate expressions that each pick exactly |
| bucketize | Alias of timechart. |
| dedup | Collapses duplicate rows while preserving the input column layout and dataset kind. |
| distinct | Alias of dedup. |
| fill | Adds rows on an aligned temporal grid wherever a group has no row in an alignment bucket, keeps values from real observations on |
| histogram | Generate an approximated equi-width histogram for the selected expression. |
| make_reference | Builds a compact reference Table by aggregating a temporal input across the full history available in the dataset, materialized so |
| make_session | Group events or intervals that are close to each other into sessions, and calculate aggregations over each session. |
| makesession | Deprecated alias of make_session. |
| merge_events | Merge consecutive events into new events based on merge conditions. |
| pivot | The pivot verb rotates a table by transforming rows into columns. |
| reaggregate | Deprecated alias of aggregate. |
| rollup | Rollup raw metrics into aligned metrics. |
| statsby | Computes aggregate expressions over the input, optionally partitioned by `group_by(. |
| timechart | Bin (in time) and aggregate point or interval table columns through time, based on (optional) grouping columns. |
| timestats | Aggregate columns at every point in time, based on (optional) grouping columns. |
| unpivot | The unpivot verb rotates a table by transforming a list of columns into rows. |
Filter Verbs
Filter verbs select particular rows from the input row set, emitting the matches as output.
| Verb | Description |
|---|---|
| always | Keeps every row whose primary-key group had the predicate evaluate to true on every grouped row inside the chosen time window. |
| bottomk | Retains rows that belong to the lowest-ranked groups in the current query window and adds an int64 _c_rank column giving each |
| ever | Keeps every row whose primary-key group had at least one row where the predicate evaluated to true inside the chosen time window. |
| filter | Exclude rows from the input dataset that do not match the given predicate expression. |
| filter_last | Keeps rows whose boolean predicate is true and drops all others while preserving the input schema (aside from optional internal |
| limit | Select the first count rows based on the current ordering of the input and filter out the rest. |
| never | Keeps every row whose primary-key group had no row where the predicate evaluated to true inside the chosen time window. |
| topk | Retains rows that belong to the highest-ranked groups in the current query window and adds an int64 _c_rank column giving each |
Join Verbs
Join verbs link multiple datasets together. They may generate more, fewer, or the same number of output rows as input rows, depending on the join condition and verb.
Join cost considerations
Joins are a powerful tool that can increase the amount of data collected and credits consumed. In order to keep your queries efficient, it is useful to check that the join being used is the most efficient one that will work for your use case.
From most expensive to least expensive:
lookuporleftjoinbetween two Resource or Interval datasets- This uses an interval overlapping band-join and an additional temporal left outer join. Prefer join to lookup in this case.
joinbetween two Resource or Interval datasets- This only uses an interval overlapping band-join
lookup,leftjoin, orjoinbetween one Event and one Resource or Interval dataset (with or without window functions)joinorleftjoinbetween two Event datasets with window functions- These all use a point-in-interval band-join
lookup,leftjoin, orjoinbetween two Event datasets (without window functions), or between one Table and another dataset.- These use simple relational joins
| Verb | Description |
|---|---|
| exists | Keeps rows from the primary input where the join predicate matches at least one row from another dataset at some time within the |
| follow | Returns rows from the joined dataset (the single additional dataset referenced in the predicates) where the join condition matches |
| follow_not | Returns rows from the joined dataset (the single additional dataset referenced in the predicates) where the join condition matches |
| fulljoin | Performs a temporal full outer join between the primary input and one other dataset: every row from either side appears at |
| join | Performs an inner join between the pipeline input and exactly one other dataset, keeping rows where the join predicate holds and |
| leftjoin | Performs a temporal left outer join between the default input and exactly one other dataset, preserving every left-hand row and |
| lookup | Enriches each input row with columns from a related dataset using a left-outer temporal join, while keeping the input dataset kind |
| lookup_ip_info | Using a value or column of type IPv4, look up corresponding fields associated with the IP address. |
| merge_event | Deprecated alias of update_resource. |
| mergeevent | Deprecated alias of update_resource. |
| not_exists | Returns rows from the primary input where the join predicate matches no row from the additional dataset at any time within the |
| surrounding | Outer joins datasets by matching row time to a specified frame as well as column values. |
| union | Combines the main pipeline input with every referenced additional dataset into one dataset whose rows are the union of all inputs. |
| update_resource | Builds new columns on a Resource dataset by matching each resource row to Event rows on a full primary-key or candidate-key |
Metadata Verbs
Metadata verbs modify the metadata of the dataset and the columns in the dataset, rather than directly acting on the data itself.
Certain metadata verbs may affect how the dataset is presented, such as whether column links are shown, or whether data is packed.
| Verb | Description |
|---|---|
| add_key | Declares an extra candidate key: the listed columns together are treated as identifying a resource instance for metadata and |
| addfk | Deprecated alias of set_link. |
| addkey | Deprecated alias of add_key. |
| addmetric | Deprecated alias of set_metric. |
| changelog | Deprecated alias of make_event. |
| colenum | Deprecated alias of set_col_enum. |
| colimmutable | Deprecated alias of set_col_immutable. |
| colshow | Deprecated alias of set_col_visible. |
| drop_interface | Removes named interface bindings from dataset metadata without changing row data or column definitions. |
| droptime | Deprecated alias of make_table. |
| fkdrop | Deprecated alias of unset_link. |
| interface | Map fields of this dataset to a pre-defined interface. |
| make_event | Creates an Event dataset from an input Table, Interval, or Resource dataset. |
| make_interval | Creates an Interval dataset from an input Table, Event, or Resource dataset. |
| make_metric | Creates a metric dataset from dataset with precomputed time grid. |
| make_reference | Builds a compact reference Table by aggregating a temporal input across the full history available in the dataset, materialized so |
| make_resource | Creates a Resource dataset from an input Event or Interval dataset. |
| make_session | Group events or intervals that are close to each other into sessions, and calculate aggregations over each session. |
| make_table | Converts a non-Table dataset to kind Table by clearing temporal metadata and key constraints while keeping the same data columns. |
| makeresource | Deprecated alias of make_resource. |
| makesession | Deprecated alias of make_session. |
| merge_events | Merge consecutive events into new events based on merge conditions. |
| set_col_enum | Mark specified columns as enumerations, or not, so that the user interface can present them appropriately. |
| set_col_immutable | Mark resource columns as time immutable (time-invariant), or not. |
| set_col_searchable | Include or exclude columns from any future full-text search operation. |
| set_col_visible | Controls whether existing columns are shown or hidden in the UI by updating column visibility metadata on the output schema. |
| set_label | Declare the 'label' of the output to be the designated column. |
| set_link | Declares a foreign key from the current dataset to another bound dataset or stage so resources can be resolved through the |
| set_metric | Register a metric, with its metadata defined in an options object. |
| set_metric_metadata | Points the current metric dataset at another bound dataset that implements the "metric_metadata" interface so type, unit, and |
| set_pk | Alias of set_primary_key. |
| set_primary_key | Declare the primary key of the output as consisting of one or more named columns. |
| set_timestamp | Declare the timestamp of the output to be the named column, which must have the timestamp type. |
| set_valid_from | Declare the valid from of the output to be the named column, which must be a timestamp column. |
| set_valid_to | Declare the valid to of the output to be the named column, which must be a timestamp column. |
| setlabel | Deprecated alias of set_label. |
| setpk | Deprecated alias of set_primary_key. |
| setvf | Deprecated alias of set_valid_from. |
| setvt | Deprecated alias of set_valid_to. |
| sort | Sort rows in the dataset based on ordering functions applied to column fields or column name provided as arguments. |
| timeshift | Shifts each row forward or backward in time by adding a compile-time duration to the valid-from timestamp and, when the dataset |
| timewrap | Replicates the input count times along a timeline, shifting each copy by 0, 1, …, count - 1 multiples of a fixed |
| unset_all_links | Drop all outbound links without having to specify the individual labels. |
| unset_keys | Remove the primary and candidate key declarations from the dataset. |
| unset_link | Drop a foreign key by specifying the foreign keys label. |
| unsort | Clears all column sort ordering metadata that may have been recorded by a prior sort verb. |
Metrics Verbs
Metrics verbs create, aggregate, or normalize time series values. They are often used with aggregate functions.
| Verb | Description |
|---|---|
| addmetric | Deprecated alias of set_metric. |
| aggregate | Combine aggregate functions over metric columns while collapsing rows to the grouping keys you choose. |
| align | Turns a raw metric-interface dataset into interval rows on a time grid by evaluating aggregate expressions that each pick exactly |
| make_metric | Creates a metric dataset from dataset with precomputed time grid. |
| reaggregate | Deprecated alias of aggregate. |
| rollup | Rollup raw metrics into aligned metrics. |
| set_metric | Register a metric, with its metadata defined in an options object. |
| timeshift | Shifts each row forward or backward in time by adding a compile-time duration to the valid-from timestamp and, when the dataset |
| timewrap | Replicates the input count times along a timeline, shifting each copy by 0, 1, …, count - 1 multiples of a fixed |
Projection Verbs
Projection verbs create, remove, or rename columns in a dataset.
| Verb | Description |
|---|---|
| coldrop | Deprecated alias of drop_col. |
| colmake | Deprecated alias of make_col. |
| colpick | Deprecated alias of pick_col. |
| colregex | Deprecated alias of extract_regex. |
| colrename | Deprecated alias of rename_col. |
| drop_col | Removes one or more named columns from the default dataset while preserving every other column, its type, and the dataset kind. |
| extract_regex | Add one or more columns by matching capture names in a regular expression against a given source expression. |
| make_col | Evaluates one or more name:expression bindings per row, inserting new columns or replacing existing columns of the same name |
| pick_col | Projects the pipeline to an explicit list of columns in argument order, binding each output name to an expression and dropping |
| rename_col | Renames existing columns while keeping the full row shape, using `newName:@. |
Semistructured Verbs
Semistructured verbs act upon, or produce, arrays or objects that are treated as scalars. E.g., a single column may contain multiple values in an array, or multiple key/value tuples in an object.
The values inside semi-structured containers are stored as variants, and must be cast to the type you expect them to have. If the array or object contains a value of an unexpected type, the cast returns null.
| Verb | Description |
|---|---|
| colregex | Deprecated alias of extract_regex. |
| extract_regex | Add one or more columns by matching capture names in a regular expression against a given source expression. |
| flatten | Expands a column that holds an object, array, or variant into many rows—one per nested path—while copying the other input |
| flatten_all | Given an object or array input, recursively flatten all child and intermediate elements into key-value columns. |
| flatten_leaves | Given an object or array input, recursively flatten all child elements into key-value columns, returning only leaf values. |
| flatten_single | Given an object or array input, flatten the first level of child elements into key-value columns. |
| flattenall | Deprecated alias of flatten_all. |
| flattenleaves | Deprecated alias of flatten_leaves. |
| flattensingle | Deprecated alias of flatten_single. |
Updated about 24 hours ago