OPAL Verbs
Verbs process sets of rows. This page lists every OPAL verb grouped by category.
Aggregate Verbs
Aggregate verbs establish an aggregate context: multiple input rows generate fewer, summarizing, output rows.
| Verb | Description |
|---|---|
| aggregate | Aggregates metrics across tag dimensions. |
| align | Align raw metrics onto a time grid (defined by the input period) by aggregating nearby data points together. |
| bucketize | Alias of timechart. |
| dedup | dedup collapses all rows in a dataset with the same timestamp and identical values in specified columns to one row. |
| distinct | Alias of dedup. |
| fill | Fills all the missing data in the query window or fills the missing data ahead of the last bucket by the given frame. |
| histogram | Generate an approximated equi-width histogram for the selected expression. |
| make_reference | Create a reference table by aggregating data over time. |
| make_session | Group events or intervals that are close to each other into sessions, and calculate aggregations over each session. |
| makesession | Deprecated alias of make_session. |
| merge_events | Merge consecutive events into new events based on merge conditions. |
| pivot | The pivot verb rotates a table by transforming rows into columns. |
| reaggregate | Deprecated alias of aggregate. |
| rollup | Rollup raw metrics into aligned metrics. |
| statsby | Calculate statistics of columns with aggregate functions, based on (optional) grouping columns. |
| timechart | Bin (in time) and aggregate point or interval table columns through time, based on (optional) grouping columns. |
| timestats | Aggregate columns at every point in time, based on (optional) grouping columns. |
| unpivot | The unpivot verb rotates a table by transforming a list of columns into rows. |
Filter Verbs
Filter verbs select particular rows from the input row set, emitting the matches as output.
| Verb | Description |
|---|---|
| always | When the verb input is a Resource dataset, always selects all resources that matched the predicate at all times. |
| bottomk | Selects all data for each of the bottom k ranked groups. |
| ever | When the verb input is a Resource dataset, ever selects all resources that at some point matched the predicate. |
| filter | Exclude rows from the input dataset that do not match the given predicate expression. |
| filter_last | Exclude rows from the input dataset that do not match the given predicate expression. |
| limit | Select the first count rows based on the current ordering of the input and filter out the rest. |
| never | When the verb input is a Resource dataset, never selects all resources that at no point matched the predicate. |
| topk | Selects all data for each of the top k ranked groups. |
Join Verbs
Join verbs link multiple datasets together. They may generate more, fewer, or the same number of output rows as input rows, depending on the join condition and verb.
Join cost considerations
Joins are a powerful tool that can increase the amount of data collected and credits consumed. In order to keep your queries efficient, it is useful to check that the join being used is the most efficient one that will work for your use case.
From most expensive to least expensive:
lookuporleftjoinbetween two Resource or Interval datasets- This uses an interval overlapping band-join and an additional temporal left outer join. Prefer join to lookup in this case.
joinbetween two Resource or Interval datasets- This only uses an interval overlapping band-join
lookup,leftjoin, orjoinbetween one Event and one Resource or Interval dataset (with or without window functions)joinorleftjoinbetween two Event datasets with window functions- These all use a point-in-interval band-join
lookup,leftjoin, orjoinbetween two Event datasets (without window functions), or between one Table and another dataset.- These use simple relational joins
| Verb | Description |
|---|---|
| exists | Return the rows from the default dataset that have a match anywhere in the query time window. |
| follow | Return the rows from the additional joined dataset that have a match anywhere in the query time window. |
| follow_not | Return the rows from the joined dataset that do not have a match anywhere in the query time window. |
| fulljoin | Temporal full join, adding new columns in the output dataset. |
| join | Temporal inner join, adding new columns in the output dataset. |
| leftjoin | Temporal left join, adding new columns in the output dataset. |
| lookup | Find matching rows in a dataset, retrieving fields and adding them to the output dataset as new columns. |
| lookup_ip_info | Using a value or column of type IPv4, look up corresponding fields associated with the IP address. |
| merge_event | Deprecated alias of update_resource. |
| mergeevent | Deprecated alias of update_resource. |
| not_exists | Return the rows from the default dataset that do not have a match anywhere in the query time window. |
| surrounding | Outer joins datasets by matching row time to a specified frame as well as column values. |
| union | Create a new dataset consisting of the rows from the main input and each of the arguments. |
| update_resource | Augments the input Resource dataset using events from another dataset. |
Metadata Verbs
Metadata verbs modify the metadata of the dataset and the columns in the dataset, rather than directly acting on the data itself.
Certain metadata verbs may affect how the dataset is presented, such as whether column links are shown, or whether data is packed.
| Verb | Description |
|---|---|
| add_key | Add a candidate key to the output. |
| addfk | Deprecated alias of set_link. |
| addkey | Deprecated alias of add_key. |
| addmetric | Deprecated alias of set_metric. |
| changelog | Deprecated alias of make_event. |
| colenum | Deprecated alias of set_col_enum. |
| colimmutable | Deprecated alias of set_col_immutable. |
| colshow | Deprecated alias of set_col_visible. |
| drop_interface | Removes one or more interface implementations from the current dataset. |
| droptime | Deprecated alias of make_table. |
| fkdrop | Deprecated alias of unset_link. |
| interface | Map fields of this dataset to a pre-defined interface. |
| make_event | Creates an Event dataset from an input Table, Interval, or Resource dataset. |
| make_interval | Creates an Interval dataset from an input Table, Event, or Resource dataset. |
| make_metric | Creates a metric dataset from dataset with precomputed time grid. |
| make_reference | Create a reference table by aggregating data over time. |
| make_resource | Creates a Resource dataset from an input Event or Interval dataset. |
| make_session | Group events or intervals that are close to each other into sessions, and calculate aggregations over each session. |
| make_table | Creates a Table dataset from an input Event, Interval, or Resource dataset. |
| makeresource | Deprecated alias of make_resource. |
| makesession | Deprecated alias of make_session. |
| merge_events | Merge consecutive events into new events based on merge conditions. |
| set_col_enum | Mark specified columns as enumerations, or not, so that the user interface can present them appropriately. |
| set_col_immutable | Mark resource columns as time immutable (time-invariant), or not. |
| set_col_searchable | Include or exclude columns from any future full-text search operation. |
| set_col_visible | Show or hide columns by name. |
| set_label | Declare the 'label' of the output to be the designated column. |
| set_link | Add a foreign key to the output, which identifies the target dataset and columns used to find a target resource. |
| set_metric | Register a metric, with its metadata defined in an options object. |
| set_metric_metadata | Specify the dataset that stores the metadata of the metrics defined in the current dataset. |
| set_pk | Alias of set_primary_key. |
| set_primary_key | Declare the primary key of the output as consisting of one or more named columns. |
| set_timestamp | Declare the timestamp of the output to be the named column, which must have the timestamp type. |
| set_valid_from | Declare the valid from of the output to be the named column, which must be a timestamp column. |
| set_valid_to | Declare the valid to of the output to be the named column, which must be a timestamp column. |
| setlabel | Deprecated alias of set_label. |
| setpk | Deprecated alias of set_primary_key. |
| setvf | Deprecated alias of set_valid_from. |
| setvt | Deprecated alias of set_valid_to. |
| sort | Sort rows in the dataset based on ordering functions applied to column fields or column name provided as arguments. |
| timeshift | Shifts the timestamps of rows ahead in time by the specified interval. |
| unset_all_links | Drop all outbound links without having to specify the individual labels. |
| unset_keys | Remove the primary and candidate key declarations from the dataset. |
| unset_link | Drop a foreign key by specifying the foreign keys label. |
| unsort | Remove all column sort ordering metadata that may have been inserted by the sort verb. |
Metrics Verbs
Metrics verbs create, aggregate, or normalize time series values. They are often used with aggregate functions.
| Verb | Description |
|---|---|
| addmetric | Deprecated alias of set_metric. |
| aggregate | Aggregates metrics across tag dimensions. |
| align | Align raw metrics onto a time grid (defined by the input period) by aggregating nearby data points together. |
| make_metric | Creates a metric dataset from dataset with precomputed time grid. |
| reaggregate | Deprecated alias of aggregate. |
| rollup | Rollup raw metrics into aligned metrics. |
| set_metric | Register a metric, with its metadata defined in an options object. |
| timeshift | Shifts the timestamps of rows ahead in time by the specified interval. |
Projection Verbs
Projection verbs create, remove, or rename columns in a dataset.
| Verb | Description |
|---|---|
| coldrop | Deprecated alias of drop_col. |
| colmake | Deprecated alias of make_col. |
| colpick | Deprecated alias of pick_col. |
| colregex | Deprecated alias of extract_regex. |
| colrename | Deprecated alias of rename_col. |
| drop_col | Exclude one or more columns from the input dataset to the output dataset. |
| extract_regex | Add one or more columns by matching capture names in a regular expression against a given source expression. |
| make_col | Add one or more new columns from the input dataset to the output dataset. |
| pick_col | Exclude all columns except the specified columns from the input dataset to the output dataset. |
| rename_col | Renames a column. |
Semistructured Verbs
Semistructured verbs act upon, or produce, arrays or objects that are treated as scalars. E.g., a single column may contain multiple values in an array, or multiple key/value tuples in an object.
The values inside semi-structured containers are stored as variants, and must be cast to the type you expect them to have. If the array or object contains a value of an unexpected type, the cast returns null.
| Verb | Description |
|---|---|
| colregex | Deprecated alias of extract_regex. |
| extract_regex | Add one or more columns by matching capture names in a regular expression against a given source expression. |
| flatten | Given an object or array input, recursively flatten all child elements into key-value columns, with null intermediates. |
| flatten_all | Given an object or array input, recursively flatten all child and intermediate elements into key-value columns. |
| flatten_leaves | Given an object or array input, recursively flatten all child elements into key-value columns, returning only leaf values. |
| flatten_single | Given an object or array input, flatten the first level of child elements into key-value columns. |
| flattenall | Deprecated alias of flatten_all. |
| flattenleaves | Deprecated alias of flatten_leaves. |
| flattensingle | Deprecated alias of flatten_single. |
Updated 8 days ago