documentationset_valid_from¶
Type of operation: Metadata
Description¶
Declare the valid from of the output to be the named column, which must be a
timestamp column. Beware changing time to a field that is too far off from the
current timestamp field, because it may end up falling outside of the
processing time window. Rows where the difference between the old and new valid from timestamps
in either direction exceeds a configurable parameter max_time_diff are excluded from the output.
Alternatively, a frame can be used instead of the max_time_diff option to specify different values
for the allowed time range in either direction. In case of event datasets, any row with a new timestamp
value that lies outside this frame around its original timestamp is not included in the output.
In the case of resource or interval datasets, any row with a new timestamp value that is earlier than
its original timestamp beyond frame.back
is not included in the output.
Specifying both frame and max_time_diff option is not allowed. If neither is specified,
a default value of 1 hour is assumed for max_time_diff.
Also, in the case of resource of interval datasets, setting a valid from that is after
the valid to time will cause the row to be filtered out.
The valid from value must be a column that already exists, not an expression,
and must be a timestamp type, so you may need to re-shape columns with
make_col before you use this verb.
If you use set_valid_from and see no data, chances are that the timestamps in the new column
are too far away from the previous time column value, and you need a bigger max_time_diff option.
Usage¶
set_valid_from [ options ], [ frame ], columnname
Argument |
Type |
Optional |
Repeatable |
Restrictions |
|---|---|---|---|---|
options |
options |
yes |
no |
constant |
frame |
frame |
yes |
no |
constant |
columnname |
timestamp |
no |
no |
column |
Options¶
Option |
Type |
Meaning |
|---|---|---|
max_time_diff |
duration |
The new timestamp will be no further away from the old timestamp than this (default 1h) |
Accelerable¶
set_valid_from is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.
Examples¶
set_valid_from ts_col
Sets the valid from designation of the output dataset as the ts_col field.
set_valid_from options(max_time_diff: 1h), ts_col
Sets the valid from designation of the output dataset as the ts_col field, and the maximum time difference between the original valid from field and ts_col is less than one hour.
set_valid_from frame(back: 2h, ahead:20m), ts_col
Sets the valid from designation of the output dataset as the ts_col field, and filters the output to only contain rows
where ts_col is at most 2 hours behind or 20 minutes ahead of the original valid from field.
Aliases¶
setvf(deprecated)