set_valid_from

Type of operation: Metadata

Description

Declare the valid from of the output to be the named column, which must be a timestamp column. Beware changing time to a field that is too far off from the current timestamp field, because it may end up falling outside of the processing time window. Rows where the difference between the old and new valid from timestamps in either direction exceeds a configurable parameter max_time_diff are excluded from the output.

Alternatively, a frame can be used instead of the max_time_diff option to specify different values for the allowed time range in either direction. In case of event datasets, any row with a new timestamp value that lies outside this frame around its original timestamp is not included in the output. In the case of resource or interval datasets, any row with a new timestamp value that is earlier than its original timestamp beyond frame.back is not included in the output.

Specifying both frame and max_time_diff option is not allowed. If neither is specified, a default value of 1 hour is assumed for max_time_diff.

Also, in the case of resource of interval datasets, setting a valid from that is after the valid to time will cause the row to be filtered out.

The valid from value must be a column that already exists, not an expression, and must be a timestamp type, so you may need to re-shape columns with make_col before you use this verb.

If you use set_valid_from and see no data, chances are that the timestamps in the new column are too far away from the previous time column value, and you need a bigger max_time_diff option.

Usage

set_valid_from [ options ], [ frame ], columnname

Argument	Type	Optional	Repeatable	Restrictions
options	options	yes	no	constant
frame	frame	yes	no	constant
columnname	timestamp	no	no	column

Options

Option	Type	Meaning
max_time_diff	duration	The new timestamp will be no further away from the old timestamp than this (default 1h)

Accelerable

set_valid_from is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.

Examples

set_valid_from ts_col

Sets the valid from designation of the output dataset as the ts_col field.

set_valid_from options(max_time_diff: 1h), ts_col

Sets the valid from designation of the output dataset as the ts_col field, and the maximum time difference between the original valid from field and ts_col is less than one hour.

set_valid_from frame(back: 2h, ahead:20m), ts_col

Sets the valid from designation of the output dataset as the ts_col field, and filters the output to only contain rows where ts_col is at most 2 hours behind or 20 minutes ahead of the original valid from field.

Aliases

• setvf (deprecated)