tokenize#

Description#

Splits the string into an array based on separator, which is treated as a set of characters.

The default separator is “ “.

Return type#

array

Domain#

This is a scalar function (calculates a single output value for a single input row.)

Categories#

Usage#

tokenize(value, [ separator ])

Argument

Type

Optional

Repeatable

Restrictions

value

string

no

no

none

separator

string

yes

no

constant

Examples#

tokenize("a b c d")

Returns [“a”, “b”, “c”, “d”].

tokenize("[email protected]", "@.")

Returns [“hello”, “example”, “com”].