tokenize

Description

Splits the string into an array based on separator, which is treated as a set of characters. The default separator is ” “.

Return type

array

Domain

This is a scalar function (calculates a single output value for a single input row.)

Categories

• Semistructured

• String

Usage

tokenize( value [ , separator ] )

Argument	Type	Required	Multiple	Constant
value	string	Required	Only one	Variable
separator	string	Optional	Only one	Constant

Examples

tokenize("a b c d")

Returns [“a”, “b”, “c”, “d”].

tokenize("[email protected]", "@.")

Returns [“hello”, “example”, “com”].