tokenize

Description

Splits the string into an array based on separator, which is treated as a set of characters.

The default separator is “ “.

Return type

array of string

Domain

This is a scalar function (calculates a single output value for a single input row.)

Categories

Usage

tokenize(value, [ separator ])

Argument

Type

Optional

Repeatable

Restrictions

value

string

no

no

none

separator

string

yes

no

constant

Examples

tokenize("a b c d")

Returns [“a”, “b”, “c”, “d”].

tokenize("[email protected]", "@.")

Returns [“hello”, “example”, “com”].