flatten#

Type of operation: Semistructured

Description#

Given an object or array input, recursively flatten all child elements into key-value columns, with null intermediates.

The key and value child element columns are named ‘c_NAME_path’ and ‘_c_NAME_value’. NAME is replaced with the original column name.

Flatten is a more expensive operation than flatten_leaves or flatten_single. The default is to not suggest column types (‘suggesttypes’ = ‘false’.) See also flatten_leaves, flatten_single, and flatten_all.

Usage#

flatten pathexpression, [ suggesttypes ]

Argument

Type

Optional

Repeatable

Restrictions

pathexpression

variant

no

no

column

suggesttypes

bool

yes

no

none

Accelerable#

flatten is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.

Examples#

flatten foo

Given this JSON in column foo:

foo

{"a":{"aa":1},"b":{"bb":[{"bb1":2},{"bb2":3}]}}

flatten produces:

_c_foo_path

_c_foo_value

b.bb[1].bb2

3

a.aa

1

b.bb[0].bb1

2

a

null

b.bb[1]

null

b

null

b.bb[0]

null

b.bb

null

It recurses the JSON object to produce new columns that contain every possible path and its corresponding value, with null values for intermediate key paths so the full tree is returned. Column ‘foo’ will be removed.

flatten foo, true

Given this JSON in column foo:

foo

{"a":{"aa":1},"b":{"bb":[{"bb1":2},{"bb2":3}]}}

flatten produces:

_c_foo_path

_c_foo_value

_c_foo_type

b.bb[1].bb2

3

int64

a.aa

1

int64

b.bb[0].bb1

2

int64

a

null

null

b.bb[1]

null

null

b

null

null

b.bb[0]

null

null

b.bb

null

null

It recurses the JSON object to produce new columns that contain every possible path and its corresponding value, with null values for intermediate key paths so the full tree is returned. It will also attempt to determine the value’s type, creating a third (hidden) column named ‘_c_foo_type’. Column ‘foo’ will be removed.