flatten_all

Type of operation: Semistructured

Description

Given an object or array input, recursively flatten all child and intermediate elements into key-value columns.

The key and value child and intermediate element columns are named ‘c_NAME_path’ and ‘_c_NAME_value’. NAME is replaced with the original column name.

Flatten_all is a more expensive operation than flatten_leaves or flatten_single. The default is to not suggest column types (‘suggesttypes’ = ‘false’.) See also flatten, flatten_leaves, and flatten_single.

Usage

flatten_all pathexpression, [ suggesttypes ]

Argument

Type

Optional

Repeatable

Restrictions

pathexpression

array or object

no

no

column

suggesttypes

bool

yes

no

none

Accelerable

flatten_all is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.

Examples

flatten_all foo

Given this JSON in column foo:

foo

{"a":{"aa":1},"b":{"bb":[{"bb1":2},{"bb2":3}]}}

flatten_all produces:

_c_foo_path

_c_foo_value

b.bb[1].bb2

3

a.aa

1

b.bb[0].bb1

2

a

{"aa":1}

b.bb[1]

{"bb2":3}

b

{"bb":[{"bb1":2},{"bb2":3}]}

b.bb[0]

{"bb1":2}

b.bb

[{"bb1":2},{"bb2":3}]

It recurses the JSON object and produces new columns that contain every possible path and its corresponding value. Column ‘foo’ will be removed.

flatten_all foo, true

Given this JSON in column foo:

foo

{"a":{"aa":1},"b":{"bb":[{"bb1":2},{"bb2":3}]}}

flatten_all produces:

_c_foo_path

_c_foo_value

_c_foo_type

b.bb[1].bb2

3

int64

a.aa

1

int64

b.bb[0].bb1

2

int64

a

{"aa":1}

object

b.bb[1]

{"bb2":3}

object

b

{"bb":[{"bb1":2},{"bb2":3}]}

object

b.bb[0]

{"bb1":2}

object

b.bb

[{"bb1":2},{"bb2":3}]

array

It recurses the JSON object and produces new columns that contain every possible path and its corresponding value. It will also attempt to determine the value’s type, creating a third (hidden) column named ‘_c_foo_type’. Column ‘foo’ will be removed.

Aliases

  • flattenall (deprecated)