flatten_all

Type of operation: Semistructured

Description

Given an object or array input, recursively flatten all child and intermediate elements into key-value columns.

The key and value child and intermediate element columns are named ‘c_NAME_path’ and ‘_c_NAME_value’. NAME is replaced with the original column name.

Flatten_all is a more expensive operation than flatten_leaves or flatten_single. The default is to not suggest column types (‘suggesttypes’ = ‘false’.) See also flatten, flatten_leaves, and flatten_single.

Usage

flatten_all pathexpression, [ suggesttypes ]

Argument	Type	Optional	Repeatable	Restrictions
pathexpression	variant	no	no	column
suggesttypes	bool	yes	no	none

Accelerable

flatten_all is always accelerable if the input is accelerable. A dataset that only uses accelerable verbs can be accelerated, making queries on the dataset respond faster.

Examples

flatten_all foo

Given this JSON in column foo:

foo
{"a":{"aa":1},"b":{"bb":[{"bb1":2},{"bb2":3}]}}

flatten_all produces:

_c_foo_path	_c_foo_value
b.bb[1].bb2	3
a.aa	1
b.bb[0].bb1	2
a	{"aa":1}
b.bb[1]	{"bb2":3}
b	{"bb":[{"bb1":2},{"bb2":3}]}
b.bb[0]	{"bb1":2}
b.bb	[{"bb1":2},{"bb2":3}]

It recurses the JSON object and produces new columns that contain every possible path and its corresponding value. Column ‘foo’ will be removed.

flatten_all foo, true

Given this JSON in column foo:

foo
{"a":{"aa":1},"b":{"bb":[{"bb1":2},{"bb2":3}]}}

flatten_all produces:

_c_foo_path	_c_foo_value	_c_foo_type
b.bb[1].bb2	3	int64
a.aa	1	int64
b.bb[0].bb1	2	int64
a	{"aa":1}	object
b.bb[1]	{"bb2":3}	object
b	{"bb":[{"bb1":2},{"bb2":3}]}	object
b.bb[0]	{"bb1":2}	object
b.bb	[{"bb1":2},{"bb2":3}]	array

It recurses the JSON object and produces new columns that contain every possible path and its corresponding value. It will also attempt to determine the value’s type, creating a third (hidden) column named ‘_c_foo_type’. Column ‘foo’ will be removed.

Aliases

• flattenall (deprecated)