Observe Performance Cookbook: Extract from JSON instead of using Flatten

Problem

A JSON flattening operation is taking a long time or using a lot of query credits.

Solution

Use Observe’s built-in ability to extract a value directly from JSON instead of flattening the object at all. For example, make_col city:string(geo.city) is more efficient than flattening the entire geo object to extract city.

Explanation

The flatten, flatten_all, flatten_leaves, and flatten_single OPAL verbs perform more work than extracting a single field requires. They recurse the JSON object’s structure to create key-value pairs.