Observe Performance Cookbook: Prefer Timechart to Timestats

Problem

Using timestats in explorers or worksheets that use Resources or Intervals performs more slowly than expected.

Solution

Using the OPAL editor, change timestats usage to timechart and re-verify that the use cases are still working as desired.

Explanation

Similar to the exact versions of approximate verbs, the timestats verb counts precisely the number of matching elements in each bucket of time. For instance, if you have a series of user requests modeled as Intervals and you want to count precisely how many concurrent requests are occurring at any given time, timestats is the correct tool.

However, in many cases you may not need the precision to state that a set of transactions is occurring “at any given time”. If the use case can be met with bucketized events and a computation of how many requests are extant in each bucket of time, timechart is a more efficient choice.

The timechart verb is particularly useful when combined with a join verb. For instance, you may want to lookup a resource from an event dataset. If the resource has many intervals (due to a high rate of change), the band-join inside lookup can be expensive. Applying a timechart to the resource to bucketize it turns the band-join into a regular join, because the output of timechart falls onto a time grid. Note that this approach loses the accuracy of presenting exact value at each event time, but instead presents approximate values for each event time,

Example:

@grid_resource <- @resource {
  timechart 10s, value:any_not_null(value), group_by(key)
}

lookup on(key=@grid_resource.key), resource_value:@grid_resource.value

Note: Changing the Resource dataset definition might also be warranted for a high rate of resource attribute change.