How do I avoid the 5,000 row limitation for grouping operations?

When a query uses a grouping operation (such as group_by, statsby, or timechart with group_by), Observe enforces a hard limit of 5,000 rows per group. If any single group exceeds this limit, Observe stops processing additional rows for that group and displays the following warning:

Group limit for a single group reached (5000). Additional results will not be shown

The results for the affected group are incomplete — only the first 5,000 rows are included.

When this occurs

This limit is most commonly encountered in the following situations:

  • Grouping by a high-cardinality dimension (such as request ID, trace ID, or user ID) that produces groups with many rows.
  • Querying over a long time window without sufficient filtering.
  • Using statsby or timechart with groupings that produce large per-group result sets.

How to resolve

To avoid hitting the per-group row limit:

  1. Add filters before grouping. Use filter to narrow the dataset before applying group_by or statsby. This reduces the number of rows in each group.

    filter status_code = 500
    statsby count: count(1), group_by(service_name)
    
  2. Use aggregation instead of raw rows. If you need summary statistics rather than individual rows, use aggregate functions such as count, sum, or avg with statsby or timechart to collapse rows within each group.

  3. Reduce the time window. Shorter query time ranges produce fewer rows per group.

  4. Increase grouping granularity. Add additional group_by columns to split large groups into smaller ones.

  5. Use topk to limit results. If you only need the top results, apply topk to limit the output.