Observe Datasets and Time¶
Observe collects all data, system and application logs, metrics, and tracing spans, into observations, which then transform into datasets. Datasets are structured with times or time intervals, as well as relations linking to or from other datasets. These relations between different parts of the system provide Observe with superpowers for discovering the hidden meaning in your data.
Datasets¶
A dataset lives within a named project, and also has a name. Project names must be unique for your customer, and dataset names must be unique within their project. When you log into Observe, the Explore page lets you browse the different datasets that exist for your customer ID.
A dataset has a schema, which is a set of named columns and definitions of data types stored in those columns (such as event or resource).
Event Datasets¶
If an incident occurs “at a time” and has a well-defined timestamp, then the dataset is an “event dataset”. Events have a single point in time, and typically link or relate to one or more other tables in the system. For example, “user X logged into system Y at time Z” is an event, which also links to the “user” dataset and the “system” dataset.
Resource Datasets¶
Finally, objects with permanence over time, and whose state changes over time, are stored in resource datasets. Any field value for a resource has a valid time interval — a start time, and an end time. For a resource, you can ask questions like “what was the name at time T?” Additionally, a primary key identifies a resource.