Correlation tags are a flexible way to link data in Observe to support pivoting, or correlating, data. You can connect logs, metrics, traces and other data in Observe regardless of the underlying data format. It’s a way to say, “This column in this Dataset describes the same stuff as that column in that Dataset.” When multiple Datasets have columns (or nested fields within JSON objects) tagged with the same correlation tag, you can pivot between them by filtering them by the same common concept.

Benefits of correlation tags

Correlation tags offer an easier and more flexible way to link data in Observe compared to Resources .

Correlation tags have the following benefits:

• Can be added via the UI without writing OPAL.
• Can be added to app-installed content.
• Can be added to nested fields within JSON, such as attributes.name.
• Can be added and removed without triggering re-materialization or an increase in transform costs.
• Can help Observe's AI features work better. For example, it's necessary for AI SRE to be able to map an input such as apiserver to the correct Dataset and field in order to provide accurate answers.

Add correlation tags to a Dataset

A correlation tag is a piece of metadata added to a Dataset, either automatically by the Observe system or manually by a user. Each correlation tag consists of a name and a path within a Dataset, which can either be a column or a nested path within JSON.

📘

Note

Only Observe admins have permissions to add correlation tags to Datasets.

Correlation tags can be added to Datasets via the Dataset edit page. Below is an example of how to add a correlation tag:

1. Click Datasets in the left navigation.

2. Find the Dataset you want to edit, hover on the Dataset name and click the Open Dataset page icon ().

   

3. Click the Properties tab.

4. Click Correlation tags.

5. Click New tag.

6. Select the column and tag you want to use to create the correlation tag. In this example, we add the correlation tag k8s.cluster.name and attach it to the cluster column in the Kubernetes Logs:

   

7. Click Add to list.

You are taken back to the list of tags, where you can see your new tag in the table.

Tag inheritance

After you create a new tag, you are taken back to the list of tags, where you can see your new tag in the table. Here, we see the tag k8s.cluster.name appears in the table:

The Origin column tells you about the origination of each tag. Tags created manually from the Dataset, as described on this page, have the Local label in the Origin column. A tag such as resource_attributes."deployment.environment" in this example comes from a parent Dataset and was inherited to this Dataset.

When a child Dataset is derived from a parent Dataset, if any columns remain unchanged between the parent and child Datasets, any correlation tags defined on such columns get inherited to the child Dataset. For example, suppose you have a parent Dataset with the following columns:

And you have the following tags:

• k8s.cluster.name on cluster
• k8s.namespace.name on path attributes.namespace

Below is a mapping from child Dataset definitions and tags that would be inherited:

Delete a tag

You can hover and click the delete icon () to delete any local tag. Inherited tags can only be deleted from the origin Dataset.

Create correlation tag parameters in dashboards

After adding a correlation tag, you can pivot via the pivot context menu. Using the pivot menu, users can pivot to related logs, metrics, and traces.

In this example, when we return to the Log Explorer and view the Kubernetes Logs, we can pivot on the cluster column. Adding the k8s.cluster.name tag and attaching it to the cluster column enables the pivots highlighted below:

See Create a correlation tag dashboard parameter for an example of how to add a correlation tag parameter to a dashboard, so that the dashboard can be destination for pivots from the Log Explorer.

Log correlation using correlation tags

Correlation tags provide the ability to query across multiple log sources that shared common attributes, such as IDs. In other observability systems, this capability is known as log subqueries.

For example, if two teams submit logs to Observe containing a service name in two different naming formats, such as serviceid or one and service_name for the other, you can add a correlation tag such as service.name to these two fields to ensure that users can run queries across both log Datasets.

To select tags to use in your queries, click in the filter bar

Link applications logs to the Service Explorer

Our native Service Explorer is a core part of the Observe application performance monitoring (APM) offering.

You can link custom application logs to the Service Explorer by simply adding the service.name correlation tag to a column or path to any log Dataset.

📘

Note

You must be on version 2.2.0 or higher of the OTEL app for this functionality to work.

Automatic tagging

Observe automatically tags data coming from the Observe Agent with relevant tags following the Open Telemetry Semantic Convention for resources.