Source Datasets store the data that Observe by Snowflake ingests from your applications and infrastructure. Use the Source Datasets page to find your data, check ingest activity, and manage dataset settings.

How ingestion works

The Source Dataset workflow separates storage, authentication, and routing:

Your client sends data to a supported ingest endpoint with an ingest token. Observe by Snowflake authenticates the request and determines its destination by matching it to routing rules. It then applies the Destination Dataset's drop filters before storing the remaining data.

Find your source datasets

Select Data & integrations > Source Datasets. Some accounts show this list in the Source datasets tab of Add data.

Use the search bar and filters to narrow the list. You can filter by Created by, Package, Health, and Type. Accounts with data lakes enabled also provide a Data Lake filter.

The list includes these fields:

When data lakes are enabled, the Data lake column shows where the Dataset stores its data.

Check Last received alongside Bytes/hour and Observations/hour to confirm that data is arriving. If the Dataset shows an error, see Troubleshoot data ingestion.

Choose a data type

Type-specific source datasets provide a schema for a particular telemetry format. The list can include these types:

Choose a type compatible with your client's ingest endpoint. For new typed OpenTelemetry ingestion, use /v2/otel and the matching Source Dataset type. An OpenTelemetry Collector appends the signal suffix to this base URL; direct HTTP requests need the complete path shown above. See OpenTelemetry before changing existing /v1/otel clients.

You can't change a Source Dataset's type after creation. The Create source dataset dialog offers OpenTelemetry logs, OpenTelemetry metrics, Prometheus metrics, and generic data. Use the relevant integration setup to provision trace and entity Datasets.

Create a source dataset

1. Open Source Datasets and select Create source dataset.
2. Select the Data type.
3. If the Data lake field is available, choose where to store the data: in the Observe by Snowflake managed data lake or in your own Iceberg-backed data lake.
4. Enter a Package name and Name. The package groups logically related datasets together. Use one dataset of each data type per package.
5. Optionally, add a Description.
6. Set Data Retention to the period you need.
7. Select Continue.

For supported types, Observe by Snowflake opens Setup instructions with an ingest token creation step, an endpoint, and configuration examples.

For custom ingestion, use a standalone token without destination associations. Configure and enable routing rules and verify the default destination before sending data. Creating a Dataset alone does not redirect existing data to it.

Generated integration instructions may include X-Observe-Target-Package, which selects a managed destination before user-defined rules. Preserve that header when following those instructions. See Choose a routing method for this alternative and how to switch safely.

After sending data, return to the list and check Last received. Select the dataset name to inspect the data.

Manage Dataset settings

Use the controls on a Dataset row to manage it:

• Edit source dataset changes its name, description, and retention period. The package and data type cannot be changed here. The data lake, when present, is also fixed after creation.
• Config examples for source datasets reopens setup instructions for supported types.
• Delete source dataset removes the Source Dataset after confirmation.

Data retention

Set retention when creating a Source Dataset, or change it through Edit source dataset. Observe by Snowflake removes data older than the retention period. Shortening retention can therefore remove historical data.

Manage ingest tokens

Manage standalone ingest tokens separately from source datasets. Open Data & integrations > Ingest tokens, or Manage account > Ingest tokens, depending on your navigation.

See Ingest Tokens for creation, the one-time credential display, permissions, rotation, and API management.

Use a separate token for each cluster, deployment, or team so you can replace credentials without changing other clients. Store each token securely and configure your client to send it using the authorization format shown in its setup instructions.

Standalone ingest tokens are different from the tokens used for the Observe by Snowflake management API. See Observe API authentication for API access.

Existing Datastream ingestion

You may still encounter Datastream terminology in older integrations, configuration examples, and APIs. A Datastream token has an associated destination; a standalone ingest token does not.

Do not replace an existing Datastream token with a standalone token without checking the destination routing. Preserve required package headers or configure an appropriate route, then verify where test data lands before switching clients.

See Datastreams for the existing token workflow and Ingest Routing for managed routes.