Snowflake outbound sharing allows Observe datasets to be queried from Snowflake using SQL. This allows you to join Observe data with other tables in Snowflake and extract data in larger volumes than supported by the Observe API.

Prerequisites

Verify the following requirements are met before using Snowflake outbound sharing:

Your Snowflake account must be in the same cloud and region as your Observe account. The cloud is always AWS. For most customers, the region is us-west-2 (Oregon).
You must have administrator privileges in Observe to configure data sharing.

The examples in this document use Terraform to manage the Observe objects needed to configure outbound sharing. While the feature does not technically require Terraform, Observe currently does not provide a user interface for configuring Outbound Sharing. Instead, use Terraform to interact with Outbound Sharing.

Configure a share

In Observe, find your Snowflake organization and account names.

In Observe, a Snowflake Outbound Share object configures the underlying Snowflake share. Before creating a Share, you must find the organization and account names for your Snowflake account.

📘
Note
The Account Name in that example is at the top of the menu with the copy icon. The similar Locator field may be different in your environment. To be sure you have copied the Account Name, please use the Copy account identifier button in Snowflake.

resource "observe_snowflake_outbound_share" "example" { 
  workspace = data.observe_workspace.default.oid

  name = "example-share"

  account {
  organization = "my-org"
  account      = "my-account"
  }
}

output "share_name" {  
  value = observe_snowflake_outbound_share.example.share_name
}

In this example, Terraform exposes share_name as a Terraform output. Run terraform apply and confirm the resulting plan to create the share. After completion, run terraform output share_name to get the full share name. You need this value to complete the process in Snowflake.

In Snowflake, create a database to use the Share.

When you have a Share available in your Snowflake account, run SHOW SHARES in Snowflake to verify the Share with a kind of INBOUND matching the Share created in the previous step. For example:

SHOW SHARES

owner_account	name
AB83707.CD79077	1234567890_41139097_example-share

To use the Share, create a database from it:

CREATE DATABASE EXAMPLE_SHARED_DB FROM SHARE AB83707.CD79077."1234567890_41139097_example-share";

From here, you can grant access to this database to roles in your Snowflake account as needed. Until you add datasets to the Share, this database remains empty.

Share a Dataset

Once you configure your Share, you can add individual Datasets to expose them as tables in the database. In Observe, a Dataset Outbound Share object manages a sharing configuration for a Dataset. Add the following to the previous example code:

data "observe_dataset" "example" { 
  workspace = data.observe_workspace.default.oid
  name      = "kubernetes/Container Logs"
}

resource "observe_dataset_outbound_share" "example" { 
  workspace = data.observe_workspace.default.oid
  name 		  = "example"

  dataset         = data.observe_dataset.example.oid
  outbound_share  = observe_snowflake_outbound_share.example.oid
  change_tracking = "false"

  schema_name = "example_schema"
  view_name		= "example_view"

  freshness_goal = data.observe_dataset.example.freshness
}

When creating a Dataset Outbound Share, you must choose the schema and view names created within the shared database. The Outbound Share name is shared to the target Snowflake account and will appear in the listing when you run the ❄️DESCRIBE SHARE command in the target account.

Then, you can run the following command to map the Outbound Share to a Snowflake database. Replace <provider_account> with your Observe provider account ID.

CREATE DATABASE <database_name> FROM SHARE <provider_account>.<outbound_share_name>

Every table shared within the Outbound Share is landed in the Snowflake database you just created. Snowflake uses the following structure, so that when you share the table, you must provide the Share to identify the database and target account, along with the schema and view name:

<database>.<schema>.<table/view/object name>

These are completely arbitrary - it’s just a matter of how you want them to appear in the target database. See Object name resolution in the Snowflake documentation.

Additionally, you must select a freshness goal. Similar to the Dataset freshness goal, this value determines how often Observe materializes new data. However, Observe does not apply freshness decay to the shared database. Even if you never query the Dataset directly in Observe, Observe maintains the freshness goal as it appears in Snowflake SQL queries.

In the configuration above, you set the freshness goal to the current freshness goal configured for the Dataset. Setting a larger value allows limited freshness decay, while setting a smaller value forces more frequent re-materialization of the Dataset to maintain the share desired freshness, incurring greater transform costs.

Set change_tracking to true to create a stream containing updates of the shared view. Then, you can run the following query on your Snowflake account, and replace {DATABASE}, {SCHEMA}, and {STREAM_TABLE_NAME} with the names of a database, schema, and stream that are not part of the shared database, and replace <shared_view> with the name of the shared view created on your tenant:

CREATE OR REPLACE STREAM "{DATABASE}"."{SCHEMA}"."{STREAM_TABLE_NAME}" ON VIEW "<shared view>";

Query a Dataset

After configuring the share and adding a Dataset, you can query the Dataset from Snowflake. Using the example values provided in earlier steps:

// Grant privileges to SYSADMIN\
USE ROLE ACCOUNTADMIN;
GRANT IMPORTED PRIVILEGES ON DATABASE EXAMPLE_SHARED_DB TO ROLE SYSADMIN;

// Switch to role
USE ROLE SYSADMIN;
USE DATABASE EXAMPLE_SHARED_DB;
SELECT * FROM "example_schema"."example_view";

Keep in mind that Snowflake shares are strictly read-only, so you can SELECT data from a shared Dataset and not insert or update it. To change the table’s definition, amend the OPAL query for the Dataset in Observe to define the columns as desired.