Snowflake Outbound Sharing

Warning

Data sharing is an advanced feature in Public Preview. Use the in-product Contact Support button to contact Observe for more information. On the left side navigation menu, click Docs & Support, Contact Support, and Send Us a Message to contact an Observe Data Engineer.

Snowflake Outbound Sharing allows Observe datasets to be queried from Snowflake using SQL. This allows you to join Observe data with other tables in Snowflake and extract data in larger volumes than supported by the Observe API.

Prerequisites

To use Snowflake data sharing with Observe, you must have a Snowflake account in the same cloud and region as your Observe account. The cloud is always AWS. For most customers, the region is us-west-2 (Oregon).

You must have administrator privileges in Observe to configure data sharing.

The examples in this document use Terraform to manage the Observe objects needed to configure outbound sharing. While the feature does not technically require Terraform, Observe currently does not provide a user interface for configuring Outbound Sharing. Instead, use Terraform to interact with Outbound Sharing.

Configuring a Share

In Observe

In Observe, a Snowflake Outbound Share object configures the underlying Snowflake share. Before creating a Share, you need to find the organization and account names for your Snowflake account.

Note

Note step 3 in Snowflake’s documentation: the Account Name in that example is at the top of the menu with the copy icon. The similar Locator field may be different in your environment. To be sure you have copied the Account Name, please use the “Copy account identifier” button in Snowflake.

resource "observe_snowflake_outbound_share" "example" {
  workspace = data.observe_workspace.default.oid

  name = "example-share"

  account {
    organization = "my-org"
    account      = "my-account"
  }
}

output "share_name" {
  value = observe_snowflake_outbound_share.example.share_name
}

In this example, Terraform exposes share_name as a Terraform output. Run terraform apply and confirm the resulting plan to create the share. After completion, run terraform output share_name to get the full share name. You need this value to complete the process in Snowflake.

In Snowflake

At this point, you have a Share available in your Snowflake account. You can confirm this by running SHOW SHARES, which should include a Share with a kind of INBOUND matching the Share created in the previous step.

To use the Share, create a database from it:

CREATE DATABASE EXAMPLE_SHARED_DB FROM SHARE <share>;

The value of <share> should be substituted with the full Share name obtained in the previous step, which includes the provider account information for Observe’s Snowflake account. From here, you can grant access to this database to roles in your Snowflake account as needed. Until you add datasets to the Share, this database remains empty.

SHOW SHARES

owner_account	name
AB83707.CD79077	1234567890_41139097_example-share

CREATE DATABASE EXAMPLE_SHARED_DB FROM SHARE AB83707.CD79077."1234567890_41139097_example-share";

Sharing a Dataset

Once you configure your Share, you can add individual datasets to expose them as tables in the database. In Observe, a Dataset Outbound Share object manages a sharing configuration for a dataset. Add the following to the previous example code:

data "observe_dataset" "example" {
  workspace = data.observe_workspace.default.oid
  name      = "kubernetes/Container Logs"
}

resource "observe_dataset_outbound_share" "example" {
  workspace = data.observe_workspace.default.oid
  name 		  = "example"
  
  dataset        = data.observe_dataset.example.oid
  outbound_share = observe_snowflake_outbound_share.example.oid
  
  schema_name = "example_schema"
  view_name		= "example_view"
  
  freshness_goal = data.observe_dataset.example.freshness
}

When creating a dataset outbound share, you must choose the schema and view names created within the shared database. Additionally, you must select a freshness goal. Similar to the dataset freshness goal, this value determines how often Observe materializes new data. However, Observe does not apply freshness decay to the shared database. Even if you never query the dataset directly in Observe, Observe maintains the freshness goal as it appears in Snowflake SQL queries.

In the configuration above, you set the freshness goal to the current freshness goal configured for the dataset. Setting a larger value allows limited freshness decay, while setting a smaller value forces more frequent re-materialization of the dataset to maintain the share desired freshness, incurring greater transform costs.

Querying a Dataset

After configuring the share and adding a dataset, you can query the dataset from Snowflake. Using the example values provided in earlier steps:

// Grant privileges to SYSADMIN
USE ROLE ACCOUNTADMIN;
GRANT IMPORTED PRIVILEGES ON DATABASE EXAMPLE_SHARED_DB TO ROLE SYSADMIN;

// Switch to role
USE ROLE SYSADMIN;
USE DATABASE EXAMPLE_SHARED_DB;
SELECT * FROM "example_schema"."example_view";

Keep in mind that Snowflake shares are strictly read-only, so you can SELECT data from a shared dataset and not insert or update it. To change the table’s definition, amend the OPAL query for the dataset in Observe to define the columns as desired.