Home
User Guides
Start typing to search…
  1. Products
  2. Snowflake observability

Send data from Snowflake to Observe


Send data from the Observe for Snowflake app

You can send data from Snowflake to Observe using the Observe Snowflake app from the UI or from the Console.

Send data from the Observe for Snowflake app using the UI

The Observe for Snowflake native application is ready to send data to Observe. Now you need to specify which management views and tables to send.

  1. Log into your Snowflake instance in Snowsight console.
  2. Change to the ACCOUNTADMIN role.
  3. Navigate to Data Products\Apps.
  4. Click Observe for Snowflake app.
  5. In the ACCOUNT_USAGE History section, choose the view you want in the Add History View dropdown and click Add. You can add all views using Add All.
📘

Note

Some views required you to have Snowflake Enterprise edition or higher. Not all views may contain data, and the contents of each view may depend on your usage.

  1. In the ACCOUNT_USAGE Object section, choose the view you want from the Add Object View dropdown and click Add.
  1. In the EVENT TABLE section, add either default SNOWFLAKE.TELEMETRY.EVENTS table or the custom event table you have configured for your account:

Send data from the Observe for Snowflake app using the Console

  • Log into your Snowflake instance in Snowsight console.
  • Navigate to Project\Worksheets.
  • Use the previously created o4s scripted install Worksheet.
  • Run the following commands. Uncomment each view you want to submit:
--------------------------------------------------------------------------------
-- [ Streamlit UI ]
-- Step 9: Add various supported ACCOUNT_USAGE views for collection
--------------------------------------------------------------------------------
USE ROLE ACCOUNTADMIN;

USE APPLICATION OBSERVE_FOR_SNOWFLAKE;

-- History views
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.ACCESS_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.ALERT_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.AUTOMATIC_CLUSTERING_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.BLOCK_STORAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.COMPLETE_TASK_GRAPHS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.COPY_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DATA_CLASSIFICATION_LATEST');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DATA_QUALITY_MONITORING_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DATA_TRANSFER_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DATABASE_REPLICATION_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DATABASE_STORAGE_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DOCUMENT_AI_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.DYNAMIC_TABLE_REFRESH_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.EVENT_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.HYBRID_TABLE_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.INTERNAL_DATA_TRANSFER_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.LOAD_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.LOCK_WAIT_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.LOGIN_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.MATERIALIZED_VIEW_REFRESH_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.METERING_DAILY_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.METERING_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.PIPE_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.QUERY_ACCELERATION_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.QUERY_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.REPLICATION_GROUP_REFRESH_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.REPLICATION_GROUP_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SEARCH_OPTIMIZATION_BENEFITS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SEARCH_OPTIMIZATION_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SERVERLESS_TASK_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SESSIONS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SNOWPARK_CONTAINER_SERVICES_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SNOWPIPE_STREAMING_CLIENT_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.SNOWPIPE_STREAMING_FILE_MIGRATION_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.STAGE_STORAGE_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.STORAGE_USAGE');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.TABLE_DML_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.TABLE_PRUNING_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.TASK_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.WAREHOUSE_EVENTS_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.WAREHOUSE_LOAD_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ACCOUNT_USAGE.WAREHOUSE_METERING_HISTORY');
-- These only work when you have reader accounts
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('READER_ACCOUNT_USAGE.LOGIN_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('READER_ACCOUNT_USAGE.QUERY_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('READER_ACCOUNT_USAGE.STORAGE_USAGE');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('READER_ACCOUNT_USAGE.WAREHOUSE_METERING_HISTORY');
-- These work only in the Org admin account
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ORGANIZATION_USAGE.LISTING_AUTO_FULFILLMENT_USAGE_HISTORY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_HISTORY('ORGANIZATION_USAGE.RATE_SHEET_DAILY');

-- Object views
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.DATABASES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.FILE_FORMATS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.FUNCTIONS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.GRANTS_TO_ROLES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.GRANTS_TO_USERS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.HYBRID_TABLES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.MASKING_POLICIES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.NETWORK_POLICIES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.NETWORK_RULES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.PASSWORD_POLICIES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.PIPES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.PROCEDURES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.ROW_ACCESS_POLICIES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.ROLES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.SCHEMATA');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.SECRETS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.SEQUENCES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.SERVICES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.SESSION_POLICIES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.STAGES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.TABLES');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.TABLE_STORAGE_METRICS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.TAGS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.USERS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ACCOUNT_USAGE.VIEWS');
-- These work only in the Org admin account
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('ORGANIZATION_USAGE.ACCOUNTS');
-- These work only if you you are in the marketplace account and have apps
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.APPLICATION_STATE');
-- These work only if you you are in the marketplace account and have apps
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.APPLICATION_STATE');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.LISTING_AUTO_FULFILLMENT_DATABASE_STORAGE_DAILY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.LISTING_AUTO_FULFILLMENT_REFRESH_DAILY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.LISTING_CONSUMPTION_DAILY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.LISTING_EVENTS_DAILY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.LISTING_TELEMETRY_DAILY');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.MARKETPLACE_DISBURSEMENT_REPORT');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.MARKETPLACE_LISTING_INVOICE_STATUS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_ACCOUNT_USAGE_OBJECT('DATA_SHARING_USAGE.MONETIZED_USAGE_DAILY');

--------------------------------------------------------------------------------
-- [ Streamlit UI ]
-- Step 10: Add EVENT_TABLE to collection and adjust schedule
-- variables to replace below: <event_table_db>, <event_table_schema>, <event_table_name>
--------------------------------------------------------------------------------
USE ROLE ACCOUNTADMIN;

USE APPLICATION OBSERVE_FOR_SNOWFLAKE;

-- Choose default or non-default event table based on your account configuration
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_EVENT_TABLE('SNOWFLAKE.TELEMETRY.EVENTS');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_EVENT_TABLE('<event_table_db>.<event_table_schema>.<event_table_name>');

--------------------------------------------------------------------------------
-- [ Streamlit UI ]
-- Step 11: Add any table or view
-- variables to replace below: <table_or_view_db>, <table_or_view_schema>, <table_or_view_name>
--------------------------------------------------------------------------------
USE ROLE ACCOUNTADMIN;

USE APPLICATION OBSERVE_FOR_SNOWFLAKE;

-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.ADD_REGULAR_TABLE('<table_or_view_db>.<table_or_view_schema>.<table_or_view_name>');

Adjust the schedule for sending data from the Observe for Snowflake app

The Observe for Snowflake native application uses tasks (running on XS warehouse or serverless) to send data. Those tasks consume compute credits. The default settings strive to strike a balance between data freshness and low runtime costs.

When tasks run on XS warehouse, the recommended settings suspend the XS warehouse after 1 minute of inactivity, which minimizes idle time.

When tasks run on serverless warehouse, you are only charged for the time that unit of compute is being used.

The data egressed to Observe will be billed in ❄️USAGE_IN_CURRENCY_DAILY under SERVICE_TYPE=DATA_TRANSFER, with costs expected to be minimal.

Either way, the more frequently these tasks run, the more credits would be consumed.

Adjust the schedule to send data using the UI

The Observe for Snowflake native application is now sending your Snowflake management data to Observe. You can adjust the frequency of sending for each management view and event table.

  1. Log into your Snowflake instance in Snowsight console.
  2. Change to the ACCOUNTADMIN role.
  3. Navigate to Data Products\Apps.
  4. Click the Observe for Snowflake app.
  5. Modify the schedule of any configured task by providing a new CRON value in theSCHEDULE setting.
  6. Click Save Changes.
  • To disable a task, uncheck the RUNNING checkbox and click Save Changes.
  • To disable all tasks, click Suspend All.
  • To start all disabled tasks, click Start All.

Adjust the schedule to send data using the Console

Perform the following tasks:

  1. Log into your Snowflake instance in Snowsight console.
  2. Navigate to Project\Worksheets.
  3. Use previously created o4s scripted install Worksheet.
  4. Run the following commands:
-- Adjust schedule for the ACCOUNT_USAGE views
-- Replace the schema and the view name accordingly
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.UPDATE_TASK_SCHEDULE('*/15 * * * * UTC', 'TASKS.O4S_TASK_COLLECT_ACCOUNT_USAGE_QUERY_HISTORY_0');

-- Adjust schedule for the SNOWFLAKE.TELEMETRY.EVENTS event table to collect less frequently if so desired
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.UPDATE_TASK_SCHEDULE('*/15 * * * * UTC', 'TASKS.O4S_TASK_SEND_EVENT_TABLE_' || SHA2('SNOWFLAKE.TELEMETRY.EVENTS') || '_0');
-- Adjust schedule for a custom event table
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.UPDATE_TASK_SCHEDULE('*/15 * * * * UTC', 'TASKS.O4S_TASK_SEND_EVENT_TABLE_' || SHA2('<event_table_db>.<event_table_schema>.<event_table_name>') || '_0');

-- Adjust schedule for a custom table or a view
-- Choose the right statement based on the object type
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.UPDATE_TASK_SCHEDULE('*/15 * * * * UTC', 'TASKS.O4S_TASK_SEND_TABLE_' || SHA2('<table_or_view_db>.<table_or_view_schema>.<table_or_view_name>') || '_0');
-- CALL OBSERVE_FOR_SNOWFLAKE.PUBLIC.UPDATE_TASK_SCHEDULE('*/15 * * * * UTC', 'TASKS.O4S_TASK_SEND_VIEW_' || SHA2('<table_or_view_db>.<table_or_view_schema>.<table_or_view_name>') || '_0');

Validate Snowflake data is received in Observe

To validate that Snowflake data is going to Observe, check the status of previously created ingest token in Snowflake app/integration.

  1. Log into your Observe instance using an admin account.
  2. Click Applications in the navigation bar.
  3. Find and open the Snowflake app.
  4. Click the Connections tab.

The previously created token should show a green status light and have some observations:

Updated about 2 months ago