Snowflake data in Observe

Snowflake data in Observe - Datasets

Observe creates a connected graph of the Snowflake management views and observability data in Event Table data in the following datasets:

Event Datasets

Event type Datasets:

Name	Purpose	Links to	Origin
COPY_HISTORY	Data loading history [COPY INTO and Snowpipe]	Account, Database, Schema, Table	❄️docs
DATABASE_STORAGE_USAGE_ HISTORY	Storage usage summary (daily)	Account, Database	❄️docs
LISTING_AUTO_FULFILLMENT_DATABASE _STORAGE_DAILY	The data storage used by Cross-Cloud Auto-Fulfillment	Account	❄️docs
LISTING_AUTO_FULFILLMENT_REFRESH _DAILY	The data refreshes performed by Cross-Cloud Auto-Fulfillment	Account	❄️docs
LISTING_AUTO_FULFILLMENT_ USAGE_HISTORY	Cross-Cloud Auto-Fulfillment costs	Account, Organizational Account	❄️docs
LISTING_EVENTS_DAILY	Activity of the marketplace listings	Account, Marketplace Application	❄️docs
LOAD_HISTORY	Data loading history [COPY INTO]	Account, Database, Schema, Table	❄️docs
LOCK_WAIT_HISTORY	Transaction lock wait history	Account, Database, Schema, Table, Query	❄️docs
LOGIN_HISTORY	User logins	Account, User	❄️docs
MARKETPLACE_DISBURSEMENT_ REPORT	History of earnings from paid listings in the Snowflake Marketplace	Account, Marketplace Application	❄️docs
MARKETPLACE_LISTING_ INVOICE_STATUS	History of invoices related to paid listings in the Snowflake Marketplace	Account, Marketplace Application	❄️docs
METERING_DAILY_HISTORY	Snowflake features usage and billing history daily	Account	❄️docs
SNOWPIPE_STREAMING_ CLIENT_HISTORY	Snowpipe Streaming client usage history	Account	❄️docs
STAGE_STORAGE_USAGE_ HISTORY	Stage storage usage summary (daily)	Account	❄️docs
STORAGE_USAGE	Storage usage summary (daily)	Account	❄️docs
TABLE_STORAGE_METRICS	Snowflake table-level storage utilization information	Account, Database, Schema, Table	❄️docs
WAREHOUSE_EVENTS_HISTORY	Virtual warehouse lifecycle events	Account, Role, User, Warehouse, Query	❄️docs

Interval Datasets

Interval type datasets:

Name	Purpose	Links to	Origin
AUTOMATIC_CLUSTERING_HISTORY	Automatic clustering usage and billing history	Account, Database, Schema, Table	❄️docs
COMPLETE_TASK_GRAPHS	Completed task graph runs usage and history	Account, Database, Schema, Task	❄️docs
DATA_TRANSFER_HISTORY	Cloud to cloud data transfer usage history	Account	❄️docs
DATABASE_REPLICATION_ USAGE_HISTORY	Database replication history	Account, Database	❄️docs
DOCUMENT_AI_USAGE_HISTORY	Document AI service usage history	Account, Query	❄️docs
DYNAMIC_TABLE_REFRESH_HISTORY	Dynamic table refresh history	Account, Database, Schema, Table, Query	❄️docs
EVENT_USAGE_HISTORY	Event table usage and billing	Account	❄️docs
HYBRID_TABLE_USAGE_HISTORY	Hybrid table usage history	Account, Hybrid Table	❄️docs
INTERNAL_DATA_TRANSFER_HISTORY	Snowpark Container Services internal data transfers usage history	Account	❄️docs
LISTING_CONSUMPTION_DAILY	Snowflake native app or share consumption	Account, Marketplace Application	❄️docs
LISTING_TELEMETRY_DAILY	Telemetry of the marketplace listings	Account	❄️docs
MATERIALIZED_VIEW_REFRESH_HISTORY	Materialized views refresh billing history	Account, Database, Schema, Table	❄️docs
METERING_HISTORY	Snowflake features usage and billing history	Account	❄️docs
MARKETPLACE_PAID_USAGE_DAILY	Daily history of your usage of a specific paid listing in the Snowflake Marketplace	Account	❄️docs
MONETIZED_USAGE_DAILY	History of daily consumer queries per listing, including charges accumulated for the usage	Account, Marketplace Application	❄️docs
PIPE_USAGE_HISTORY	Data load usage and billing [Snowpipe]	Account, Pipe	❄️docs
QUERY_HISTORY	Snowflake queries	Account, Database, Schema, Table, Role, User, Session, Session Activity	❄️docs
REPLICATION_GROUP_REFRESH_HISTORY	Refresh history for a replication or failover group	Account, Query	❄️docs
REPLICATION_GROUP_USAGE_HISTORY	Refresh usage and costs for a replication or failover group	Account	❄️docs
SEARCH_OPTIMIZATION_BENEFITS	Search optimization service benefits	Account, Table	❄️docs
SEARCH_OPTIMIZATION_HISTORY	Search optimization service usage history	Account, Table	❄️docs
SERVERLESS_TASK_HISTORY	Serverless tasks billing and usage history	Account, Database, Schema	❄️docs
Session	Snowflake user session	Account, User	❄️SESSIONS, ❄️QUERY_HISTORY
SNOWPARK_CONTAINER_SERVICES _HISTORY	Snowpark Container Services usage and billing history	Account	❄️docs
SNOWPIPE_STREAMING_FILE _MIGRATION_HISTORY	Snowpipe Streaming file to table migration history	Account, Database, Schema, Table	❄️docs
TABLE_DML_HISTORY	Table data manipulation history and effects	Account, Database, Schema, Table	❄️docs
TABLE_PRUNING_HISTORY	Table partition pruning history	Account, Database, Schema, Table	❄️docs
TASK_HISTORY	Task runs usage and history	Account, Database, Schema, Task, Query	❄️docs
WAREHOUSE_LOAD_HISTORY	Virtual warehouse workload history	Account, Warehouse	❄️docs
WAREHOUSE_METERING_HISTORY	Virtual warehouse usage and billing history	Account, Warehouse	❄️docs

Resource Datasets

Resource type datasets:

Name	Purpose	Links to	Origin
Account	Snowflake account		❄️QUERY_HISTORY
Database	Snowflake database	Account, Role (owner)	❄️DATABASES
File Format	Snowflake file format	Account, Database, Schema, Role (owner)	❄️FILE_FORMATS
Function	Snowflake function	Account, Database, Schema, Role (owner)	❄️FUNCTIONS
Grant to Role	Privileges that have been granted to a role	Account, Role (Grantee), Role (Granted By)	❄️GRANTS_TO_ROLES
Grant to User	Roles that have been granted to a user	Account, Role (Granted), Role (Granted By), User	❄️GRANTS_TO_USERS
Hybrid Table	Snowflake hybrid table	Account, Database, Schema, Role (owner)	❄️HYBRID_TABLES
Marketplace Application	Snowflake Apps installed from listings in marketplace	Account	❄️APPLICATION_STATE
Network Policy	Snowflake network policy	Account, Role (owner)	❄️docs
Network Rule	Snowflake network rule	Account, Database, Schema, Role (owner)	❄️NETWORK_RULES
Organization Rate Sheet	Snowflake Organization rate sheet	Account	❄️APPLICATION_STATE
Organizational Account	Snowflake organization account	Account	❄️ACCOUNTS
Password Policy	Snowflake password policy	Account, Database, Schema, Role (owner)	❄️PASSWORD_POLICIES
Pipe	Snowflake pipe	Account, Database, Schema, Role (owner)	❄️PIPES
Procedure	Snowflake procedure	Account, Database, Schema, Role (owner)	❄️PROCEDURES
Role	Snowflake role	Account	❄️ROLES
Schema	Snowflake schema	Account, Database, Role (owner)	❄️SCHEMATA
Secret	Snowflake secret	Account, Database, Schema, Role (owner)	❄️SECRETS
Sequence	Snowflake sequence	Account, Database, Schema, Role (owner)	❄️SEQUENCES
Service	Snowflake compute service	Account, Database, Schema, Role (owner)	❄️SERVICES
Stage	Snowflake stage	Account, Database, Schema, Role (owner)	❄️STAGES
Table	Snowflake table	Account, Database, Schema, Role (owner), User	❄️TABLES, ❄️TABLE_STORAGE_METRICS
Task	Snowflake task	Account, Database, Schema	❄️TASK_HISTORY
User	Snowflake user	Account, Role (default), Role (secondary), Role (owner)	❄️USERS
View	Snowflake view	Account, Database, Schema, Role (owner), User	❄️VIEWS
Warehouse	Snowflake virtual warehouse	Account	❄️WAREHOUSE_EVENTS_HISTORY, ❄️QUERY_HISTORY

Metrics and sessions

Metrics type and sessions datasets:

Name	Purpose	Links to	Origin
Automatic Clustering Metering Metrics	Automatic clustering metering metrics	Account, Database, Schema, Table	AUTOMATIC_CLUSTERING_HISTORY
Compute Pool Metering Metrics	Snowpark Container Services metering metrics	Account	SNOWPARK_CONTAINER_SERVICES_HISTORY
Data Transfer Metering Metrics	Data transfer metering metrics	Account	DATA_TRANSFER_HISTORY
Event Table Metering Metrics	Event table metering metrics	Account	EVENT_USAGE_HISTORY
Hybrid Tables Metering Metrics	Hybrid table metering metrics	Account, Hybrid Table	HYBRID_TABLE_USAGE_HISTORY
Query Metrics	Query metrics	Account, Database, Schema, Table, Role, User, Session, Session Activity	QUERY_HISTORY
Serverless Task Metering Metrics	Serverless task metering metrics	Account, Database, Schema	SERVERLESS_TASK_HISTORY
Service Metering Metrics	All Snowflake services metering metrics	Account, Table, Pipe, Warehouse	METERING_HISTORY
Snowpipe Metering Metrics	Snowpipe streaming metering metrics	Account, Pipe	PIPE_USAGE_HISTORY
Table Storage Metrics	Snowflake table-level storage utilization information as metrics	Account, Database, Schema, Table	TABLE_STORAGE_METRICS
Warehouse Metering Metrics	Virtual warehouse metering metrics	Account, Warehouse	WAREHOUSE_METERING_HISTORY
Session Activity	Snowflake user session queries activity	Account, User	QUERY_HISTORY

Snowflake Event Table Datasets

Event table related datasets contain data from Snowflake ❄️Event Table:

Name	Purpose	Links to	Origin
Event Table Event	Snowflake event table - events	Account, Database, Schema	RECORD_TYPE=EVENT
Event Table Logs	Snowflake event table - logs	Account, Database, Schema, Warehouse, Role, User, Query	RECORD_TYPE=LOG
Event Table Metrics	Snowflake event table - metrics	Account, Database, Schema	RECORD_TYPE=METRICS
Event Table OTEL Metrics	Snowflake event table - metrics (via OTEL endpoint)	Account	RECORD_TYPE=METRICS
Event Table Spans	Snowflake event table - spans	Account, Database, Schema, Warehouse, Role, User, Query	RECORD_TYPE=SPAN
Span Event	Snowflake event table - span events	Account, Database, Schema, Warehouse, Role, User, Query, Event Table, Span	RECORD_TYPE=SPAN_EVENT

Dashboards

Observe provides following dashboards for display of Snowflake data:

Name	Purpose	Uses
Account	Summary of objects in account and some core metrics	Account, Object views, QUERY_HISTORY
Data Export Activity	Create external integration, create stage and COPY commands	QUERY_HISTORY
Database	Summary of objects in database and some core metrics, including queries	Account, Object views, Metric datasets, QUERY_HISTORY
Login Activity	View into all the user logins	LOGIN_HISTORY
Pipe	Consumption and usage for pipes	Object views, Metric datasets
Query History	Overview of query activity	QUERY_HISTORY
Query History Metrics	Overview of query activity via precomputed metrics	Query Metrics
Schema	Summary of objects in schema and some core metrics, including queries	Account, Object views, Metric datasets, QUERY_HISTORY
Service Metering	Usage of all compute features	SERVICE_METERING
Session Activity	Overview of session activity	Session
Table	Summary of objects in schema and some core metrics, including queries	Account, Object views, Metric datasets, QUERY_HISTORY
Warehouse	Warehouse usage and activity	Warehouse, WAREHOUSE_METERING_HISTORY

Snowflake Data in Observe - Metrics

Metering metrics

These service metering metrics provide details about consumption of compute and storage:

Dataset	Name
Automatic Clustering Metering Metrics	snowflake.metering.automatic_clustering.credits_used
Automatic Clustering Metering Metrics	snowflake.metering.automatic_clustering.num_bytes_reclustered
Automatic Clustering Metering Metrics	snowflake.metering.automatic_clustering.num_rows_reclustered
Compute Pool Metering Metrics	snowflake.metering.compute_pool.credits_used
Data Transfer Metering Metrics	snowflake.metering.data_transfer.bytes_transferred
Event Table Metering Metrics	snowflake.metering.event_table.bytes_ingested
Event Table Metering Metrics	snowflake.metering.event_table.credits_used
Hybrid Tables Metering Metrics	snowflake.metering.hybrid_tables.credits_used
Serverless Task Metering Metrics	snowflake.metering.serverless_task.credits_used
Service Metering Metrics	snowflake.metering.bytes
Service Metering Metrics	snowflake.metering.credits_cloud
Service Metering Metrics	snowflake.metering.credits_compute
Service Metering Metrics	snowflake.metering.credits_total
Service Metering Metrics	snowflake.metering.files
Service Metering Metrics	snowflake.metering.rows
Snowpipe Metering Metrics	snowflake.metering.pipe.bytes_inserted
Snowpipe Metering Metrics	snowflake.metering.pipe.credits_used
Snowpipe Metering Metrics	snowflake.metering.pipe.files_inserted
Table Storage Metrics	snowflake.storage.active_bytes
Table Storage Metrics	snowflake.storage.failsafe_bytes
Table Storage Metrics	snowflake.storage.retained_for_clone_bytes
Table Storage Metrics	snowflake.storage.time_travel_bytes
Warehouse Metering Metrics	snowflake.metering.warehouse.credits_cloud
Warehouse Metering Metrics	snowflake.metering.warehouse.credits_compute
Warehouse Metering Metrics	snowflake.metering.warehouse.credits_total

Query history metrics

These metrics are derived from ❄️QUERY_HISTORY view:

Dataset	Name
Query Metrics	snowflake.query.bytes_deleted.count
Query Metrics	snowflake.query.bytes_deleted.max
Query Metrics	snowflake.query.bytes_deleted.min
Query Metrics	snowflake.query.bytes_deleted.sum
Query Metrics	snowflake.query.bytes_read_from_result.count
Query Metrics	snowflake.query.bytes_read_from_result.max
Query Metrics	snowflake.query.bytes_read_from_result.min
Query Metrics	snowflake.query.bytes_read_from_result.sum
Query Metrics	snowflake.query.bytes_scanned.count
Query Metrics	snowflake.query.bytes_scanned.max
Query Metrics	snowflake.query.bytes_scanned.min
Query Metrics	snowflake.query.bytes_scanned.sum
Query Metrics	snowflake.query.bytes_sent_over_network.count
Query Metrics	snowflake.query.bytes_sent_over_network.max
Query Metrics	snowflake.query.bytes_sent_over_network.min
Query Metrics	snowflake.query.bytes_sent_over_network.sum
Query Metrics	snowflake.query.bytes_spilled_local.count
Query Metrics	snowflake.query.bytes_spilled_local.max
Query Metrics	snowflake.query.bytes_spilled_local.min
Query Metrics	snowflake.query.bytes_spilled_local.sum
Query Metrics	snowflake.query.bytes_spilled_remote.count
Query Metrics	snowflake.query.bytes_spilled_remote.max
Query Metrics	snowflake.query.bytes_spilled_remote.min
Query Metrics	snowflake.query.bytes_spilled_remote.sum
Query Metrics	snowflake.query.bytes_written_to_result.count
Query Metrics	snowflake.query.bytes_written_to_result.max
Query Metrics	snowflake.query.bytes_written_to_result.min
Query Metrics	snowflake.query.bytes_written_to_result.sum
Query Metrics	snowflake.query.bytes_written.count
Query Metrics	snowflake.query.bytes_written.max
Query Metrics	snowflake.query.bytes_written.min
Query Metrics	snowflake.query.bytes_written.sum
Query Metrics	snowflake.query.child_queries_wait_time.count
Query Metrics	snowflake.query.child_queries_wait_time.max
Query Metrics	snowflake.query.child_queries_wait_time.min
Query Metrics	snowflake.query.child_queries_wait_time.sum
Query Metrics	snowflake.query.compilation_time.count
Query Metrics	snowflake.query.compilation_time.max
Query Metrics	snowflake.query.compilation_time.min
Query Metrics	snowflake.query.compilation_time.sum
Query Metrics	snowflake.query.execution_time.count
Query Metrics	snowflake.query.execution_time.max
Query Metrics	snowflake.query.execution_time.min
Query Metrics	snowflake.query.execution_time.sum
Query Metrics	snowflake.query.list_external_files_time.count
Query Metrics	snowflake.query.list_external_files_time.max
Query Metrics	snowflake.query.list_external_files_time.min
Query Metrics	snowflake.query.list_external_files_time.sum
Query Metrics	snowflake.query.queued_overload_time.count
Query Metrics	snowflake.query.queued_overload_time.max
Query Metrics	snowflake.query.queued_overload_time.min
Query Metrics	snowflake.query.queued_overload_time.sum
Query Metrics	snowflake.query.queued_provisioning_time.count
Query Metrics	snowflake.query.queued_provisioning_time.max
Query Metrics	snowflake.query.queued_provisioning_time.min
Query Metrics	snowflake.query.queued_provisioning_time.sum
Query Metrics	snowflake.query.queued_repair_time.count
Query Metrics	snowflake.query.queued_repair_time.max
Query Metrics	snowflake.query.queued_repair_time.min
Query Metrics	snowflake.query.queued_repair_time.sum
Query Metrics	snowflake.query.rows_deleted.count
Query Metrics	snowflake.query.rows_deleted.max
Query Metrics	snowflake.query.rows_deleted.min
Query Metrics	snowflake.query.rows_deleted.sum
Query Metrics	snowflake.query.rows_inserted.count
Query Metrics	snowflake.query.rows_inserted.max
Query Metrics	snowflake.query.rows_inserted.min
Query Metrics	snowflake.query.rows_inserted.sum
Query Metrics	snowflake.query.rows_produced.count
Query Metrics	snowflake.query.rows_produced.max
Query Metrics	snowflake.query.rows_produced.min
Query Metrics	snowflake.query.rows_produced.sum
Query Metrics	snowflake.query.rows_unloaded.count
Query Metrics	snowflake.query.rows_unloaded.max
Query Metrics	snowflake.query.rows_unloaded.min
Query Metrics	snowflake.query.rows_unloaded.sum
Query Metrics	snowflake.query.rows_updated.count
Query Metrics	snowflake.query.rows_updated.max
Query Metrics	snowflake.query.rows_updated.min
Query Metrics	snowflake.query.rows_updated.sum
Query Metrics	snowflake.query.rows_written_to_result.count
Query Metrics	snowflake.query.rows_written_to_result.max
Query Metrics	snowflake.query.rows_written_to_result.min
Query Metrics	snowflake.query.rows_written_to_result.sum
Query Metrics	snowflake.query.total_elapsed_time.count
Query Metrics	snowflake.query.total_elapsed_time.max
Query Metrics	snowflake.query.total_elapsed_time.min
Query Metrics	snowflake.query.total_elapsed_time.sum
Query Metrics	snowflake.query.transaction_blocked_time.count
Query Metrics	snowflake.query.transaction_blocked_time.max
Query Metrics	snowflake.query.transaction_blocked_time.min
Query Metrics	snowflake.query.transaction_blocked_time.sum

Platform event table metrics

Snowflake ❄️ Platform Metrics will be visible in Observe in either Event Table Metrics or Event Table OTEL Metrics datasets:

Dataset	Name
Event Table Metrics	container.cpu.limit
Event Table Metrics	container.cpu.requested
Event Table Metrics	container.cpu.usage
Event Table Metrics	container.gpu.limit
Event Table Metrics	container.gpu.memory.capacity
Event Table Metrics	container.gpu.memory.usage
Event Table Metrics	container.gpu.requested
Event Table Metrics	container.gpu.utilization
Event Table Metrics	container.memory.limit
Event Table Metrics	container.memory.requested
Event Table Metrics	container.memory.usage
Event Table Metrics	container.restarts
Event Table Metrics	container.state.finished
Event Table Metrics	container.state.last.finished.reason
Event Table Metrics	container.state.pending
Event Table Metrics	container.state.pending.reason
Event Table Metrics	container.state.running
Event Table Metrics	container.state.started
Event Table Metrics	process.cpu.utilization
Event Table Metrics	process.memory.usage
Event Table Metrics	network.egress.received.bytes
Event Table Metrics	network.egress.received.packets
Event Table Metrics	network.egress.transmitted.bytes
Event Table Metrics	network.egress.transmitted.packets
Event Table Metrics	volume.capacity
Event Table Metrics	volume.io.inflight
Event Table Metrics	volume.read.iops
Event Table Metrics	volume.read.throughput
Event Table Metrics	volume.usage
Event Table Metrics	volume.write.iops
Event Table Metrics	volume.write.throughput

Custom event table metrics

If you have an application that emits its own ❄️ Application Metrics to the Event Table, you will see these metrics show up in either Event Table Metrics or Event Table OTEL Metrics datasets.

Updated about 2 months ago