• PRODUCT
  • LEARN
    • BLOG
    • RESOURCES
    • DOCS
    • OBSERVABILITY COURSE
  • CAREERS
  • ABOUT US
Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
Observability Cloud documentation
Light Logo Dark Logo

Welcome to Observe

  • Welcome to Observe
  • Get help with Observe
    • Use O11y Slack to manage an incident
    • Support policies
      • Observe support terms
      • Create an incident
      • Escalate an issue
      • View your requests
      • Share requests with your team
      • Support policy for accidental ingestion of sensitive data
      • The Observe system user
      • Observe support holiday schedule
    • Observe helpful hints
      • Can I change the name of an Observe instance?
      • What is my customer ID?
      • How do I use a formula?
      • How many monitors are we using?
      • How many queries are we using?
      • How much ingest and transform are we using?
      • How do I make a service appear in Service Explorer?
      • What is the system datastream?
  • Observe tutorials
    • Tutorial: Model weather data
    • Tutorial: Search for improbable travel
    • Tutorial: Create a single stat dashboard
    • Tutorial: Shape metrics
    • Tutorial: Shape aggregated metrics
    • Tutorial: Shape host system metrics
    • Tutorial: Batch data ingestion
  • What’s new
    • Observe platform
    • Observe Agent updates
    • Helm chart updates

Add Data

  • Observe Agent
    • Install Docker image
    • Install on a host
      • Install on Linux
      • Install on Windows
      • Install on macOS
      • Configure the Observe Agent on Linux, Windows, and macOS
    • Install on Kubernetes
      • Helm Chart components
      • Collect annotations and labels
      • Add or delete attributes
      • Prometheus autodiscovery
      • Application RED Metrics
      • Trace Tail Sampling
      • Filter logs or metrics
      • Handle multiline log records
      • Mask sensitive data
      • Collect Statsd metrics
      • Collect Statsd metrics using UDS
      • Node Affinity, Taints, and Tolerations
      • Deploy the Observe Agent to multiple Kubernetes clusters using Rancher
      • Deploy the Observe Agent to a Serverless Kubernetes cluster (ex AWS EKS Fargate)
      • Tune service resource requests and limits
    • Install on Amazon ECS
      • Install on Amazon ECS (EC2)
      • Install on Amazon ECS (Fargate)
    • Install on Ansible
      • Install using Ansible on Linux
      • Install using Ansible on Windows
    • Manage application data volume
    • Configure your own OTel collector
      • Full host example
      • Full Kubernetes example
    • Troubleshooting
  • Cloud integrations
    • Get AWS data into Observe
      • AWS-at-scale data ingestion
      • AWS data collection
    • Get Microsoft Azure data into Observe
      • Azure Active Directory (AD)
      • Azure app services
      • Azure cognitive services
      • Azure functions
      • Azure Kubernetes Service (AKS)
      • Azure SQL database
      • Azure SQL managed instances
      • Azure storage account
      • Azure virtual machines
      • Uninstall the Microsoft Azure app
    • Get Google Cloud Platform data into Observe
      • Configure your GCP project
      • Install the Google Cloud Platform Quickstart app
      • View GCP data in Observe
      • Uninstall the Google Cloud Platform Quickstart app
  • Observe integrations
    • Observe apps
    • Threat Intel (Basic)
      • Install the basic threat intel app
      • Use unified basic threat intel Datasets with Observe
        • Example: Use unified hosts-domains and URL threatlists
        • Example: Use unified IPv4 threatlists
        • Example: Use unified IPv4 IAAS providers list
    • Fastly
      • Install the Fastly app
      • Uninstall the Fastly app
    • GitHub
      • Install the GitHub app
      • Uninstall the GitHub app
      • Upgrade to GitHub app v0.7.0
    • GitLab
      • Install the GitLab app
      • Uninstall the GitLab app
      • Upgrade the GitLab app to v0.4.0
    • Host Quickstart
    • MongoDB Atlas
      • Install the MongoDB Atlas app
      • Update the MongoDB Atlas app and poller
    • MySQL Database Service
      • Install the MySQL DB app
      • Uninstall the MySQL DB app
    • Prometheus Node Exporter
    • Orca Security
      • Install the Orca Security app
    • PostgreSQL Database Service
      • Install the PostgreSQL DB app
      • Uninstall the PostgreSQL DB app
    • Prometheus Metrics
      • Install the Prometheus Metrics app
      • Tutorial: Get started with Observe and Prometheus
    • Security Onion
      • Install the Security Onion app
    • Service level objectives (SLOs)
      • Install the Service Level Objective app
  • Custom data ingestion
    • Datastreams
    • Sources
      • Amazon API Gateway logs
      • Amazon S3
      • AWS AppSync
      • AWS CloudTrail
      • Amazon EventBridge
      • Amazon CloudWatch Metrics Streams
      • Amazon CloudWatch logs
      • GitHub
      • Google Workspace audit logs
      • Jira tickets
      • Webhook
      • Windows servers
      • Zendesk tickets
    • Forwarders
      • Amazon Kinesis Firehose
      • Elastic Beats
      • Fluent Bit
      • Fluentd
      • Log4j
      • Logstash
      • Observe Lambda
      • OpenTelemetry Collector
      • Prometheus
      • Telegraf
    • Endpoints
      • Datadog Metrics
      • Elasticsearch
      • HTTP
      • Kinesis
      • OpenTelemetry
      • Prometheus
    • Troubleshooting data ingestion
  • APM instrumentation
    • Send Java application data to Observe
    • Send .NET application data to Observe
    • Send Node.js application data to Observe
      • Instrument your Node.js application on a host
      • Instrument your Node.js application in Kubernetes
      • Troubleshooting
    • Send Python application data to Observe
    • Send Ruby application data to Observe
    • Send PHP application data to Observe
    • Auto-Instrumentation with OpenTelemetry Operator in Kubernetes
  • LLM instrumentation
    • Node.js (server) instrumentation for LLM observability
    • Python instrumentation for LLM observability
    • Instrumenting other languages for LLM observability

OPAL Reference

  • What is OPAL?
    • OPAL language syntax
    • OPAL data types and operators
    • OPAL examples
    • OPAL case sensitivity
    • Parsing time strings in OPAL
    • All OPAL functions
    • OPAL Functions By Category
      • OPAL Aggregate Functions
        • any
        • any_not_null
        • array_agg
        • array_agg_distinct
        • array_union_agg
        • avg
        • count
        • count_distinct
        • count_distinct_exact
        • delta
        • delta_monotonic
        • deriv
        • first
        • first_not_null
        • hash_agg
        • hash_agg_distinct
        • histogram_combine
        • last
        • last_not_null
        • max
        • median
        • median_exact
        • min
        • object_agg
        • otel_exponential_histogram_sum
        • otel_histogram_sum
        • percentile
        • percentile_cont
        • percentile_disc
        • prom_quantile
        • rate
        • stddev
        • string_agg
        • string_agg_distinct
        • sum
        • tdigest_agg
        • tdigest_combine
        • topk_agg
      • OPAL Boolean Functions
        • array_contains
        • arrays_overlap
        • bool
        • bool_null
        • contains
        • ends_with
        • eq
        • gt
        • gte
        • in
        • ipv4_address_in_network
        • is_null
        • like
        • lt
        • lte
        • match_regex
        • ne
        • path_exists
        • same
        • search
        • starts_with
      • OPAL Misc Functions
        • asc
        • coalesce
        • desc
        • exponential_histogram_null
        • float64
        • frame
        • frame_exact
        • frame_following
        • frame_preceding
        • hash
        • histogram_null
        • histogram_quantile
        • if
        • if_null
        • int64
        • m_exponential_histogram
        • m_histogram
        • m_tdigest
        • nullsfirst
        • nullslast
        • numeric_null
        • on
        • order_by
        • parse_hex
        • strlen
        • tdigest
        • tdigest_null
        • variant_null
      • OPAL Networking Functions
        • int64_to_ipv4
        • ipv4
        • ipv4_address_in_network
        • ipv4_network_int64
        • ipv4_to_int64
        • parse_ip
      • OPAL Numeric Functions
        • abs
        • arccos_deg
        • arccos_rad
        • arcsin_deg
        • arcsin_rad
        • arctan_deg
        • arctan_rad
        • avg
        • ceil
        • cos_deg
        • cos_rad
        • count
        • degrees
        • delta
        • delta_monotonic
        • dense_rank
        • deriv
        • ewma
        • exp
        • float64_null
        • floor
        • haversine_distance_km
        • int64_null
        • int_div
        • ln
        • log
        • median
        • median_exact
        • mod
        • percentile
        • percentile_cont
        • percentile_disc
        • pi
        • pow
        • prom_quantile
        • radians
        • rank
        • rate
        • round
        • row_number
        • sin_deg
        • sin_rad
        • sqrt
        • stddev
        • sum
        • tan_deg
        • tan_rad
        • uniform
        • width_bucket
        • zipf
      • OPAL Regex Functions
        • count_regex_matches
        • get_regex
        • get_regex_all
        • match_regex
        • regex
        • replace_regex
      • OPAL Semistructured Functions
        • append_item
        • array
        • array_agg
        • array_agg_distinct
        • array_contains
        • array_distinct
        • array_length
        • array_max
        • array_min
        • array_null
        • array_to_string
        • array_union_agg
        • arrays_overlap
        • concat_arrays
        • detect_browser
        • drop_fields
        • embed_sql_params
        • get_field
        • get_item
        • get_jmespath
        • get_regex_all
        • index_of_item
        • insert_item
        • intersect_arrays
        • m_object
        • make_array
        • make_array_range
        • make_fields
        • make_object
        • merge_objects
        • object
        • object_agg
        • object_keys
        • object_null
        • otel_exponential_histogram_quantile
        • otel_exponential_histogram_sum
        • otel_histogram_quantile
        • otel_histogram_sum
        • parse_csv
        • parse_ip
        • parse_json
        • parse_kvs
        • parse_url
        • path_exists
        • pick_fields
        • pivot_array
        • prepend_item
        • slice_array
        • sort_array
        • split
        • tokenize
        • topk_agg
        • unpivot_array
      • OPAL Special Functions
        • case
        • group_by
        • m
        • metric
        • options
        • primary_key
        • pk
        • tags
        • valid_for
        • window
      • OPAL String Functions
        • array_to_string
        • check_json
        • concat_strings
        • contains
        • decode_base64
        • decode_uri
        • decode_uri_component
        • detect_browser
        • editdistance
        • embed_sql_params
        • encode_base64
        • encode_uri
        • encode_uri_component
        • ends_with
        • format_time
        • get_regex
        • int64_to_ipv4
        • ipv4_address_in_network
        • label
        • left
        • like
        • lower
        • lpad
        • ltrim
        • parse_csv
        • parse_kvs
        • parse_timestamp
        • pivot_array
        • position
        • regex
        • replace
        • replace_regex
        • right
        • rpad
        • rtrim
        • sha2
        • split
        • split_part
        • starts_with
        • string
        • string_agg
        • string_agg_distinct
        • string_null
        • substring
        • tokenize
        • tokenize_part
        • trim
        • unpivot_array
        • upper
        • variant_type_name
      • OPAL Time Functions
        • abs
        • avg
        • bin_end_time
        • bin_size
        • bin_start_time
        • delta
        • delta_monotonic
        • deriv
        • duration
        • duration_hr
        • duration_min
        • duration_ms
        • duration_null
        • duration_sec
        • ewma
        • format_time
        • from_milliseconds
        • timestamp_ms
        • from_nanoseconds
        • timestamp_ns
        • from_seconds
        • timestamp_s
        • histogram_fraction
        • median
        • median_exact
        • now
        • parse_duration
        • parse_isotime
        • parse_timestamp
        • percentile
        • percentile_cont
        • percentile_disc
        • query_end_time
        • query_start_time
        • rate
        • row_end_time
        • row_timestamp
        • row_start_time
        • stddev
        • sum
        • tdigest_agg
        • tdigest_quantile
        • timestamp_null
        • to_days
        • to_hours
        • to_milliseconds
        • to_minutes
        • to_nanoseconds
        • to_seconds
        • to_weeks
        • valid_for
      • OPAL Window Functions
        • any
        • any_not_null
        • array_union_agg
        • avg
        • count
        • count_distinct
        • count_distinct_exact
        • delta
        • delta_monotonic
        • dense_rank
        • deriv
        • ewma
        • first
        • first_not_null
        • hash_agg
        • hash_agg_distinct
        • lag
        • lag_not_null
        • last
        • last_not_null
        • lead
        • lead_not_null
        • max
        • median
        • median_exact
        • min
        • object_agg
        • percentile
        • percentile_cont
        • percentile_disc
        • rank
        • rate
        • row_number
        • stddev
        • sum
        • tdigest_agg
        • tdigest_combine
        • topk_agg
      • OPAL Deprecated Function Aliases
        • any_null
        • array_pivot
        • array_unpivot
        • countdistinct
        • countdistinctexact
        • decodebase64
        • denserank
        • encodebase64
        • endswith
        • groupby
        • ifnull
        • isnull
        • makeobject
        • match_regex_all
        • medianexact
        • milliseconds
        • nanoseconds
        • orderby
        • parsehex
        • parseip
        • parseisotime
        • parsejson
        • parsekvs
        • parseurl
        • percentilecont
        • percentiledisc
        • primarykey
        • queryendtime
        • querystarttime
        • regex_match
        • regex_replace
        • row_endtime
        • rownumber
        • seconds
        • startswith
        • strcat
        • string_concat
        • validfor
    • All OPAL verbs
    • OPAL Verbs By Category
      • OPAL Aggregate Verbs
        • aggregate
        • align
        • dedup
        • distinct
        • fill
        • histogram
        • make_reference
        • make_session
        • merge_events
        • pivot
        • rollup
        • statsby
        • timechart
        • bucketize
        • timestats
        • unpivot
      • OPAL Filter Verbs
        • always
        • bottomk
        • ever
        • filter
        • filter_last
        • limit
        • never
        • topk
      • OPAL Join Verbs
        • exists
        • follow
        • follow_not
        • fulljoin
        • join
        • leftjoin
        • lookup
        • lookup_ip_info
        • not_exists
        • surrounding
        • union
        • update_resource
      • OPAL Metadata Verbs
        • add_key
        • drop_interface
        • interface
        • make_event
        • make_interval
        • make_metric
        • make_reference
        • make_resource
        • make_session
        • make_table
        • merge_events
        • set_col_enum
        • set_col_immutable
        • set_col_searchable
        • set_col_visible
        • set_label
        • set_link
        • set_metric
        • set_metric_metadata
        • set_primary_key
        • set_pk
        • set_timestamp
        • set_valid_from
        • set_valid_to
        • sort
        • timeshift
        • unset_all_links
        • unset_keys
        • unset_link
        • unsort
      • OPAL Metrics Verbs
        • aggregate
        • align
        • make_metric
        • rollup
        • set_metric
        • timeshift
      • OPAL Projection Verbs
        • drop_col
        • extract_regex
        • make_col
        • pick_col
        • rename_col
      • OPAL Semistructured Verbs
        • extract_regex
        • flatten
        • flatten_all
        • flatten_leaves
        • flatten_single
      • OPAL Deprecated Verb Aliases
        • addfk
        • addkey
        • addmetric
        • changelog
        • coldrop
        • colenum
        • colimmutable
        • colmake
        • colpick
        • colregex
        • colrename
        • colshow
        • droptime
        • fkdrop
        • flattenall
        • flattenleaves
        • flattensingle
        • makeresource
        • makesession
        • merge_event
        • mergeevent
        • reaggregate
        • setlabel
        • setpk
        • setvf
        • setvt
  • OPAL tutorials
    • OPAL 101 – Get started with OPAL
    • OPAL 102 – Shape structured and unstructured data using stages
  • OPAL performance cookbook
    • Use approximate values when feasible
    • Avoid large JSON blobs
    • Cast data columns extracted from JSON
    • Create intermediate Datasets
    • Filter earlier in OPAL scripts
    • Use filter instead of ever
    • Flatten less first
    • Limit Worksheet time windows
    • Limit resource time windows
    • Limit valid event time windows
    • Look for hidden columns
    • Use make_events before window functions
    • Mark immutable resource columns
    • Make resources from multiple Datasets
    • Prefer join to lookup
    • Prefer lead and lag to first and fast
    • Prefer timechart to timestats
    • Limit query time windows
    • Define stricter time filters in queries
    • Reduce columns earlier in OPAL scripts
    • Extract from JSON instead of using flatten
    • Type data columns
    • Use interval for ephemeral things
  • OPAL helpful hints
    • What characters are allowed in a field name?
    • How should I aggregate data?
    • How do I make a standard deviation anomaly detection monitor?
    • How do I find average values over time?
    • How do I change a field type?
    • How do I compare time ranges in OPAL?
    • How to create an array from existing columns?
    • How do I compute a cumulative count over any interval grouped by multiple fields?
    • OPAL duration conversion
    • How do I filter by a list of terms?
    • How do I test for multiple values in a dashboard parameter?
    • Filter out unwanted data
    • How do I Find the size of a column?
    • Formatting large numbers for readability
    • How do I measure drift in a metric over time?
    • How do I sort dates by time when they are sorted alphabetically?
    • How to sort digits numerically when they are sorted alphabetically?
    • How do I map fields to each other?
    • How do I pivot a dataset?
    • How do I measure drift in a resource over time?
    • How do I unpivot data?
    • Can I use OPAL to rename a dataset?
    • How do I prevent lost columns?
    • What is best practice for OPAL field extraction?
    • What is best practice for case statements in OPAL?
    • What is the best practice for field naming in OPAL?
    • What is best practice for managing the schema interface between datasets?
    • What is the best practice for using durations in OPAL?
    • OPAL case sensitive filtering with contains
    • OPAL case sensitive filtering with equals
    • OPAL case sensitive filtering with match_regex
    • OPAL case sensitive filtering with tilde and regex
    • OPAL case sensitive filtering with tilde
    • How do I compare values in OPAL?
    • How do I extract the numeric parts of a message?
    • How do I extract parameters from a URL?
    • How should I rollup aggregated data?
    • How do I search by time?
    • How to set the type of a column?
    • How do I split a field?
    • How do I calculate a running standard deviation?
    • OPAL timestamp conversion
    • How do I find a weighted average?
    • How do I use time window functions?

Products

  • Observe AI
    • Observe AI SRE
    • MCP Server
    • Get help with o11y AI
  • Log management
    • Log Explorer
    • Use log correlation
    • Use live mode in Logs Explorer
    • Add new datasets to Log Explorer
    • Unified Search syntax
      • Migrate to Unified Search syntax
    • Query history
  • APM observability
    • Service management
    • Troubleshoot slow databases / n+1 issues
    • Monitor and track new deploys on your service
    • Associate Infrastructure Metrics with Services
    • Trace Explorer
    • Monitor business applications
    • View logs associated with a trace
    • APM reference
  • LLM observability
    • LLM telemetry reference
  • Metrics
    • Collect and use metrics
    • Metrics Explorer
    • Add custom metric Datasets
    • Add metrics using the Metrics Expression Builder
  • Kubernetes observability
    • Install the Observe Agent
    • Kubernetes visibility
    • Kubernetes Resource Utilization
    • Kubernetes data collection and agent interface
  • Snowflake observability
    • Observe for Snowflake components
    • Prepare Observe to receive Snowflake data
    • Create virtual warehouse to run Observe for Snowflake
    • Install the Observe for Snowflake application
    • Configure the Observe for Snowflake application
    • Send data from Snowflake to Observe
    • Snowflake data in Observe
    • Manage Snowflake with Observe

Platform Capabilities

  • Create dashboards
    • Use dashboards to visualize data
    • Generate dashboard reports
    • Create data links
  • Create Monitors and alerts
    • Create a threshold monitor
    • Create a count monitor
    • Create a promote monitor
    • Monitor rules and severities
    • Mute monitors
    • Configure shared actions
      • Customize alert messages
      • Mustache template reference
      • Sample action for Microsoft Teams
      • Sample action for PagerDuty
    • Work with alerts in Observe
      • Alerting example: shared actions and monitors
    • Tune and troubleshoot monitor health
    • Negative Monitoring
    • Monitor anti-patterns
  • Create and share worksheets
    • Work with data formats and types
  • Create and share Datasets
  • Pivot between data types
  • Correlation tags
  • Resources
  • Data export
  • Conditional formatting

Manage Observe

  • Select light and dark mode settings
  • Workspace settings
    • Configure name, icon, and query settings
    • Set default permissions using RBAC
      • Adding RBAC with Terraform
    • Configure connections
    • Instance settings
  • Manage users and access
    • Manage local users
    • Local authentication policies
    • Manage groups and members
    • Single sign-on (SSO)
      • Configure Microsoft Entra ID (formerly Azure Active Directory) single sign-on (SSO)
      • Configure Microsoft Active Directory Federation Service (ADFS)
      • Configure Google Workspace SAML and single sign-on (SSO)
      • Configure Okta for SAML and single sign-on (SSO)
      • Configure OneLogin for single sign-on (SSO)
      • Configure Ping Identity PingOne for single sign-on (SSO)
    • Audit trail
  • Manage credits and usage
    • Use Acceleration Manager with Datasets
    • Use Credit Manager to manage compute usage
      • Configuring Credit Manager settings via Terraform
      • View your ingest usage in the License Dashboard
      • View your compute credit usage in the Usage Dashboard
    • Usage attribution
    • Invoices
    • Drop filters
  • Customize the Home page
    • Create and use favorites
  • Uploaded documents

Develop with Observe

  • Export query results
  • Developer toolkit overview
  • URL query parameters
  • CLI tool overview
    • login command
    • help command
    • get command
    • list command
    • query command
  • Snowflake outbound sharing
  • Observe Terraform provider

Knowledge Base

  • Key Observe concepts
    • Advanced Observe concepts
    • Explore data
    • About queries and on-demand acceleration
    • Observe Datasets and time
      • Foreign keys
      • Resource primary keys
      • Resource times
      • Reference tables
  • Observasaurus
    • Observasaurus: Accelerate
    • Observasaurus: Agents
    • Observasaurus: Channel
    • Observasaurus: Channel Action
    • Observasaurus: Collectors
    • Observasaurus: Console
    • Observasaurus: Dashboards
    • Observasaurus: Dataset Graph
    • Observasaurus: Datasets
    • Observasaurus: Datastreams
    • Observasaurus: Distributed Tracing
    • Observasaurus: Endpoints
    • Observasaurus: Explorers
    • Observasaurus: Freshness
    • Observasaurus: Link
    • Observasaurus: Logs
    • Observasaurus: Log Analytics
    • Observasaurus: Machine Data
    • Observasaurus: Metrics
    • Observasaurus: Metrics Analytics
    • Observasaurus: Metrics Tags
    • Observasaurus: Monitoring
    • Observasaurus: Monitors
    • Observasaurus: Observability
    • Observasaurus: OPAL
    • Observasaurus: Pollers
    • Observasaurus: Queries
    • Observasaurus: Resources
    • Observasaurus: Security Observability
    • Observasaurus: SIEM
    • Observasaurus: Spans
    • Observasaurus: Stages
    • Observasaurus: Streamable
    • Observasaurus: Telemetry
    • Observasaurus: Temporal SQL
    • Observasaurus: Time Series
    • Observasaurus: Tokens
    • Observasaurus: Traces
    • Observasaurus: Unstreamable
    • Observasaurus: Worksheets
  • Visualization types
  • Observe deployment regions
  • Keyboard shortcuts
  • Units of measurement
Back to top

Cloud integrations¶

Use the Add Data portal to get your cloud data into Observe.

  • Get AWS data into Observe

  • Get Microsoft Azure data into Observe

  • Get Google Cloud data into Observe

Next
Get AWS data into Observe
Previous
Troubleshooting
Copyright © 2017-2025 Observe, Inc.
Made with Furo

Have comments about the Observe docs?