Manage your Monitors

Introduction to the Monitor Explorer.

Select Monitoring from the left navigation rail to view a list of existing Monitors configured on your instance.

You can filter and search Monitors by any attribute, including the Description field. Use Created By and Modified By to search by people who have worked on a Monitor.

Click the Last Triggered value to see Alerts generated by this Monitor.

Edit a Monitor

Clicking a Monitor will open it in read-only mode; click the Edit button at top right to change its definition if your account has access. The read-only page for a Monitor includes access to that Monitor's logs and insight to its metrics, so you can evaluate its performance.

Monitor health states

Reported health can be in one of the following states:

Health state	Description
Running	The Monitor is working as expected.
Warning	The Monitor is executed as expected but has issues that need investigating. Warnings may be caused by trouble reaching destinations, upstream data changes that invalidate findings, or similar concerns.
Failed	The Monitor's most recent attempt to evaluate data ended in an error.
Disabled	The Monitor is not evaluating because a user has manually disabled this Monitor. To enable a Monitor, hover on its name in the Monitor Explorer, click on the vertical ellipsis icon () and slide the toggle in the Disabled field to the on position. The text changes from Disabled to Enabled.
Disabled due to alert rate	The Monitor is generating a very large number of alerts and has automatically been disabled. See Monitors generating too many alerts are automatically disabled .
Exceeding cost	The Monitor has exceeded 100 credits for more than 24 consecutive hours. Contact support to have Observe help you adjust the query in this Monitor.

When a Monitor is disabled, notifications are sent using the appropriate channels. See Notifications sent when a Monitor is disabled.

Monitors generating too many alerts are automatically disabled

Noisy Monitors generating a high number of alerts are automatically disabled.

You can configure this threshold in the Evaluation settings when you configure any Monitor. By default, the threshold is set to 100 alerts per hour, but you can change this number if you expect to have a Monitor generating a higher number of alerts.

Observe automatically disables any Monitor generating more than 3,600 alerts per hour.

Notifications sent when a Monitor is disabled

When a Monitor is disabled, the following notifications are sent:

Emails to the Monitor owner and last updater (most recent user to update the Monitor). In the case that the Monitor owner or last updater are not available, for example, they left the company, the recipients from the Monitor email actions (other users or static addresses without templates) are notified.
Everyone else the Monitor is configured to notify in the action rules. All webhook, Slack, and PagerDuty actions are run for all URLs that are valid and allowed.

Common trouble states

The following table summarizes the common trouble state for health monitors:

Trouble state	Description
Failed to evaluate monitor	This causes a Failed health state that lasts until the next successful execution. A Monitor may fail to evaluate because of dataset re-materialization. Materialization failures may indicate hitting an on-demand materialization limit. Resolving this state requires reviewing your Usage Dashboard and Acceleration Manager to determine if Datasets the monitor needs are unable to materialize.
Detected upstream data updates	Observe maintains awareness of monitored data stability, and will set a Warning state due to changes to data that has already been evaluated. Changes in upstream data can lead to flapping of alert states, false negative, or false positive alerts. The root cause of a change can be due to late-arriving data or a stability delay that is not sufficient for the expected upstream data. Note that upstream stability is not only a matter of external data providers; data instability can be produced by joining a dataset with a periodic update as well. To correct this issue, gradually increase the Monitor's stabilization delay value.

Trouble state

Description

Failed to evaluate monitor

This causes a Failed health state that lasts until the next successful execution. A Monitor may fail to evaluate because of dataset re-materialization. Materialization failures may indicate hitting an on-demand materialization limit. Resolving this state requires reviewing your Usage Dashboard and Acceleration Manager to determine if Datasets the monitor needs are unable to materialize.

Detected upstream data updates

Observe maintains awareness of monitored data stability, and will set a Warning state due to changes to data that has already been evaluated. Changes in upstream data can lead to flapping of alert states, false negative, or false positive alerts. The root cause of a change can be due to late-arriving data or a stability delay that is not sufficient for the expected upstream data. Note that upstream stability is not only a matter of external data providers; data instability can be produced by joining a dataset with a periodic update as well. To correct this issue, gradually increase the Monitor's stabilization delay value.

Updated 11 days ago