Alerting example: Channels, Channel Actions, and Monitors

Here is an example that illustrates how Channels and Channel Actions work with Monitors to create alerts.

The goal:

You want to alert on too many errors in the TestWebApp application. Alerts should go to the owner of the application, [email protected], and a Slack channel, #test-web-app.

How?

Use a Monitor to watch the number of errors and alert when it exceeds a threshold. You expect similar alertable conditions in the future, so set up a Channel and Channel Actions in advance:

  • Create Channel Actions for each type of recipient (email or webhook.)
  • Create a Channel containing the desired actions (recipients.)
  • Create a Monitor whose notifications send alerts to this Channel when the triggering condition occurs.

Monitor notifications may send alerts to multiple Channels, and a Channel may contain multiple actions. Channels allow you to configure alerts for a group of recipients without selecting them individually for each new monitor.

(You can also create Channels on the fly. See below for more.)

Create Channel Actions

A Channel Action sends a single type of alert. For multiple types, or for different alert messages for different recipients, create a Channel Action for each.

To send an email to [email protected]:

  • In the left rail Other menu, click Channel Actions and then New Channel Action.
  • In the Content section, select Email and add the recipient's email address. (You could add others too.)
  • Leave the other fields at their defaults.

(For more about the message body, see Customizing Alert Messages)

Click Continue to go to the Delivery section:

  • You haven't created a Channel yet, so skip selecting one for now.
  • Change the default Delivery Frequency if you like.
  • You may also enable reminder or ended notifications for all Monitors that use this action.

Reminder and ended notifications are additional alert messages for Monitors that are still actively triggering, or have ended. To use these optional notifications, first enable them for a Channel Action in the Status Updates section. Later, for each Monitor, select if you wish that Monitor to send reminders.

Click Continue to go to the Channel Action Name section:

  • Give this action a name: "TestWebApp owner: [email protected]".
  • Click Create Channel Action to save.
TestWebApp owner a-developer@example.com

Now create a second Channel Action for the Slack message:

  • Select Webhook for the action type.
  • Fill in your Slack webhook URL. Slack uses POST, so Method is ok.
  • Under Headers, add a "Content-type" header with the value "application/json".
  • Use the default Body template.

Click Continue to go to the Delivery section:

  • No Channel yet, click Continue to go to the Channel Action Name section.
  • Name this action "TestWebApp Slack #test-web-app" and save.

Note: You can send alerts to any service that accepts incoming webhooks. For Slack’s docs, see Incoming webhooks for Slack.

Create a Channel

Now that you have some actions, create a Channel so you can send both types of alerts at the same time:

  • From the left rail Other menu, click Channels, then New Channel.
  • Give this Channel a meaningful name, like "TestWebApp Team".
  • Under Channel Actions, add the two actions you created.
  • You don't have a monitor yet, so click Create Channel to save.
TestWebApp channel configuration with two actions

Create a Monitor

With that setup done, later when you create a Monitor you only need to select the appropriate Channel. For this example, presume you have a Worksheet that shows errors from TestWebApp. (You can create Monitors from Landing Pages too.)

You want to trigger an alert when there are more than 50 errors in the previous 10 minutes, so create a Monitor with that condition:

  • In the Worksheet, click the More button More button and select Create a Monitor.

  • You want to compare the number of events to a static value, so choose a Count monitor.

  • In Trigger Conditions, set your desired condition: greater than, 50, 10, and minutes.

For more about condition options, see Monitor Notification Options.

Click Continue to go to the Notification section.

  • This is a basic count notification, so choose Don't group.
  • But it is urgent, so select Important.
  • For Notification Delivery, select the TestWebApp Team channel. (You could add others as well.)

Click Continue to go to the Monitor Name section.

  • Give your Monitor a meaningful name, like "TestWebApp errors".
  • Click Create Monitor to save.

Your new Monitor displays in the Monitors list:

TestWebApp errors, the number of Events is greater than 50 during the last 10 minutes

Create a New Channel While Creating a Monitor

You may also create a new Channel at the same time you are creating a Monitor. In the Notification section of the Create a Monitor page, select Create New Channel at the bottom of the list of Channels. Then complete your Monitor configuration as above.

Your new Channel won’t have any Channel Actions, so return to the Channel Actions list in the left rail and create a new Channel Action. Then add it to your Channel.