Set up custom metrics monitoring¶

On a deployment's Custom Metrics > Settings tab, you can define "at risk" and "failing" thresholds to monitor the status of the custom metrics you created on the Custom Metrics tab.

	Field	Description
Definition
	At Risk / Failing	Enables DataRobot to apply logical statements to calculated custom metric values. You can define threshold statements for a custom metric to categorize the deployment as "at risk" or "failing" if either statement evaluates to true.
Notifications
	Send notifications	Configures the schedule for custom metric monitoring notification checks.

Define custom metric monitoring¶

Configure thresholds to alert you when a deployed model is "at risk" or "failing" to meet the standards you set for the selected custom metric.

You can customize the rules used to calculate the custom metrics status for your deployment on the Custom Metrics Settings page:

1. In a deployment you want to monitor custom metrics for, click Custom Metrics > Settings.

2. In the Definition section, click + Add new definition to define logical statements for any of your custom metrics' At Risk and Failing thresholds. For example, a statement could be: "The ad campaign is At Risk if the Campaign ROI is less than 10000."

   

   Setting	Description
   Metric	Select the custom metric to add a threshold definition for.
   Condition	Select one of the following conidition statements to set the custom metric's threshold for At Risk or Failing: is less than is less than or equal to is greater than is greater than or equal to
   Value	Enter a numerical value to set the custom metric's threshold for At Risk or Failing.

3. After adding custom metrics monitoring definitions, click Save.

Schedule notification checks¶

To schedule recurring checks to determine if custom metric monitoring email notifications should be sent:

1. On the Custom Metric Settings page, in the Notifications section, enable Send notifications.

2. Configure the settings for custom metric notifications. The following table lists the scheduling options. All times are displayed in UTC:

   Frequency	Description
   Every hour	Each hour on the selected minute past the hour.
   Every day	Each day at the selected time.
   Every week	Each selected day at the selected time.
   Every month	Each month, on each selected day, at the selected time. The selected days in a month are provided as numbers (1 to 31) in a comma-separated list.
   Every quarter	Each month of a quarter, on each selected day, at the selected time. The selected days in each month are provided as numbers (1 to 31) in a comma-separated list.
   Every year	Each selected month, on each selected day, at the selected time. The selected days in each month are provided as numbers (1 to 31) in a comma-separated list.
   Use advanced scheduler
   Minute	Each minute defined in a comma-separated list of numbers between 0 and 59, or * for all.
   Hour	Each hour defined in a comma-separated list of numbers between 0 and 23, or * for all.
   Day of month	Each day defined in a comma-separated list of numbers between 1 and 31, or * for all.
   Month	Each month defined in a comma-separated list of numbers between 1 and 12, or * for all.
   Day of week	Each weekday defined in a comma-separated list of numbers between 0 and 6, or * for all.

3. After updating the scheduling settings, click Save. At the configured time, DataRobot sends emails to subscribers.

   Important

   The configured time for the Notifications setting schedules a status check specific to the type of monitoring. DataRobot only sends notifications to subscribers if a monitoring event occurred between checks; for example, if a 4xx or 5xx error is received for service health, or if a data drift status change (from either passing to failing or failing to passing) occurs during the specified window, as shown below: