Custom Metrics tab¶
On a deployment's Custom Metrics tab, you can use the data you collect from the Data Export tab (or data calculated through other custom metrics) to compute and monitor up to 25 custom business or performance metrics. These metrics are recorded on the configurable Custom Metric Summary dashboard, where you monitor, visualize, and export each metric's change over time. This feature allows you to implement your organization's specialized metrics, expanding on the insights provided by DataRobot's built-in Service Health, Data Drift, and Accuracy metrics.
Availability information
Custom metrics don't support segmented modeling.
To access custom metrics:
-
In the top navigation bar, click Deployments.
-
On the Deployments tab, click on the deployment for which you want to create custom metrics.
-
In the deployment, click the Custom Metrics tab.
Add custom metrics¶
The Custom Metrics tab can track up to 25 metrics. To add custom metrics:
-
On the Custom Metrics tab, click + Add Custom Metric.
-
In the Add Custom Metric dialog box, click Add external metric, click Next, and then configure the metric settings:
Field Description Name (Required) A descriptive name for the metric. This name appears on the Custom Metric Summary dashboard. Description A description of the custom metric; for example, you could describe the purpose, calculation method, and more. Name of y-axis (Required) A descriptive name for the dependent variable. This name appears on the custom metric's chart on the Custom Metric Summary dashboard. Default interval Determines the default interval used by the selected Aggregation type. Only HOUR is supported. Baseline Determines the value used as a basis for comparison when calculating the x% better or x% worse values. Aggregation type Determines if the metric is calculated as a Sum, Average, or Gauge. Metric direction Determines the directionality of the metric and changes how changes the metric are visualized. You can select Bigger is better or Lower is better. For example, if you choose Lower is better a 10% decrease in the calculated value of your custom metric will be considered 10% better, displayed in green. Timestamp column (Required) The column in the dataset containing a timestamp. This field can't be edited after you create the metric. Value column (Required) The column in the dataset containing the values used for custom metric calculation. This field can't be edited after you create the metric. Date format The date format used by the timestamp column. This field can't be edited after you create the metric. Is Model Specific When enabled, this setting links the metric to the model with the Model Package ID provided in the dataset. This setting influences when values are aggregated (or uploaded). For example: - Model specific (enabled): Model accuracy metrics are model specific, so the values are aggregated completely separately. When you replace a model, the chart for your custom accuracy metric onlys show data for the days after the replacement.
- Not model specific (disabled): Revenue metrics aren't model specific, so the values are aggregated together. When you replace a model, the chart for your custom revenue metric doesn't change.
-
Click Add custom metric.
Upload data to custom metrics¶
After you create a custom metric, you can provide data to calculate the metric:
-
On the Custom Metrics tab, locate the custom metric for which you want to upload data, and then click the Upload Data icon.
-
In the Upload Data dialog box, select an upload method, and then click Next:
Upload method Description Upload data as file In the Choose file dialog box, drag and drop file(s) to upload, or click Choose file > Local file to browse your local filesystem, and then click Submit data. You can upload up to 10GB uploaded in one file. Use AI Catalog In the Select a dataset from the AI Catalog dialog box, click a dataset from the list, and then click Select a dataset. The AI Catalog includes datasets from the Data Export tab. Use API In the Use API Client dialog box, click Copy to clipboard, and then modify and use the API snippet to upload a dataset. You can upload up to 10,000 values in one API call.
Manage custom metrics¶
On the Custom Metrics dashboard, after you've added your custom metrics, you can edit, arrange, or delete them.
To edit or delete a metric, on the Custom Metrics tab, locate the custom metric you want to manage, and then click the more options icon:
-
To edit a metric, click Edit, update any configurable settings, and then click Update custom metric.
-
To delete a metric, click Delete.
To arrange or hide metrics on the Custom Metric Summary dashboard, locate the custom metric you want to move or hide:
-
To move a metric, click the grid icon (
) on the left side of the metric tile and then drag the metric to a new location. -
To hide a metric's chart, clear the checkbox next to the metric name.
Configure the custom metric dashboard display settings¶
Configure the following settings to specify the custom metric calculations you want to view on the dashboard:
| Setting | Description | |
|---|---|---|
 |
Model | Select the deployment's model, current or previous, to show custom metrics for. |
 |
Range (UTC) | Select the start and end dates of the period from which you want to view custom metrics. |
 |
Resolution | Select the granularity of the date slider. Select from hourly, daily, weekly, and monthly granularity based on the time range selected. If the time range is longer than 7 days, hourly granularity is not available. |
 |
Refresh | Refresh the custom metric dashboard. |
|
Reset | Reset the custom metric dashboard's display settings to the default. |