Deployment dashboard¶
Once models are deployed, the deployment Dashboard is the central hub for deployment management activity. It serves as a coordination point for all stakeholders involved in bringing models into production. From the dashboard, you can monitor deployed model performance and take action as necessary, as it provides an interface to all actively deployed models.
Dashboard update frequency
The Dashboard automatically refreshes every 30 seconds, updating the deployment indicators and prediction information.
In the top row of the Dashboard, you can review a summary of your entire inventory of deployments at a glance. The tiles in this row summarize the following information:
| Tile | Description |
|---|---|
| Active Deployments | Indicates the number of deployments currently in use. In the progress bar, active deployments are displayed in blue, all other active deployments are displayed in white, and available new deployments are displayed in gray. Inactive deployments are displayed below the progress bar and do not count toward the allocated limit. |
| Predictions | Indicates the number of predictions made since the last refresh. |
| Service Health | Indicates the total number of deployments with each type of health status (passing, at risk, and failing) for service health monitoring. |
| Drift | Indicates the total number of deployments with each type of health status (passing, at risk, and failing) for data drift monitoring. |
| Accuracy | Indicates the total number of deployments with each type of health status (passing, at risk, and failing) for accuracy monitoring. |
Hover on the Active Deployments tile to get more information about the deployments in your Dashboard. In the example below, the user's organization is allotted 100 deployments. The user has 63 active deployments, and seven other deployments are active in the organization. Users within the organization can create 30 more active deployments before reaching the limit. This organization has four inactive deployments not counted towards the deployment limit:
Deployments in other organizations
If you're active in multiple organizations, under Your active deployments, you can see how many of those active deployments are in This organization or Other organizations. Your deployments in Other organizations do not count toward the allocated limit in the current organization.
Deployment columns¶
The deployment inventory contains a variety of deployment information in the table below the summary tiles. This table is initially sorted by the most recent creation date (reported in the Created column). You can click a different column title to sort by that metric instead. A blue arrow appears next to the sort column's title, indicating if the order is ascending or descending.
Sort order persistence
When you sort the deployment inventory, your most recent sort selection persists in your local settings until you clear your browser's local storage data. As a result, the deployment inventory is usually sorted by the column you selected last.
You can sort in ascending or descending order by:
| Column | Sorting method |
|---|---|
| Name | In alphabetical order. |
| Service Health, Drift, Accuracy | In order of severity by status (for example, descending order proceeds from failing, to at risk, to passing). |
| Activity | In numerical order by the average number of predictions per day. |
| Created | In chronological order by the deployment creation date. |
Secondary sort
The list is sorted secondarily by the time of deployment creation (unless the primary sort is by Created). For example, if you sorted by drift status, all deployments whose status is passing would be ordered from most recent creation to oldest, followed by failing deployments most recent to oldest.
Health indicators¶
The Service Health, Data Drift, and Accuracy summaries in the top part of the display provide an at-a-glance indication of health and accuracy for all deployed models. To view this more detailed information for an individual model, click on the model in the inventory list:
What does the Service Health tile report?
The Service Health tile measures the following error types recorded over the last 24 hours for an individual model on the Monitoring > Service health tab:
- 4xx errors indicate problems with the prediction request submission.
- 5xx errors indicate problems with the DataRobot prediction server.
If you've enabled timeliness tracking on the Settings > Usage tab, you can view timeliness indicators in the inventory. Timeliness indicators show if the prediction or actuals upload frequency meets the standards set by your organization.
Use the table below to interpret the color indicators for each deployment health category:
| Color | Service Health | Data Drift | Accuracy | Timeliness | Action |
|---|---|---|---|---|---|
| Green / Passing | Zero 4xx or 5xx errors. | All attributes' distributions have remained similar since the model was deployed. | Accuracy is similar to when the model was deployed. | Prediction and/or actuals timeliness standards met. | No action needed. |
| Yellow / At risk | At least one 4xx error and zero 5xx errors. | At least one lower-importance attribute's distribution has shifted since the model was deployed. | Accuracy has declined since the model was deployed. | N/A | Concerns found but no immediate action needed; monitor. |
| Red / Failing | At least one 5xx error. | At least one higher-importance attribute's distribution has shifted since the model was deployed. | Accuracy has severely declined since the model was deployed. | Prediction and/or actuals timeliness standards not met. | Immediate action needed. |
| Gray / Disabled | Unmonitored deployment. | Data drift tracking disabled. | Accuracy tracking disabled. | Timeliness tracking disabled. | Enable monitoring and make predictions. |
| Gray / Not started | No service health events recorded. | Data drift tracking not started. | Accuracy tracking not started. | Timeliness tracking not started. | Make predictions. |
| Gray / Unknown | No predictions made. | Insufficient predictions made (min. 100 required). | Insufficient predictions made (min. 100 required). | N/A | Make predictions. |
Filter deployments¶
To filter the deployment inventory, click Filter deployments at the top of the Dashboard page. The filter menu opens, allowing you to select one or more criteria to filter the deployment list:
| Filter | Description |
|---|---|
| Ownership | Filters by deployment owner. Select Owned by me to display only those deployments for which you have the owner role. |
| Activation Status | Filters by deployment activation status. Active deployments are able to monitor and return new predictions. Inactive deployments can only show insights and statistics about past predictions. |
| Fairness Status | Filters by deployment fairness status. Choose to filter by passing , at risk , and failing . |
| Service Status | Filters by deployment service health status. Choose to filter by passing , at risk , and failing . If a deployment has never had service health enabled, then it will not be included when this filter is applied. |
| Drift Status | Filters by deployment data drift status. Choose to filter by passing , at risk , and failing . If a deployment previously had data drift enabled and reported a status, then the last-reported status is used for filtering, even if you later disabled data drift for that deployment. If a deployment has never had drift enabled, then it will not be included when this filter is applied. |
| Accuracy Status | Filters by deployment accuracy status. Choose to filter by passing , at risk , and failing . If a deployment does not have accuracy information available, it is excluded from results when you apply the filter. |
| Importance | Filters by the criticality of deployments, based on prediction volume, exposure to regulatory requirements, and financial impact. Choices include Critical, High, Moderate, and Low. |
| Build environment | Filters by the environment in which the model was built. |
| Prediction Environment Platforms | Filters by the platform the prediction environment runs on. |
After selecting all desired filters, click Apply filters to update the deployment inventory. The Filter deployments button updates to indicate the number of filters applied. To remove your filters, click the Clear all button, or click x next to the badge for each filter you want to remove:
