Set up service health monitoring¶

On a deployment's Service Health > Settings tab, you can enable segmented analysis for service health; however, to use segmented analysis for data drift and accuracy, you must also enable the following deployment settings:

• Enable target monitoring (required to enable data drift and accuracy tracking)

• Enable feature drift tracking (required to enable data drift tracking)

Once you've enabled the tracking required for your deployment, configure segment analysis to access segmented analysis of service health, data drift, and accuracy statistics by filtering them into unique segment attributes and values.

On a deployment's Service Health > Settings tab, you can configure the Service Health Settings:

After enabling segmented analysis, you must specify the segment attributes to track in training and prediction data before making predictions. Selecting a segment attribute for tracking causes the model's data to be segmented by the attribute, allowing users to closely analyze the segment values that comprise the attributes selected for tracking. Attributes used for segmented analysis must be present in the training dataset for a deployed model, but they don't need to be features of the model. The list of segment attributes available for tracking is limited to categorical features, except the selected series ID used by multiseries deployments. To track an attribute, add it to the Track attributes for segmented analysis of training data and predictions field. The "Consumer" attribute (representing users making prediction requests) is always listed by default.

For time series deployments with segmented analysis enabled, DataRobot automatically adds up to two segmented attributes: Forecast Distance and series id (the ID is only provided for multiseries models). Forecast distance is automatically available as a segment attribute without being explicitly present in the training dataset; it is inferred based on the forecast point and the date being predicted on. These attributes allow you to view accuracy and drift for a specific forecast distance, series, or other defined attribute.

When you have finalized the attributes to track, click Save.

Make predictions and navigate to the tab you want to analyze for your deployment by segment: Service Health, Data Drift, or Accuracy.

Service health tracks metrics about a deployment’s ability to respond to prediction requests quickly and reliably. You can view the service health status in the deployment inventory and visualize service health on the Service Health tab. Service health monitoring represents the occurrence of 4XX and 5XX errors in your prediction requests or prediction server:

• 4xx errors indicate problems with the prediction request submission.
• 5xx errors indicate problems with the DataRobot prediction server.

Color	Description	Action
/ Green	Passing: Zero 4xx or 5xx errors	No action needed.
/ Yellow	At risk: At least one 4xx error and zero 5xx errors	Concerns found but no immediate action needed; monitor.
/ Red	Failing: At least one 5xx error	Immediate action needed.
/ Gray	Unknown: No predictions made	Make predictions.