Skip to content

Click in-app to access the full platform documentation for your version of DataRobot.

Deployment settings

You can add data about a deployment and configure monitoring, notifications, and challenger behavior using the Settings tab.

Use the Data tab

The Data tab allows you to add learning, inference, and outcome data to your deployment that you did not initially provide when creating it. The additional data and information can enable more deployment capabilities. This tab is identical to the interface used when you add a new deployment.

If you edit the information for any fields in the Data tab, be sure to click Save Changes when finished editing.

Note

Data cannot be altered for a deployment once it is added.

Model section

The Model section details information about the model being used to make predictions for your deployment. DataRobot uses the files and information from the deployment to complete these fields, so they are greyed out and not editable.

Field Description
Model The name of your model.
Target The dataset column name the model will predict on.
Prediction type The type of prediction the model is making, for example, Regression, Classification, Multiclass, Anomaly Detection, Clustering, etc.

Fairness section

The Fairness section allows you to define Bias and Fairness settings for your deployment to identify any biases in the model's predictive behavior. If fairness settings are defined prior to deploying a model, the fields are automatically populated. For additional information, see the section on defining fairness tests.

Field Description
Protected features The dataset columns to measure fairness of model predictions against; must be categorical.
Primary fairness metric The statistical measure of parity constraints used to assess fairness.
Favorable target outcome The outcome value perceived as favorable for the protected class relative to the target.
Fairness threshold The fairness threshold helps measure if a model performs within appropriate fairness bounds for each protected class.

Inference section

This Inference section provides details about your deployment's inference (also known as scoring) data—the data that contains prediction requests and results from the model.

Field Description
Inference data DataRobot stores a deployment's inference data when a deployment is created. It cannot be uploaded separately.
Prediction timestamp Determines the method for time-stamping prediction rows. Use the time of the prediction request or use a date/time feature (e.g., forecast date) provided with prediction data to determine the timestamp. Forecast date time-stamping is set automatically for time series deployments. It allows for a common time axis to be used between training data and the basis of data drift and accuracy statistics. This setting cannot be changed after the deployment is created and predictions are made.
Prediction environment Environment where predictions are generated. Prediction environments allow you to establish access controls and approval workflows.
Association ID The column name that contains the association ID in the prediction dataset for your model. Association IDs are required for setting up accuracy tracking in a deployment. The association ID functions as an identifier for your prediction dataset so you can later match up outcome data (also called "actuals") with those predictions. Note that the Create deployment button is inactive until you enter an association ID or turn off this toggle.
Enable data drift tracking Configures DataRobot to track target and feature drift in a deployment.
Enable prediction rows storage for challenger analysis Enables the use of challenger models, which allow you to compare models post-deployment and replace the champion model if necessary.
Track attributes for segmented analysis of training data and predictions When enabled, allows DataRobot to monitor deployment predictions by segments, for example by categorical features.

Learning section

The Learning section provides details about the deployment's learning (also known as training) data—the data used to train and build a model.

Actuals section

Use the Actuals section to enable Accuracy for model deployments. To activate Actuals, you must first set an Association ID in the Inference Data section. Reference the documentation for setting up accuracy for specific information about this section.

Data Drift section

When deploying a model, there is a chance that the dataset used for training and validation differs from the prediction data. Following are drift-related configuration settings.

Data drift tracking

DataRobot monitors data drift information by default and displays results in the Data Drift dashboard. Use this toggle to turn off tracking if, for example, you have sensitive data that should not be monitored in the deployment.

You can customize how data drift is monitored. See the data drift page for more information on customizing data drift status for deployments.

Note that this feature is only available for deployments using deployment-aware prediction API routes (i.e., https://example.datarobot.com/predApi/v1.0/deployments/<deploymentId>/predictions).

Prediction row storage for challenger analysis

DataRobot can securely store prediction request data at the row level for deployments (not supported for external model deployments). This setting must be enabled for any deployment using the Challengers tab. In addition to enabling challenger analysis, access to stored prediction request rows enables you to thoroughly audit the predictions and use that data to troubleshoot operational issues. For instance, you can examine the data to understand an anomalous prediction result or why a dataset was malformed.

Note

Contact your DataRobot representative to learn more about data security, privacy, and retention measures or to discuss prediction auditing needs.

Enable prediction request row collection either:

  • From a deployment's Settings > Data tab, under the Inference header
  • During deployment creation

Toggle Enable prediction rows storage for challenger analysis to on. During deployment creation, this toggle appears under the Inference Data section.

Once enabled, prediction requests made for the deployment are collected by DataRobot. Prediction explanations are not stored.

Note

Note that prediction requests are collected only if the prediction data is in a valid data format interpretable by DataRobot, such as CSV or JSON. Failed prediction requests with a valid data format are also collected (i.e., missing input features).

Set prediction intervals for time series deployments

Time series users have the additional capability to add a prediction interval to the prediction response of deployed models. When enabled, prediction intervals will be added to the response of any prediction call associated with the deployment.

To enable prediction intervals, navigate to the Settings > Prediction Intervals tab.

Toggle Enable prediction intervals to on and select an interval (read more about prediction intervals here).

When you have set an interval, you can copy the ID from the deployment URL (or from the snippet in the Prediction API tab) and check that the deployment was added to the database as well as add the ID to the deployment prediction script. You can compare the results from your API output with prediction preview in the UI to verify results.

For more information on working with prediction intervals via the API, access the API documentation by signing in to DataRobot, clicking the question mark on the upper right, and selecting API Documentation. In the API documentation, select Time Series Projects > Prediction Intervals.


Updated January 7, 2022
Back to top