Review and add prediction environments¶
On the Prediction Environments page, you can review the DataRobot prediction environments available to you and create external prediction environments for both DataRobot models running on the Portable Prediction Server and remote models monitored by the monitoring agent.
Review DataRobot prediction environments¶
While you can't create DataRobot prediction environments, you can still view them on the Prediction Environments tab. Review the DataRobot prediction environments available to your organization by locating the environments with DataRobot in the Platform column:
These prediction environments are created by DataRobot and cannot be configured; however, you can deploy models to these prediction environments from this page.
Add a new external prediction environment¶
You can create, manage, and share external prediction environments across DataRobot. This allows you to specify the prediction environments used for both DataRobot models running on the Portable Prediction Server and remote models monitored by the monitoring agent.
To deploy models on external infrastructure, you create a custom external prediction environment:
-
Click Deployments > Prediction Environments and then click + Add prediction environment.
-
In the Add prediction environment dialog box, complete the following fields:
Field Description Name Enter a descriptive prediction environment name. Description (Optional) Enter a description of the external prediction environment. Platform Select the external platform on which the model is running and making predictions. -
Under Supported Model Formats, select one or more formats to control which models can be deployed to the prediction environment, either manually or using the management agent. The available model formats are DataRobot or DataRobot Scoring Code, External Model, and Custom Model.
Important
You can only select one of DataRobot or DataRobot Scoring Code.
-
(Optional) If you want to manage your external model with DataRobot MLOps, click Use Management Agent to allow the MLOps Management Agent to automate the deployment, replacement, and monitoring of models in this prediction environment.
-
Once you configure the environment settings, click Add environment.
The environment is now available from the Prediction Environments page.
Select a prediction environment for a deployment¶
After you add a prediction environment to DataRobot, you can deploy a model and use the prediction environment for the deployment.
Specify the prediction environment in the Prediction History and Service Health section:
Warning
After you specify a prediction environment and create the deployment, you cannot change the prediction environment.
Predictions on DataRobot serverless environments¶
Availability information
Predictions on DataRobot serverless prediction environments are off by default. Contact your DataRobot representative or administrator for information on enabling this feature.
Feature flag: Enable Real-time (Interactive) Predictions on K8s Prediction Environments, Enable Real-time GenAI Predictions on K8s Prediction Environments
Preview
Predictions on DataRobot serverless prediction environments are on by default.
Preview
Predictions on DataRobot serverless prediction environments are on by default. Create a serverless environment to make predictions in DataRobot.
Create DataRobot serverless prediction environments to make scalable predictions with configurable compute instance settings.
Add a DataRobot serverless prediction environment¶
To add a DataRobot serverless prediction environment:
-
Click Deployments > Prediction Environments and then click + Add prediction environment.
-
In the Add prediction environment dialog box, complete the following fields:
Field Description Name Enter a descriptive name for the prediction environment. Description (Optional) Enter a description of the external prediction environment. Platform Select DataRobot Serverless. Batch jobs Max Concurrent Jobs Decrease the maximum number of concurrent jobs for this serverless environment from the organization's defined maximum. Priority Set the importance of batch jobs on this environment. How is the maximum concurrent job limit defined?
There are two limits on max concurrent jobs and these limits depend on the details of your DataRobot installation. Each batch job is subject to both limits, meaning that the conditions of both must be satisfied for a batch job to run on the prediction environment. The first limit is the organization-level limit (default of
30
for Self-Managed installations or10
for SaaS) defined by an organization administrator; this should be the higher limit. The second limit is the environment-level limit defined here by the prediction environment creator; this limit should be lower than the organization-level limit. -
Once you configure the environment settings, click Add environment.
The environment is now available from the Prediction Environments page.
Deploy a model to the DataRobot serverless prediction environment¶
To deploy a model to the DataRobot serverless prediction environment:
-
On the Prediction Environments page, in the Platform row, locate the DataRobot Serverless prediction environments, and click the environment you want to deploy a model to.
-
On the Details tab, under Usages, in the Deployment column, click + Add new deployment.
-
In the Select model version from the registry dialog box, enter the name of the model you want to deploy in the Search box, click the model, and then click the DataRobot model version you want to deploy.
-
Click Select model version and then configure the deployment settings.
-
To enable real-time predictions on this environment, click Show advanced options, scroll down to Advanced Predictions Configuration, click Enable Real-time Predictions and set the following options:
Field Description Minimum compute instances (Premium feature) Set the minimum to a number from 0
to8
. If your organization doesn't have access to "always-on" predictions, this setting is set to0
and isn't configurable. With the minimum compute instances set to0
, the inference server will be stopped after an inactivity period of 30 minutes or more.Maximum compute instances Set the maximum to a number from the current minimum to 8
. To limit compute resource usage, set maximum value equal to the minimum.Premium feature: Always-on predictions
Always-on predictions are a premium feature. Contact your DataRobot representative or administrator for information on enabling the feature.
Update compute instances settings
If, after deployment, you need to update the number of compute instances available to the model, you can change these settings on the Predictions Settings tab.
-
Click Deploy model.
Depending on the availability of compute resources, it can take a few minutes after deployment for a prediction environment to be available for real-time predictions.
Alternate deployment methods
If you don't want to deploy from the Prediction Environments page, you can deploy a model from the Leaderboard or the Model Registry, ensuring that you open the Advanced settings and click Enable Real-time Predictions during deployment configuration.
Make predictions¶
After you've created a DataRobot serverless environment and deployed a model to that environment you can make real-time or batch predictions. Batch predictions are always enabled for these deployments; however, to make real-time predictions, you need to enable real-time predictions during deployment creation or from the Predictions Settings tab.
To make real-time predictions on the DataRobot Serverless prediction environment:
-
In the Deployments inventory, locate and open a deployment associated with a DataRobot serverless environment. To do this, click Filter, select DataRobot Serverless, and then click Apply filters.
-
In a deployment associated with a DataRobot serverless prediction environment, click Predictions > Prediction API.
-
On the Prediction API Scripting Code page, under Prediction Type, click Real-time.
-
Under Language, select Python or cURL, optionally enable Show secrets, and click Copy script to clipboard.
Real-time predictions setting
If you encounter the
Real-time predictions are not enabled for the deployment
error, navigate to the deployment's Predictions > Settings tab to Enable Real-time Predictions. -
Run the Python or cURL snippet to make a prediction request to the DataRobot serverless deployment.
To make batch predictions on the DataRobot Serverless prediction environment, follow the standard process for UI batch predictions or Prediction API scripting predictions.