Review and add prediction environments¶

On the Prediction Environments page, you can review the DataRobot prediction environments available to you and create external prediction environments for both DataRobot models running on the Portable Prediction Server and remote models monitored by the monitoring agent.

Review DataRobot prediction environments¶

While you can't create DataRobot prediction environments, you can still view them on the Prediction Environments tab. Review the DataRobot prediction environments available to your organization by locating the environments with DataRobot in the Platform column:

These prediction environments are created by DataRobot and cannot be configured; however, you can deploy models to these prediction environments from this page.

Add an external prediction environment¶

You can create, manage, and share external prediction environments across DataRobot. This allows you to specify the prediction environments used for both DataRobot models running on the Portable Prediction Server and remote models monitored by the monitoring agent.

The monitoring agent typically runs outside of DataRobot, reporting metrics from a configured spooler populated by calls to the DataRobot MLOps library in the external model's code. Now available for public preview, you can run the monitoring agent inside DataRobot by creating an external prediction environment with an external spooler's credentials and configuration details.

To create a prediction environment:

1. Click Prediction Environments and then click Add prediction environment.

   

2. In the Add prediction environment dialog box, complete the following fields:

   

   Field	Description
   Name	Enter a descriptive prediction environment name.
   Description	(Optional) Enter a description of the external prediction environment.
   Platform	Select the external platform on which the model is running and making predictions.

3. Under Supported model formats, select one or more formats to control which models can be deployed to the prediction environment, either manually or using the management agent. The available model formats are DataRobot or DataRobot Scoring Code only, External Model, and Custom Model.

   

   Important

   You cannot select both DataRobot and DataRobot Scoring Code only.

4. Under Managed by, select one of the following:

   

   Option	Description
   Self-managed	Manually manage models on your infrastructure and report data manually to DataRobot.
   Managed by Management Agent	Manage models with the management agent on your own infrastructure.
   Managed by DataRobot	Manage models with the management agent inside DataRobot. This option is only available if the Platform selected is Azure—a premium feature—or Snowflake or Amazon Web Services (AWS)—both preview features.

5. (Optional) To run the monitoring agent in DataRobot, under Monitoring settings, select a Queue:

   The default setting is No Queue.

   Amazon SQSGoogle Pub/SubAzure Event Hubs

   Select the AWS S3 Credentials for your Amazon SQS spooler and configure the following Amazon SQS fields:

   

   Field	Description
   Region	Select the AWS region used for the queue.
   SQS Queue URL	Select the URL of the SQS queue used for the spooler.
   Visibility timeout	(Optional) The visibility timeout before the message is deleted from the queue. This is an Amazon SQS configuration not specific to the monitoring agent.

   After you configure the Queue settings you can provide any Environment variables accepted by the Amazon SQS spooler. For more information, see the Amazon SQS spooler reference.

   

   Select the GCP Credentials for your Google Pub/Sub spooler and configure the following Google Pub/Sub fields:

   Field	Description
   Pub/Sub project	Select the Pub/Sub project used by the spooler.
   Pub/Sub topic	Select the Pub/Sub topic used by the spooler; this should be the topic name within the project, not the fully qualified topic name path that includes the project ID.
   Pub/Sub subscription	Select the Pub/Sub subscription name of the subscription used by the spooler.
   Pub/Sub acknowledgment deadline	(Optional) Enter the amount of time (in seconds) for subscribers to process and acknowledge messages in the queue.

   After you configure the Queue settings you can provide any Environment variables accepted by the Google Pub/Sub spooler. For more information, see the Google Cloud Pub/Sub spooler reference.

   Select the Azure Service Principal Credentials for your Azure Event Hubs spooler and configure the Azure Subscription and Azure Resource Group fields accessible using the provided Credentials:

   

   Azure Service Principal credentials required

   DataRobot management of Scoring Code in AzureML requires existing Azure Service Principal Credentials. If you don't have existing credentials, the Azure Service Principal credentials required alert appears, directing you to Go to Credentials to create Azure Service Principal credentials.

   To create the required credentials, for Credential type, select Azure Service Principal. Then, enter a Client ID, Client Secret, Azure Tenant ID, and a Display name. To validate and save the credentials, click Save and sign in.

   You can find these IDs and the display name on Azure's App registrations > Overview tab (1). You can generate secrets on the App registration > Certificates and secrets tab(2):

   

   Next, configure the following Azure Event Hubs fields:

   

   Field	Description
   Event Hubs Namespace	Select a valid Event Hubs namespace retrieved from the Azure Subscription ID.
   Event Hub Instance	Select an Event Hubs instance within your namespace for monitoring data.

   After you configure the Queue settings, you can provide any additional Environment variables to the agent.

6. Once you configure the environment settings, click Add environment.

   The environment is now available from the Prediction environments page.

Predictions on DataRobot serverless environments¶

Now available for preview, you can create DataRobot serverless prediction environments to make scalable predictions with configurable compute instance settings.

Add a DataRobot serverless prediction environment¶

To add a DataRobot serverless prediction environment:

1. In the Console, click Prediction environments and then click + Add prediction environment.

   

2. In the Add prediction environment dialog box, complete the following fields:

   

   Field	Description
   Name	Enter a descriptive name for the prediction environment.
   Description	(Optional) Enter a description of the external prediction environment.
   Platform	Select DataRobot Serverless.
   Batch jobs
   Max Concurrent Jobs	Decrease the maximum number of concurrent jobs for this serverless environment from the organization's defined maximum.
   Priority	Set the importance of batch jobs on this environment.

   How is the maximum concurrent job limit defined?

   There are two limits on max concurrent jobs and these limits depend on the details of your DataRobot installation. Each batch job is subject to both limits, meaning that the conditions of both must be satisfied for a batch job to run on the prediction environment. The first limit is the organization-level limit (default of 30 for Self-Managed installations or 10 for SaaS) defined by an organization administrator; this should be the higher limit. The second limit is the environment-level limit defined here by the prediction environment creator; this limit should be lower than the organization-level limit.

3. Once you configure the environment settings, click Add environment.

The environment is now available from the Prediction environments page.

Deploy a model to the DataRobot serverless prediction environment¶

To deploy a model to the DataRobot serverless prediction environment:

1. On the Prediction environments page, in the Platform row, locate the DataRobot Serverless prediction environments, and click the environment you want to deploy a model to.

   

2. On the Details tab, under Usages, in the Deployment column, click + Add new deployment.

   

3. In the Select a compatible model package to deploy dialog box, enter the name of the model you want to deploy in the Search box, click the model, and then click the DataRobot model version you want to deploy.

4. Click Select model package and then configure the deployment settings.

5. To enable real-time predictions on this environment, click Show advanced options, scroll down to Advanced Predictions Configuration, click Enable Real-time Predictions and set the following options:

   

   Field	Description
   Minimum compute instances	(Premium feature) Set the minimum to a number from 0 to 8. If your organization doesn't have access to "always-on" predictions, this setting is set to 0 and isn't configurable. With the minimum compute instances set to 0, the inference server will be stopped after an inactivity period of 30 minutes or more.
   Maximum compute instances	Set the maximum to a number from the current minimum to 8. To limit compute resource usage, set maximum value equal to the minimum.

   Premium feature: Always-on predictions

   Always-on predictions are a premium feature. Contact your DataRobot representative or administrator for information on enabling the feature.

   Update compute instances settings

   If, after deployment, you need to update the number of compute instances available to the model, you can change these settings on the Settings > Predictions tab.

6. Click Deploy model.

   Depending on the availability of compute resources, it can take a few minutes after deployment for a prediction environment to be available for real-time predictions.

Make predictions¶

After you've created a DataRobot serverless environment and deployed a model to that environment you can make real-time or batch predictions. Batch predictions are always enabled for these deployments; however, to make real-time predictions, you need to enable real-time predictions during deployment creation or from the Settings > Predictions tab.