Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

Real-time predictions on DataRobot serverless prediction environments

Availability information

Real-time predictions on DataRobot serverless prediction environments are off by default. Contact your DataRobot representative or administrator for information on enabling this feature.

Feature flag: Enable Real-time (Interactive) Predictions on K8s Prediction Environments, Enable Real-time GenAI Predictions on K8s Prediction Environments

Now available for preview, you can create DataRobot serverless prediction environments to make scalable real-time predictions on Kubernetes, with configurable compute instance settings.

Create a DataRobot serverless prediction environment

To create a DataRobot serverless prediction environment for real-time predictions in Kubernetes:

  1. Click Deployments > Prediction Environments and then click + Add prediction environment.

  2. In the Add prediction environment dialog box, complete the following fields:

    Field Description
    Name Enter a descriptive prediction environment name.
    Description (Optional) Enter a description of the external prediction environment.
    Platform Select DataRobot Serverless.
  3. Once you configure the environment settings, click Add environment.

The environment is now available from the Prediction Environments page.

Deploy a model to the DataRobot serverless prediction environment

To deploy a model to the DataRobot serverless prediction environment:

  1. On the Prediction Environments page, in the Platform row, locate the DataRobot Serverless prediction environments, and click the environment you want to deploy a model to.

  2. On the Details tab, under Usages, in the Deployment column, click + Add new deployment.

  3. In the Select model version from the registry dialog box, enter the name of the model you want to deploy in the Search box, click the model, and then click the DataRobot model version you want to deploy.

  4. Click Select model version and then configure the deployment settings, ensuring you click Show advanced options, scroll down to Advanced Predictions Configuration, click Enable Real-time Predictions and set the following options:

    Field Description
    Minimum compute instances Set the minimum to a number from 1 to 8.
    Maximum compute instances Set the maximum to a number from the current minimum to 8. Set maximum value equal to the minimum to limit compute resource usage.

    Update compute instances settings

    If, after deployment, you need to update the number of compute instances available to the model, you can change these settings on the Predictions Settings tab.

  5. Click Deploy model.

    Depending on the availability of compute resources, it can take a few minutes after deployment for a prediction environment to be available for real-time predictions.

Alternate deployment methods

If you don't want to deploy from the Prediction Environments page, you can deploy a model from the Leaderboard or the Model Registry, ensuring that you open the Advanced settings and click Enable Real-time Predictions during deployment configuration.

Make real-time predictions

To make real-time predictions on the DataRobot Serverless prediction environment:

  1. In the Deployments inventory, locate and open a deployment associated with a DataRobot serverless environment. To do this, click Filter, select DataRobot Serverless, and then click Apply filters.

  2. In a deployment associated with a DataRobot serverless prediction environment, click Predictions > Prediction API.

  3. On the Prediction API Scripting Code page, under Prediction Type, click Real-time.

  4. Under Language, select Python or cURL, optionally enable Show secrets, and click Copy script to clipboard.

  5. Run the Python or cURL snippet to make a prediction request to the DataRobot serverless deployment.

Updated May 16, 2024