Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

Deploy a custom model in a Portable Prediction Server

Custom inference models allow you to bring your pre-trained models into DataRobot through the Custom Model Workshop. DataRobot supports custom models built in various programming languages, including Python, R, and Java. Once you've created a custom model in DataRobot, you can deploy it to a containerized DataRobot prediction environment called a Portable Prediction Server (PPS). To deploy a custom model to a PPS, you can prepare and test it in the Custom Model Workshop, and then add it to the Model Registry. You can then deploy the custom model using a PPS bundle, which includes everything you need to deploy the model externally while monitoring it alongside models deployed within DataRobot.

To create and deploy a custom model in a PPS, follow the workflow outlined below:

graph TB
  A[Create a custom model] --> B{Custom model environment?} 
  B --> |Yes|C[Create a custom model environment]
  B --> |No|D[Prepare the custom model];
  C --> D
  D --> E{Test locally?}
  E --> |No|H[Test the custom model in DataRobot]
  E --> |Yes|F[Install the DataRobot Model Runner]
  F --> G[Test the custom model locally]
  G --> H
  H --> I[Register the custom model]
  I --> J{Create an external prediction environment?}
  J --> |No|L[Deploy the custom model to an external prediction environment]
  J --> |Yes|K[Add an external prediction environment]
  K --> L 
  L --> M[Deploy the custom model to a PPS]

Create a custom model

Custom inference models are user-created, pre-trained models (made up of a collection of files) uploaded to DataRobot via the Custom Model Workshop.

You can assemble custom inference models in either of the following ways:

  • Create a custom model without providing the model requirements and start_server.sh file on the Assemble tab. This type of custom model must use a drop-in environment. Drop-in environments contain the requirements and start_server.sh file used by the model. They are provided by DataRobot in the Custom Model Workshop. You can also create your own drop-in custom environment.

  • Create a custom model with the model requirements and start_server.sh file on the Assemble tab. This type of custom model can be paired with a custom or drop-in environment.

Create a custom model

(Optional) Create a custom model environment

If you decide to use a custom environment or a custom drop-in environment, you must create that environment in the Custom Model Workshop. You can reuse any environments you create this way for other custom models.

You can assemble custom model environments in either of the following ways:

  • Create a custom drop-in environment with the model requirements and start_server.sh file for the model.

  • Create a custom environment without the model requirements and start_server.sh file. Instead, you must provide the requirements and start_server.sh file in the model folder for the custom model you intend to use with this environment.

Create a custom model environment

Prepare the custom model

Before adding custom models and environments to DataRobot, you must prepare and structure the files required to run them successfully. The tools and templates necessary to prepare custom models are hosted in the Custom Model GitHub repository (Log in to GitHub before clicking this link.). Once you verify the model's files and folder structure, you can proceed to test the model.

Prepare a custom model

(Optional) Test locally

The DataRobot Model Runner (DRUM) is a tool you can use to work locally with Python, R, and Java custom models. It can verify that a custom model can run and make predictions before you add it to DataRobot. However, this testing is only for development purposes, and DataRobot recommends that you use the Custom Model Workshop to test any model you intend to deploy.

Test a custom model locally

Test in DataRobot

Testing the custom model in the Custom Model Workshop ensures that the model is functional before deployment. These tests use the model environment to run the model and make predictions with test data.

Note

While you can deploy your custom inference model without testing, DataRobot strongly recommends that you ensure your model passes testing in the Custom Model Workshop before deployment.

Test a custom model in DataRobot

Register the custom model

After successfully creating and testing a custom inference model in the Custom Model Workshop, you can add it to the Model Registry as a deployment-ready model package.

Register a custom model

Deploy the custom model externally to a PPS

The custom model Portable Prediction Server (PPS) is a solution for deploying a custom model to an external prediction environment. The PPS is a downloadable tarball containing a deployed custom model, a custom model environment, and the MLOps monitoring agent. Once running, the PPS container serves predictions via the DataRobot API.

(Optional) Add an external prediction environment

To create an MLOps custom model deployment compatible with the PPS bundle, you must add the custom model package to an external prediction environment. Create an external prediction environment if you don't already have one in DataRobot.

Add an external prediction environment

Deploy the custom model package to an external prediction environment

To create an MLOps custom model deployment with an external prediction environment, deploy your custom model package to an external prediction environment.

Deploy a custom model to an external prediction environment

Deploy the custom model to a PPS

The custom model PPS bundle is provided for any custom model tagged as having an external prediction environment in the deployment inventory. You can download the custom model PPS bundle to deploy and monitor the custom model.

Deploy a custom model to a PPS


Updated February 16, 2024