Skip to content

Click in-app to access the full platform documentation for your version of DataRobot.

Custom model Portable Prediction Server

The custom model Portable Prediction Server (PPS) is a solution for deploying a custom model to an external prediction environment. It can be built and run disconnected from main installation environments. The PPS is available as a downloadable bundle containing a deployed custom model, a custom environment, and the MLOps agent. Once started, the custom model PPS installation serves predictions via the DataRobot REST API.

Download and configure the custom model PPS bundle

The custom model PPS bundle is provided for any custom model tagged as having an external prediction environment in the deployment inventory.

Note

Before proceeding, note that DataRobot supports Linux-based prediction environments for PPS. It is possible to use other Unix-based prediction environments, but only Linux-based systems are validated and officially supported.

Select the custom model you wish to use, navigate to the Predictions > Portable Predictions tab of the deployment, and select Download portable prediction package.

Alternatively, instead of downloading the contents in one bundle, you can download the custom model, custom environment, or the MLOps agent as individual components.

After downloading the .zip file, extract it locally with an unzip command:

unzip <cm_pps_installer_*>.zip

Next, access the installation script (unzipped from the bundle) to build the custom model PPS with MLOps agent support. To do so, run the command displayed in step 2:

For more build options, such as the ability to skip the MLOps agent Docker image install, run:

bash ./cm_pps_installer.sh --help

If the build passes without errors, it adds two new Docker images to the local Docker registry:

  • cm_pps_XYZ is the image assembling the custom model and custom environment.

  • datarobot/mlops-tracking-agent is the MLOps Agent Docker image, used to report prediction statistics back to DataRobot.

Make predictions with PPS

DataRobot provides two example Docker Compose configurations in the bundle to get you started with the custom model PPS:

  • docker-compose-fs.yml: uses a file system-based spooler between the model container and MLOps agent container. Recommended for a single model.

  • docker-compose-rabbit.yml: uses a RabbitMQ-based spooler between the model container and MLOps agent container. Use this configuration to run several models with a single MLOps agent instance.

After selecting the configuration to use, edit the Docker Compose file to include the deployment ID and your API key in the corresponding fields.

Once configured, start the prediction sever:

  • For single models using the file system-based spooler, run:

    docker-compose -f docker-compose-fs.yml up

  • For multiple models with a single MLOps agent instance, use the RabbitMQ-based spooler:

    docker-compose -f docker-compose-rabbit.yml up

When the PPS is running, the Docker image exposes three HTTP endpoints:

  • POST /predictions scores a given dataset.
  • GET /info returns information about the loaded model.
  • GET /ping ensures the tech stack is running.

Note

Note that prediction routes only support comma delimited (CSV) scoring datasets. The maximum payload size is 50 MB.

The following demonstrates a sample prediction request:

curl -X POST http://localhost:6788/predictions/ -H "Content-Type: text/csv" --data-binary @path/to/scoring.csv

{
    "data": [{
            "prediction": 23.03329917456927,
            "predictionValues": [{
                "label": "MEDV",
                "value": 23.03329917456927
            }],
            "rowId": 0
        },
        {
            "prediction": 33.01475956455371,
            "predictionValues": [{
                "label": "MEDV",
                "value": 33.01475956455371
            }],
            "rowId": 1
        },
    ]
}

MLOps environment variables

The following table lists the MLOps service environment variables supported for all custom models using PPS. You may want to adjust these settings based on the run environment used.

Variable Description Default
MLOPS_SERVICE_URL The address of the running DataRobot application. Autogenerated value
MLOPS_API_TOKEN Your DataRobot API token. Undefined; must be provided.
MLOPS_SPOOLER_TYPE The type of spooler used by the custom model and MLOps agent. Autogenerated value
MLOPS_FILESYSTEM_DIRECTORY The filesystem spooler configuration for the MLOps agent. Autogenerated value
MLOPS_RABBITMQ_QUEUE_URL The RabbitMQ spooler configuration for the MLOps agent. Autogenerated value
MLOPS_RABBITMQ_QUEUE_NAME The RabbitMQ spooler configuration for the MLOps agent. Autogenerated value
START_DELAY Triggers a delay before starting MLOps agent. Autogenerated value

DRUM-based environment variables

The following table lists the environment variables supported for DRUM-based custom environments:

Variable Description Default
ADDRESS The prediction server's starting address. 0.0.0.0:6788
MODEL_ID The ID of the deployed model (required for monitoring). Autogenerated value
DEPLOYMENT_ID The deployment ID. Undefined; must be provided.
MONITOR A flag that enables MLOps monitoring. True. Provide an empty value or remove this variable to disable monitoring.
MONITOR_SETTINGS Settings for the MLOps agent spooler. Autogenerated value

RabbitMQ service environment variables

Variable Description Default
RABBITMQ_DEFAULT_USER The default RabbitMQ user. Autogenerated value
RABBITMQ_DEFAULT_PASS The default RabbitMQ password. Autogenerated value

Updated November 5, 2021
Back to top