Custom model Portable Prediction Server¶

The custom model Portable Prediction Server (PPS) is a solution for deploying a custom model to an external prediction environment. It can be built and run disconnected from main installation environments. The PPS is available as a downloadable bundle containing a deployed custom model, a custom environment, and the monitoring agent. Once started, the custom model PPS installation serves predictions via the DataRobot REST API.

Download and configure the custom model PPS bundle¶

The custom model PPS bundle is provided for any custom model tagged as having an external prediction environment in the deployment inventory.

Select the custom model you wish to use, navigate to the Predictions > Portable Predictions tab of the deployment, and select Download portable prediction package.

Alternatively, instead of downloading the contents in one bundle, you can download the custom model, custom environment, or the monitoring agent as individual components.

After downloading the .zip file, extract it locally with an unzip command:

unzip <cm_pps_installer_*>.zip

Next, access the installation script (unzipped from the bundle) to build the custom model PPS with monitoring agent support. To do so, run the command displayed in step 2:

For more build options, such as the ability to skip the monitoring agent Docker image install, run:

bash ./cm_pps_installer.sh --help

If the build passes without errors, it adds two new Docker images to the local Docker registry:

• cm_pps_XYZ is the image assembling the custom model and custom environment.

• datarobot/mlops-tracking-agent is the monitoring agent Docker image, used to report prediction statistics back to DataRobot.

Make predictions with PPS¶

DataRobot provides two example Docker Compose configurations in the bundle to get you started with the custom model PPS:

• docker-compose-fs.yml: uses a file system-based spooler between the model container and the monitoring agent container. Recommended for a single model.

• docker-compose-rabbit.yml: uses a RabbitMQ-based spooler between the model container and the monitoring agent container. Use this configuration to run several models with a single monitoring agent instance.

After selecting the configuration to use, edit the Docker Compose file to include the deployment ID and your API key in the corresponding fields.

To define a runtime parameters file, modify the example Docker Compose configuration file you're using to uncomment the two lines highlighted below:

volumes:
  - shared-volume:/opt/mlops_spool
  # - <path_to_runtime_params_file_on_host>:/opt/runtime.yaml
environment:
  ADDRESS: "0.0.0.0:6788"
  MODEL_ID: "{{ model_id }}"
  DEPLOYMENT_ID: "< provide deployment id to report to >"
  RUNTIME_PARAMS_FILE: "/opt/runtime.yaml"
  # RUNTIME_PARAMS_FILE: "/opt/runtime.yaml"

When you uncomment the line under volumes, replace the <path_to_runtime_params_on_host> placeholder with the appropriate path. The line under environment is a DRUM environment variable, described in the DRUM-based environment variables section. If you aren't using an example Docker Compose configuration, add these lines to your configuration file. In both scenarios, make sure the runtime parameters file exists and the path to the file is correct.

Once the Docker Compose file is properly configured, start the prediction server:

• For single models using the file system-based spooler, run:

  docker compose -f docker-compose-fs.yml up
  

• For multiple models with a single monitoring agent instance, use the RabbitMQ-based spooler:

  docker compose -f docker-compose-rabbit.yml up
  

When the PPS is running, the Docker image exposes three HTTP endpoints:

• POST /predictions scores a given dataset.
• GET /info returns information about the loaded model.
• GET /ping ensures the tech stack is running.

The following demonstrates a sample prediction request and JSON response:

curl -X POST http://localhost:6788/predictions/ \
     -H "Content-Type: text/csv" \
     --data-binary @path/to/scoring.csv

{
    "data": [{
            "prediction": 23.03329917456927,
            "predictionValues": [{
                "label": "MEDV",
                "value": 23.03329917456927
            }],
            "rowId": 0
        },
        {
            "prediction": 33.01475956455371,
            "predictionValues": [{
                "label": "MEDV",
                "value": 33.01475956455371
            }],
            "rowId": 1
        },
    ]
}

MLOps environment variables¶

The following table lists the MLOps service environment variables supported for all custom models using PPS. You may want to adjust these settings based on the run environment used.

DRUM-based environment variables¶

The following table lists the environment variables supported for DRUM-based custom environments:

RabbitMQ service environment variables¶