Skip to content

Click in-app to access the full platform documentation for your version of DataRobot.

Portable Prediction Server

The Portable Prediction Server (PPS) is a DataRobot execution environment for DataRobot model packages (.mlpkg files) distributed as a self-contained Docker image.

Availability information

The Portable Prediction Server is a feature exclusive to DataRobot MLOps. Contact your DataRobot representative for information on enabling it.

PPS can be run disconnected from main installation environments. Once started, the image serves HTTP API via the :8080 port.

In addition to the configuration outlined below, reference the DataRobot functionality compatible with the Portable Prediction Server:

Obtain the Docker image

Depending on your DataRobot environment and version, options for accessing the latest image may differ, as described in the table below.

Deployment type Software version Access method
On-premise or private/hybrid cloud v6.3 or older Contact your DataRobot representative. The image will be provided upon request.
On-premise or private/hybrid cloud v7.0 or later Download the image from Developer Tools; install as described below. If the image is not available contact your DataRobot representative.
Managed AI Cloud Jan 2021 and later Download the image from Developer Tools; install as described below.

Load the image to Docker

Warning

Although DataRobot is currently working to reduce image size, the compressed Docker image can exceed 5GB (Docker-loaded image layers can exceed 12GB). Consider these sizes when downloading and importing PPS images.

Before proceeding, download the image from Developer Tools. It is a tar-gzipped file that can be loaded by Docker.

Once downloaded and the file checksum is verified, use docker load to load the image. Note that you do not have to uncompress the downloaded file because Docker supports loading tar-gzipped images natively:

$ docker load < datarobot-portable-prediction-api-7.0.0-r1736.tar.gz
b990432f3b4b: Loading layer [==================================================>]    343kB/343kB
789a8e70b147: Loading layer [==================================================>]   5.12kB/5.12kB
7db3df02f5f7: Loading layer [==================================================>]  1.775MB/1.775MB
4742b98a91bf: Loading layer [==================================================>]  4.096kB/4.096kB
112ec58ef2cf: Loading layer [==================================================>]  4.165GB/4.165GB
5f9c034e5766: Loading layer [==================================================>]  9.728kB/9.728kB
2f4cf6c33976: Loading layer [==================================================>]  71.17kB/71.17kB
01812df8f7a8: Loading layer [==================================================>]  3.072kB/3.072kB
87bfa8db37b4: Loading layer [==================================================>]  5.978MB/5.978MB
993ea9726377: Loading layer [==================================================>]  1.958GB/1.958GB
e2d9aaaf8de6: Loading layer [==================================================>]  70.66kB/70.66kB
005786d38507: Loading layer [==================================================>]  4.096kB/4.096kB
85b6c0dfcb4e: Loading layer [==================================================>]  4.096kB/4.096kB
f0524ee4003d: Loading layer [==================================================>]  3.584kB/3.584kB
92f123a1e860: Loading layer [==================================================>]  3.072kB/3.072kB
Loaded image: datarobot/datarobot-portable-prediction-api:7.0.0-r1736

Optionally, to save disk space, delete the compressed image archive datarobot-portable-prediction-api-<version>.tar.gz after successful load.

Running modes

There are two model modes supported by the server: single-model (SM) and multi-model (MM). Use SM mode when only a single model package has been mounted into the Docker container inside the /opt/ml/model directory. Use MM mode in all other cases. Despite being compatible predictions wise, SM mode provides a simplified HTTP API that does not require a model package to be identified on disk and preloads a model into memory on start.

The Docker container filesystem directory should match the following layouts.

For SM mode:

/opt/ml/model/
`-- model_5fae9a023ba73530157ebdae.mlpkg

For MM mode:

/opt/ml/model/
|-- fraud
|   `-- model_5fae9a023ba73530157ebdae.mlpkg
`-- revenue
    |-- config.yml
    `-- revenue-estimate.mlpkg

HTTP API (single-model)

When running in single-model mode, the Docker image exposes three HTTP endpoints:

  • POST /predictions scores a given dataset.
  • GET /info returns information about the loaded model.
  • GET /ping ensures the tech stack is up and running.

Note

Note that prediction routes only support comma delimited CSV and JSON records scoring datasets. The maximum payload size is 50 MB.

$ curl -X POST http://<ip>:8080/predictions \
    -H "Content-Type: text/csv" \
    --data-binary @path/to/scoring.csv
{
  "data": [
    {
      "predictionValues": [
        {"value": 0.250833758, "label": "yes"},
        {"value": 0.749166242, "label": "no"},
      ],
      "predictionThreshold": 0.5,
      "prediction": 0.0,
      "rowId": 0
    }
  ]
}

If CSV is the preferred output, request it using the Accept: text/csv HTTP header.

$ curl -X POST http://<ip>:8080/predictions \\
    -H "Accept: text/csv" \\
    -H "Content-Type: text/csv" \\
    --data-binary @path/to/scoring.csv
<target>_yes_PREDICTION,<target>_no_PREDICTION,<target>_PREDICTION,THRESHOLD,POSITIVE_CLASS
0.250833758,0.749166242,0,0.5,yes

HTTP API (multi-model)

In multi-model mode, the Docker image exposes the following endpoints:

  • POST /deployments/:id/predictions scores a given dataset.
  • GET /deployments/:id/info returns information about the loaded model.
  • POST /deployments/:id uploads a model package to the container.
  • DELETE /deployments/:id deletes a model package from the container.
  • GET /deployments returns a list of model packages that are in the container.
  • GET /ping ensures the tech stack is up and running.

The :id included in the /deployments routes above refers to the unique identifier for model packages on the disk. The ID is the directory name containing the model package. Therefore, if you have the following /opt/ml/model layout:

/opt/ml/model/
|-- fraud
|   `-- model_5fae9a023ba73530157ebdae.mlpkg
`-- revenue
    |-- config.yml
    `-- revenue-estimate.mlpkg

You may use fraud and revenue instead of :id in the /deployments set of routes.

Note

Note that prediction routes only support comma delimited CSV and JSON records scoring datasets. The maximum payload size is 50 MB.

$ curl -X POST http://<ip>:8080/deployments/revenue/predictions \
    -H "Content-Type: text/csv" \
    --data-binary @path/to/scoring.csv
{
  "data": [
    {
      "predictionValues": [
        {"value": 0.250833758, "label": "yes"},
        {"value": 0.749166242, "label": "no"},
      ],
      "predictionThreshold": 0.5,
      "prediction": 0.0,
      "rowId": 0
    }
  ]
}

Monitoring

Note

Before proceeding, be sure to configure monitoring for the PSS container. Reference the Environment Variables and Examples sections for further reading.

You can monitor prediction statistics such as data drift and accuracy by creating an external deployment in DataRobot's deployments inventory.

In order to connect your model package to a certain deployment, provide the deployment ID of the deployment that you want to host your prediction statistics.

If you're in Single Model (SM) mode, the deployment ID has to be provided via the MLOPS_DEPLOYMENT_ID environment variable. In Multi Model (MM) mode, a special config.yml should be prepared and dropped alongside the model package with the desired deployment_id value:

deployment_id: 5fc92906ad764dde6c3264fa

If you want to track accuracy, configure it for the deployment, and then provide extra settings for the running model:

For SM mode, set the following environment variables:

  • MLOPS_ASSOCIATION_ID_COLUMN=transaction_country (required)
  • MLOPS_ASSOCIATION_ID_ALLOW_MISSING_VALUES=False (optional, default=false)

For MM mode, set the following properties in config.yml:

association_id_settings:
  column_name: transaction_country
  allow_missing_values: false

HTTPS support

Availability information

If you are running PPS images that were downloaded previously, these parameters will not be available until the PPS image is manually updated:
* Managed AI Cloud: starting Aug 2021
* On-premise or private/hybrid cloud: starting v7.2

By default, PPS serves predictions over an insecure listener on an :8080 port (clear text HTTP over TCP). You can now also serve predictions over a secure listener on :8443 port (HTTP over TLS/SSL, or simply HTTPS). When the secure listener is enabled, the insecure listener becomes unavailable.

Note

You cannot configure PPS to be available on both ports at the same time; it is either HTTP on :8080 or HTTPS on :8443.

The configuration is accomplished using the environment variables described below:

  • PREDICTION_API_TLS_ENABLED: Master flag that enables HTTPS listener on port :8443 and disables HTTP listener on port :8080.
    Default: false (HTTPS disabled)
    Valid values (case-insensitive):
Parameter value Interpretation
true, t, yes, y, 1 true
false, f, no, n, 0 false

Note

The flag value must be interpreted as true to enable TLS. All other PREDICTION_API_TLS_* environment variables (if passed) are ignored if this setting is not enabled.

  • PREDICTION_API_TLS_CERTIFICATE: PEM-formatted content of the TLS/SSL certificate.
    Required: Yes if PREDICTION_API_TLS_ENABLED is "true", otherwise no.
    See also: NGINX SSL certificate documentation

  • PREDICTION_API_TLS_CERTIFICATE_KEY: PEM-formatted content of the secret certificate key of the TLS/SSL certificate key.
    Required: Yes if PREDICTION_API_TLS_ENABLED is true, otherwise no.
    See also: NGINX SSL certificate key documentation

  • PREDICTION_API_TLS_CERTIFICATE_KEY_PASSWORD: Passphrase for the secret certificate key passed in PREDICTION_API_TLS_CERTIFICATE_KEY.
    Required: Yes if a certificate key was created with a passphrase, otherwise no.

  • PREDICTION_API_TLS_PROTOCOLS: Encryption protocol implementation(s) to use.
    Default: TLSv1.2 TLSv1.3
    Valid values: SSLv2|SSLv3|TLSv1|TLSv1.1|TLSv1.2|TLSv1.3, or any space-separated combination of these values.

Warning

As of August 2021, all implementations except TLSv1.2 and TLSv1.3 are considered deprecated and/or insecure. DataRobot highly recommends using only these implementations. New installations may consider using TLSv1.3 exclusively as it is the most recent and secure TLS version.

Warning

TLS support is an advanced feature. The cipher suites list has been carefully selected to follow the latest recommendations and current best practices. DataRobot does not recommend overriding it.

Environment variables

Variable Description Default
PREDICTION_API_WORKERS Sets the number of workers to spin up. This option controls the number of HTTP requests the Prediction API can process simultaneously. Typically, set this to the number of CPU cores available for the container. 1
PREDICTION_API_MODEL_REPOSITORY_PATH Sets the path to the directory where DataRobot should look for model packages. If the PREDICTION_API_MODEL_REPOSITORY_PATH points to a directory containing a single model package in its root, the single-model running mode is assumed by PPS. The multi-model mode will be assumed otherwise. /opt/ml/model/
PREDICTION_API_MONITORING_ENABLED Sets whether DataRobot offloads data monitoring. If true, Prediction API will offload monitoring data to the monitoring agent. False
PREDICTION_API_MONITORING_SETTINGS Sets the connection URI to the monitoring agent. This option controls how to offload monitoring data from the Prediction API to the monitoring agent. None
MONITORING_AGENT Sets whether the monitoring agent runs alongside the Prediction API. False
MONITORING_AGENT_DATAROBOT_APP_URL Sets the URI to the DataRobot installation (e.g., https://app.datarobot.com/). None
MONITORING_AGENT_DATAROBOT_APP_TOKEN Sets a user token to be used with the DataRobot API. None
PREDICTION_API_TLS_ENABLED Sets the TLS listener master flag. Must be activated for the TLS listener to work. false
PREDICTION_API_TLS_CERTIFICATE Adds inline content of the certificate, in PEM format. None
PREDICTION_API_TLS_CERTIFICATE_KEY Adds inline content of the certificate key, in PEM format. None
PREDICTION_API_TLS_CERTIFICATE_KEY_PASSWORD Adds plaintext passphrase for the certificate key file. None
PREDICTION_API_TLS_PROTOCOLS Overrides the TLS/SSL protocols. TLSv1.2 TLSv1.3
PREDICTION_API_TLS_CIPHERS Overrides default cipher suites. Mandatory TLSv1.3, recommended TLSv1.2

PPS parameters

The following sections outline the various query parameter options for prediction requests made with PPS.

Passthrough column parameters

You can parametrize passthrough columns using the following query parameters:

Parameter name Type Notes
passthroughColumns string The list of columns from a scoring dataset returned in the prediction response. This parameter can be used multiple times in a single request if more than one column is requested. For example: ?passthroughColumns=saleId&passthroughColumns=saleDate .
passthroughColumnsSet string If passthroughColumnsSet=all is passed, all columns from the scoring dataset are returned in the prediction response.

Note the following:

  • The passthroughColumns and passthroughColumnsSet parameters cannot both be passed in the same request.
  • While there is no limit on the number of column names you can pass with the passthroughColumns query parameter, there is a limit on the HTTP request line (currently 8192 bytes).

Prediction Explanation parameters

You can parametrize the Prediction Explanations prediction request with the following query parameters:

Parameter name Type Notes
maxExplanations int Maximum number of codes generated per prediction. Default is 3. Previously called maxCodes.
thresholdLow float Prediction Explanation low threshold. Predictions must be below this value (or above the thresholdHigh value) for Prediction Explanations to compute. This value can be null.
thresholdHigh float Prediction Explanation high threshold. Predictions must be above this value (or below the thresholdLow value) for Prediction Explanations to compute. This value can be null.
excludeAdjustedPredictions string Includes or excludes exposure-adjusted predictions in prediction responses if exposure was used during model building. The default value is true (exclude exposure-adjusted predictions).

Time series parameters

You can parametrize the time series prediction request using the following query parameters:

Parameter name Type Notes
forecastPoint string An ISO 8601 formatted DateTime string, without timezone, representing the forecast point. This parameter cannot be used if predictionsStartDate and predictionsEndDate are passed.
relaxKnownInAdvanceFeaturesCheck string true or false. When true, missing values for known in advance features are allowed in the forecast window at prediction time. The default value is false. Note that the absence of known in advance values can negatively impact prediction quality.
predictionsStartDate string The time in the dataset when bulk predictions begin generating. This parameter must be defined together with predictionsEndDate. The forecastPoint parameter cannot be used if predictionsStartDate and predictionsEndDate are passed.
predictionsEndDate string The time in the dataset when bulk predictions stop generating. This parameter must be defined together with predictionsStartDate. The forecastPoint parameter cannot be used if predictionsStartDate and predictionsEndDate are passed.

External configuration

You can also use the Docker image to read and set the configuration options listed in the table above (from /opt/ml/config). The file must contain <key>=<value> pairs, where each key name is a corresponding environment variable.

Examples

  1. Run with two workers:

    $ docker run \
        -v /path/to/mlpkgdir:/opt/ml/model \
        -e PREDICTION_API_WORKERS=2 \
        datarobot/datarobot-portable-prediction-api:<version>
    
  2. Run with external monitoring configured:

    $ docker run \
        -v /path/to/mlpkgdir:/opt/ml/model \
        -e PREDICTION_API_MONITORING_ENABLED='True' \
        -e PREDICTION_API_MONITORING_SETTINGS='<settings>' \
        datarobot/datarobot-portable-prediction-api:<version>
    
  3. Run with internal monitoring configured:

    $ docker run \
        -v /path/to/mlpkgdir:/opt/ml/model \
        -e PREDICTION_API_MONITORING_ENABLED='True' \
        -e PREDICTION_API_MONITORING_SETTINGS='<settings>' \
        -e MONITORING_AGENT='True' \
        -e MONITORING_AGENT_DATAROBOT_APP_URL='https://app.datarobot.com/' \
        -e MONITORING_AGENT_DATAROBOT_APP_TOKEN='<token>' \
        datarobot/datarobot-portable-prediction-api:<version>
    
  4. Run with Python 3:

    $ docker run \
        -v /path/to/mlpkgdir:/opt/ml/model \
        -e PYTHON3_SERVICES='True' \
        datarobot/datarobot-portable-prediction-api:<version>
    

    If your model package was built on a DataRobot cluster prior to Release 7.1, omit this setting. If your model package was built on a newly installed DataRobot 7.1 cluster, set this to True to enable Python version compatibility between the Portable Prediction Server and the model package.

  5. Run with HTTPS support using default protocols and ciphers:

    $ docker run \
        -v /path/to/mlpkgdir:/opt/ml/model \
        -p 8443:8443 \\
        -e PREDICTION_API_TLS_ENABLED='true' \\
        -e PREDICTION_API_TLS_CERTIFICATE="$(cat /path/to/cert.pem)" \\
        -e PREDICTION_API_TLS_CERTIFICATE_KEY="$(cat /path/to/key.pem)" \\
        datarobot/datarobot-portable-prediction-api:<version>
    

Updated November 16, 2021
Back to top