Register external models¶
To create a model package for an external model that is monitored by the monitoring agent, navigate to Model Registry > Model Packages. Click Add new package and select New external model package.
In the resulting dialog box, complete the fields pertaining to the agent-monitored model from which you are retrieving statistics.
The following table describes the fields:
| Field | Description |
|---|---|
| Package Name | The name of the model package. |
| Package Description (optional) | Information to describe the model package. |
| Model location (optional) | The location of the model running outside of DataRobot. Describe the location as a filepath, such as folder1/opt/model.tar. |
| Build environment | The programming language in which the model was built. |
| Training data (optional) | The filename of the training data, uploaded locally or via the AI Catalog. Click Clear selection to upload and use a different file. |
| Holdout data (optional) | The filename of the holdout data, uploaded locally or via the AI Catalog. Use holdout data to set an accuracy baseline and enable support for target drift and challenger models. |
| Target | The dataset column name the model will predict on. |
| Prediction type | The type of prediction the model is making, either binary classification or regression. For a classification model, you must also provide the positive and negative class labels and a prediction threshold. |
| Prediction column | The column name in the holdout dataset containing the prediction result. |
If registering a time series model, mark the checkbox This is a time series model. You must complete additional fields:
| Field | Description |
|---|---|
| Forecast date feature | The column in the training dataset that contains date/time values used by DataRobot to detect the range of dates (the valid forecast range) available for use as the forecast point. |
| Date/time format | The format used by the date/time features in the training dataset. |
| Forecast point feature | The column in the training dataset that contains the point from which you are making a prediction. |
| Forecast unit | The time unit (seconds, days, months, etc.) that comprise the time step. |
| Forecast distance feature | The column in the training dataset containing a unique time step—a relative position—within the forecast window. A time series model outputs one row for each forecast distance. |
| Series identifier (optional, used for multiseries models | The column in the training dataset that identifies which series each row belongs to. |
Once all fields for the model package are defined, click Create package. The package populates in the Model Registry and is available for use.
Set an accuracy baseline¶
To set an accuracy baseline for external models (which enables target drift and challenger models when deployed), you must provide holdout data. This is because DataRobot cannot use the model to generate predictions that typically serve as a baseline, as the model is hosted in a remote prediction environment outside of the application. Provide holdout data when registering an external model package and specify the column containing predictions.