MLOps Reporting from Unstructured Models is off by default. Contact your DataRobot representative or administrator for information on enabling this feature.
Feature flag: Enable MLOps Reporting from Unstructured Models
Now available for preview, you can report MLOps statistics from Python custom inference models created in the Custom Model Workshop with an Unstructured (Regression), Unstructured (Binary), or Unstructured (Multiclass) target type:
Target type consideration
MLOps reporting for unstructured models is not supported for the Unstructured (Other) target type.
A dataframe containing all features to track and monitor. Exclude any features you don't want to report from the dataframe.
predictions
A list of predictions.
For regression deployments, this is a 1-dimensional list containing prediction values (e.g., [1, 2, 4, 3, 2]).
For classification deployments, this is a 2-dimensional list, where the inner list is the probabilities for each class type (e.g., [[0.2, 0.8], [0.3, 0.7]]).
association_ids
(Optional) A list of association IDs corresponding to each prediction. Association IDs are used to calculate accuracy and must be unique for each reported prediction. The number of predictions should equal the number of association_ids in the list.
class_names
(Classification only) A list of the names of predicted classes (e.g., ["class1", "class2", "class3"]). For classification deployments, the class names must be in the same order as the prediction probabilities reported. If the order isn't specified, the prediction order defaults to the order of the class names on the deployment. This argument is ignored for regression deployments.
To test an unstructured custom model with MLOps reporting locally, you must use the drum utility with the following input arguments (or the corresponding environment variables):
Input argument
Environment variable
Description
--target-type
TARGET_TYPE
Must be unstructured.
--webserver
EXTERNAL_WEB_SERVER_URL
The DataRobot external web server URL.
--api-token
API_TOKEN
The DataRobot API token.
--monitor-embedded
MLOPS_REPORTING_FROM_UNSTRUCTURED_MODELS
Enables a model to use MLOps library to report statistics.
--deployment-id
DEPLOYMENT_ID
The deployment ID for monitoring model predictions.