Monitoring jobs API¶
This integration creates a Batch Monitoring API with batchMonitoringJobDefinitions
and batchJobs
endpoints, allowing you to create monitoring jobs. Monitoring job intake and output settings are configured using the same options as batch prediction jobs. Use the following routes, properties, and examples to create monitoring jobs:
Monitoring job definition and batch job routes¶
batchMonitoringJobDefinitions
endpoints¶
Access endpoints for performing operations on batch monitoring job definitions:
Operation and endpoint | Description |
---|---|
POST /api/v2/batchMonitoringJobDefinitions/ |
Create a monitoring job definition given a payload. |
GET /api/v2/batchMonitoringJobDefinitions/ |
List all monitoring job definitions. |
GET /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ |
Retrieve the specified monitoring job definition. |
DELETE /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ |
Delete the specified monitoring job definition. |
PATCH /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ |
Update the specified monitoring job definition given a payload. |
batchJobs
endpoints¶
Access endpoints for performing operations on batch jobs:
Operation and endpoint | Description |
---|---|
POST /api/v2/batchJobs/fromJobDefinition/ |
Launch (run now) a monitoring job from a monitoringJobDefinition . The payload should contain the monitoringJobDefinitionId . |
GET /api/v2/batchJobs/ |
List the full history of monitoring jobs, including running, aborted, and executed jobs. |
GET /api/v2/batchJobs/{monitoringJobId}/ |
Retrieve a specific monitoring job. |
DELETE /api/v2/batchJobs/{monitoringJobId}/ |
Abort a running monitoring job. |
Monitoring job properties¶
monitoringColumns
properties¶
Define which columns to use for batch monitoring:
Property | Type | Description |
---|---|---|
predictionsColumns |
string | (Regression) The column in the data source containing prediction values. You must provide this field and/or actualsValueColumn . |
predictionsColumns |
array | (Classification) The columns in the data source containing each prediction class. You must provide this field and/or actualsValueColumn . (Supports a maximum of 1000 items) |
associationIdColumn |
string | The column in the data source which contains the association ID for predictions. |
actualsValueColumn |
string | The column in the data source which contains actual values. You must provide this field and/or predictionsColumns . |
actualsTimestampColumn |
string | The column in the data source which contains the timestamps for actual values. |
monitoringOutputSettings
properties¶
Configure the output settings specific to monitoring jobs:
Property | Type | Description |
---|---|---|
uniqueRowIdentifierColumns |
array | Columns from the data source that will serve as unique identifiers for each row. These columns are copied to the data destination to associate each monitored status with its corresponding source row. (Supports a maximum of 100 items) |
monitoredStatusColumn |
string | The column in the data destination containing the monitoring status for each row. |
Note
For general batch job output settings, see the Prediction output settings documentation.
monitoringAggregation
properties¶
For external models with large-scale monitoring enabled, describe the retention policy and the amount of raw data retained for challengers. To support the use of challenger models, you must send raw features. For large datasets, you can report a small sample of raw feature and prediction data to support challengers and reporting; then, you can send the remaining data in aggregate format.
Important
If you define these properties, raw data is aggregated by the MLOps library. This means that the data isn't stored in the DataRobot platform. Stats aggregation only supports feature and prediction data, not actuals data. If you've defined actualsValueColumn
or associationIdColumn
( which means actuals will be provided later), DataRobot cannot aggregate data.
Property | Type | Description |
---|---|---|
retentionPolicy |
string | The policy definition determines if the retentionValue represents a number of samples or a percentage of the dataset. enum: ['samples', 'percentage'] |
retentionValue |
integer | The amount of data to retain, this can be a percentage or the number of samples. |
Monitoring job examples¶
{
"batchJobType": "monitoring",
"deploymentId": "<deployment_id>",
"intakeSettings": {
"type": "jdbc",
"dataStoreId": "<data_store_id>",
"credentialId": "<credential_id>",
"table": "lending_club_regression",
"schema": "SCORING_CODE_UDF_SCHEMA",
"catalog": "SANDBOX"
},
"outputSettings": {
"type": "jdbc",
"dataStoreId": "<data_store_id>",
"table": "lending_club_regression_out",
"catalog": "SANDBOX",
"schema": "SCORING_CODE_UDF_SCHEMA",
"statementType": "insert",
"createTableIfNotExists": true,
"credentialId": "<credential_id>",
"commitInterval": 10,
"whereColumns": [],
"updateColumns": []
},
"passthroughColumns": [],
"monitoringColumns": {
"predictionsColumns": "PREDICTION",
"associationIdColumn": "id",
"actualsValueColumn": "loan_amnt"
},
"monitoringOutputSettings": {
"monitoredStatusColumn": "monitored",
"uniqueRowIdentifierColumns": ["id"]
}
"schedule": {
"minute": [ 0 ],
"hour": [ 17 ],
"dayOfWeek": ["*" ],
"dayOfMonth": ["*" ],
"month": [ "*” ]
},
"enabled": true
}
{
"batchJobType": "monitoring",
"deploymentId": "<deployment_id>",
"intakeSettings": {
"type": "jdbc",
"dataStoreId": "<data_store_id>",
"credentialId": "<credential_id>",
"table": "lending_club_regression",
"schema": "SCORING_CODE_UDF_SCHEMA",
"catalog": "SANDBOX"
},
"outputSettings": {
"type": "jdbc",
"dataStoreId": "<data_store_id>",
"table": "lending_club_regression_out",
"catalog": "SANDBOX",
"schema": "SCORING_CODE_UDF_SCHEMA",
"statementType": "insert",
"createTableIfNotExists": true,
"credentialId": "<credential_id>",
"commitInterval": 10,
"whereColumns": [],
"updateColumns": []
},
"monitoringColumns": {
"predictionsColumns": [
{
"className": "True",
"columnName": "readmitted_True_PREDICTION"
},
{
"className": "False",
"columnName": "readmitted_False_PREDICTION"
}
],
"associationIdColumn": "id",
"actualsValueColumn": "loan_amnt"
},
"monitoringOutputSettings": {
"uniqueRowIdentifierColumns": ["id"],
"monitoredStatusColumn": "monitored"
}
"schedule": {
"minute": [ 0 ],
"hour": [ 17 ],
"dayOfWeek": ["*" ],
"dayOfMonth": ["*" ],
"month": [ "*” ]
},
"enabled": true
}