Skip to content
No result to preview.

Click in-app to access the full platform documentation for your version of DataRobot.

Predictions > Batch prediction methods > Prediction monitoring jobs > Monitoring jobs API

Monitoring jobs API

This integration creates a Batch Monitoring API with batchMonitoringJobDefinitions and batchJobs endpoints, allowing you to create monitoring jobs. Monitoring job intake and output settings are configured using the same options as batch prediction jobs. Use the following routes, properties, and examples to create monitoring jobs:

Monitoring job definition and batch job routes

batchMonitoringJobDefinitions endpoints

Access endpoints for performing operations on batch monitoring job definitions:

Operation and endpoint Description
POST /api/v2/batchMonitoringJobDefinitions/ Create a monitoring job definition given a payload.
GET /api/v2/batchMonitoringJobDefinitions/ List all monitoring job definitions.
GET /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ Retrieve the specified monitoring job definition.
DELETE /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ Delete the specified monitoring job definition.
PATCH /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ Update the specified monitoring job definition given a payload.

batchJobs endpoints

Access endpoints for performing operations on batch jobs:

Operation and endpoint Description
POST /api/v2/batchJobs/fromJobDefinition/ Launch (run now) a monitoring job from a monitoringJobDefinition. The payload should contain the monitoringJobDefinitionId.
GET /api/v2/batchJobs/ List the full history of monitoring jobs, including running, aborted, and executed jobs.
GET /api/v2/batchJobs/{monitoringJobId}/ Retrieve a specific monitoring job.
DELETE /api/v2/batchJobs/{monitoringJobId}/ Abort a running monitoring job.

Monitoring job properties

monitoringColumns properties

Define which columns to use for batch monitoring:

Property Type Description
predictionsColumns string (Regression) The column in the data source containing prediction values. You must provide this field and/or actualsValueColumn.
predictionsColumns array (Classification) The columns in the data source containing each prediction class. You must provide this field and/or actualsValueColumn. (Supports a maximum of 1000 items)
associationIdColumn string The column in the data source which contains the association ID for predictions.
actualsValueColumn string The column in the data source which contains actual values. You must provide this field and/or predictionsColumns.
actualsTimestampColumn string The column in the data source which contains the timestamps for actual values.

monitoringOutputSettings properties

Configure the output settings specific to monitoring jobs:

Property Type Description
uniqueRowIdentifierColumns array Columns from the data source that will serve as unique identifiers for each row. These columns are copied to the data destination to associate each monitored status with its corresponding source row. (Supports a maximum of 100 items)
monitoredStatusColumn string The column in the data destination containing the monitoring status for each row.

Note

For general batch job output settings, see the Prediction output settings documentation.

monitoringAggregation properties

For external models with large-scale monitoring enabled, describe the retention policy and the amount of raw data retained for challengers. To support the use of challenger models, you must send raw features. For large datasets, you can report a small sample of raw feature and prediction data to support challengers and reporting; then, you can send the remaining data in aggregate format.

Important

If you define these properties, raw data is aggregated by the MLOps library. This means that the data isn't stored in the DataRobot platform. Stats aggregation only supports feature and prediction data, not actuals data. If you've defined actualsValueColumn or associationIdColumn ( which means actuals will be provided later), DataRobot cannot aggregate data.

Property Type Description
retentionPolicy string The policy definition determines if the retentionValue represents a number of samples or a percentage of the dataset.
enum: ['samples', 'percentage']
retentionValue integer The amount of data to retain, this can be a percentage or the number of samples.

Monitoring job examples

Regression
{
  "batchJobType": "monitoring",
  "deploymentId": "<deployment_id>",
  "intakeSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "credentialId": "<credential_id>",
      "table": "lending_club_regression",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "catalog": "SANDBOX"
  },
  "outputSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "table": "lending_club_regression_out",
      "catalog": "SANDBOX",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "statementType": "insert",
      "createTableIfNotExists": true,
      "credentialId": "<credential_id>",
      "commitInterval": 10,
      "whereColumns": [],
      "updateColumns": []
  },
  "passthroughColumns": [],
  "monitoringColumns": {
      "predictionsColumns": "PREDICTION",
      "associationIdColumn": "id",
      "actualsValueColumn": "loan_amnt"
  },
  "monitoringOutputSettings": {
     "monitoredStatusColumn": "monitored",
     "uniqueRowIdentifierColumns": ["id"]
  }
  "schedule": {
      "minute": [ 0  ],
      "hour": [ 17   ],
      "dayOfWeek": ["*" ],
      "dayOfMonth": ["*" ],
      "month": [ "*” ]
  },
  "enabled": true
}
Classification
{
  "batchJobType": "monitoring",
  "deploymentId": "<deployment_id>",
  "intakeSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "credentialId": "<credential_id>",
      "table": "lending_club_regression",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "catalog": "SANDBOX"
  },
  "outputSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "table": "lending_club_regression_out",
      "catalog": "SANDBOX",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "statementType": "insert",
      "createTableIfNotExists": true,
      "credentialId": "<credential_id>",
      "commitInterval": 10,
      "whereColumns": [],
      "updateColumns": []
  },
  "monitoringColumns": {
  "predictionsColumns": [
              {
                "className": "True",
                "columnName": "readmitted_True_PREDICTION"
              },
              {
                "className": "False",
                "columnName": "readmitted_False_PREDICTION"
              }
          ],
      "associationIdColumn": "id",
      "actualsValueColumn": "loan_amnt"
  },
  "monitoringOutputSettings": {
      "uniqueRowIdentifierColumns": ["id"],
      "monitoredStatusColumn": "monitored"
  }
  "schedule": {
      "minute": [ 0  ],
      "hour": [ 17   ],
      "dayOfWeek": ["*" ],
      "dayOfMonth": ["*" ],
      "month": [ "*” ]
  },
  "enabled": true
}
Updated August 25, 2023
Back to top