# Monitoring jobs API
URL: https://docs.datarobot.com/en/docs/classic-ui/predictions/batch/pred-monitoring-jobs/api-monitoring-jobs.html

> Use the Batch Monitoring API to create monitoring job definitions, allowing DataRobot to monitor deployments running and storing feature data and predictions outside of DataRobot.

# Monitoring jobs API

This integration creates a Batch Monitoring API with `batchMonitoringJobDefinitions` and `batchJobs` endpoints, allowing you to create monitoring jobs. Monitoring job [intake](https://docs.datarobot.com/en/docs/api/reference/batch-prediction-api/intake-options.html) and [output](https://docs.datarobot.com/en/docs/api/reference/batch-prediction-api/output-options.html) settings are configured using the same options as batch prediction jobs. Use the following routes, properties, and examples to create monitoring jobs:

> [!NOTE] Service health information for external models and monitoring jobs
> Service health information is unavailable for external [agent-monitored deployments](https://docs.datarobot.com/en/docs/classic-ui/mlops/deployment/mlops-agent/monitoring-agent/index.html) and deployments with predictions uploaded through a [prediction monitoring job](https://docs.datarobot.com/en/docs/classic-ui/predictions/batch/pred-monitoring-jobs/index.html).

> [!NOTE] Time series model consideration
> Monitoring jobs don't support monitoring predictions made by time series models.

## Monitoring job definition and batch job routes

### batchMonitoringJobDefinitions endpoints

Access endpoints for performing operations on batch monitoring job definitions:

| Operation and endpoint | Description |
| --- | --- |
| POST /api/v2/batchMonitoringJobDefinitions/ | Create a monitoring job definition given a payload. |
| GET /api/v2/batchMonitoringJobDefinitions/ | List all monitoring job definitions. |
| GET /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ | Retrieve the specified monitoring job definition. |
| DELETE /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ | Delete the specified monitoring job definition. |
| PATCH /api/v2/batchMonitoringJobDefinitions/{monitoringJobDefinitionId}/ | Update the specified monitoring job definition given a payload. |

### batchJobs endpoints

Access endpoints for performing operations on batch jobs:

| Operation and endpoint | Description |
| --- | --- |
| POST /api/v2/batchJobs/fromJobDefinition/ | Launch (run now) a monitoring job from a monitoringJobDefinition. The payload should contain the monitoringJobDefinitionId. |
| GET /api/v2/batchJobs/ | List the full history of monitoring jobs, including running, aborted, and executed jobs. |
| GET /api/v2/batchJobs/{monitoringJobId}/ | Retrieve a specific monitoring job. |
| DELETE /api/v2/batchJobs/{monitoringJobId}/ | Abort a running monitoring job. |

## Monitoring job properties

### monitoringColumns properties

Define which columns to use for batch monitoring:

| Property | Type | Description |
| --- | --- | --- |
| predictionsColumns | string | (Regression) The column in the data source containing prediction values. You must provide this field and/or actualsValueColumn. |
| predictionsColumns | array | (Classification) The columns in the data source containing each prediction class. You must provide this field and/or actualsValueColumn. (Supports a maximum of 1000 items) |
| associationIdColumn | string | The column in the data source which contains the association ID for predictions. |
| actualsValueColumn | string | The column in the data source which contains actual values. You must provide this field and/or predictionsColumns. |
| actualsTimestampColumn | string | The column in the data source which contains the timestamps for actual values. |

### monitoringOutputSettings properties

Configure the output settings specific to monitoring jobs:

| Property | Type | Description |
| --- | --- | --- |
| uniqueRowIdentifierColumns | array | Columns from the data source that will serve as unique identifiers for each row. These columns are copied to the data destination to associate each monitored status with its corresponding source row. (Supports a maximum of 100 items) |
| monitoredStatusColumn | string | The column in the data destination containing the monitoring status for each row. |

> [!NOTE] Note
> For general batch job output settings, see the [Prediction output settings](https://docs.datarobot.com/en/docs/api/reference/batch-prediction-api/output-options.html) documentation.

### monitoringAggregation properties

To support challengers for external models with [large-scale monitoring](https://docs.datarobot.com/en/docs/classic-ui/mlops/deployment/mlops-agent/monitoring-agent/agent-use.html#enable-large-scale-monitoring) enabled (meaning that raw data isn't stored in the DataRobot platform), you can report a small sample of raw feature and predictions data; then, you can send the remaining data in aggregate format. Configure the retention settings to indicate that raw data is aggregated by the MLOps library and define how much raw data should be retained for challengers.

> [!NOTE] Autosampling for large-scale monitoring
> To automatically report a small sample of raw data for challenger analysis and accuracy monitoring, you can define the `MLOPS_STATS_AGGREGATION_AUTO_SAMPLING_PERCENTAGE` when [enabling large-scale monitoring for an external model](https://docs.datarobot.com/en/docs/classic-ui/mlops/deployment/mlops-agent/monitoring-agent/agent-use.html#enable-large-scale-monitoring).

| Property | Type | Description |
| --- | --- | --- |
| retentionPolicy | string | The policy definition determines if the retentionValue represents a number of samples or a percentage of the dataset. enum: ['samples', 'percentage'] |
| retentionValue | integer | The amount of data to retain, either a percentage of data or the number of samples. |

If you define these properties, raw data is aggregated by the MLOps library. This means that the data isn't stored in the DataRobot platform. Stats aggregation only supports feature and prediction data, not actuals data for accuracy monitoring. If you've defined `actualsValueColumn` or `associationIdColumn` (which means actuals will be provided later), DataRobot cannot aggregate data.

> [!NOTE] Preview: Accuracy monitoring with aggregation
> Now available for preview, monitoring jobs for external models with aggregation enabled can support accuracy tracking. With this feature enabled, when you configure the retention settings and define the `actualsValueColumn` for accuracy monitoring with aggregation enabled, you must also define the `predictionsColumns` and `associationIdColumn`.
> 
> Feature flag OFF by default: Enable Accuracy Aggregation

## Monitoring job examples

**Example: Regression monitoring job payload:**
```
{
  "batchJobType": "monitoring",
  "deploymentId": "<deployment_id>",
  "intakeSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "credentialId": "<credential_id>",
      "table": "lending_club_regression",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "catalog": "SANDBOX"
  },
  "outputSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "table": "lending_club_regression_out",
      "catalog": "SANDBOX",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "statementType": "insert",
      "createTableIfNotExists": true,
      "credentialId": "<credential_id>",
      "commitInterval": 10,
      "whereColumns": [],
      "updateColumns": []
  },
  "passthroughColumns": [],
  "monitoringColumns": {
      "predictionsColumns": "PREDICTION",
      "associationIdColumn": "id",
      "actualsValueColumn": "loan_amnt"
  },
  "monitoringOutputSettings": {
     "monitoredStatusColumn": "monitored",
     "uniqueRowIdentifierColumns": ["id"]
  }
  "schedule": {
      "minute": [ 0  ],
      "hour": [ 17   ],
      "dayOfWeek": ["*" ],
      "dayOfMonth": ["*" ],
      "month": [ "*” ]
  },
  "enabled": true
}
```

**Example: Classification monitoring job payload:**
```
{
  "batchJobType": "monitoring",
  "deploymentId": "<deployment_id>",
  "intakeSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "credentialId": "<credential_id>",
      "table": "lending_club_regression",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "catalog": "SANDBOX"
  },
  "outputSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "table": "lending_club_regression_out",
      "catalog": "SANDBOX",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "statementType": "insert",
      "createTableIfNotExists": true,
      "credentialId": "<credential_id>",
      "commitInterval": 10,
      "whereColumns": [],
      "updateColumns": []
  },
  "monitoringColumns": {
  "predictionsColumns": [
              {
                "className": "True",
                "columnName": "readmitted_True_PREDICTION"
              },
              {
                "className": "False",
                "columnName": "readmitted_False_PREDICTION"
              }
          ],
      "associationIdColumn": "id",
      "actualsValueColumn": "loan_amnt"
  },
  "monitoringOutputSettings": {
      "uniqueRowIdentifierColumns": ["id"],
      "monitoredStatusColumn": "monitored"
  }
  "schedule": {
      "minute": [ 0  ],
      "hour": [ 17   ],
      "dayOfWeek": ["*" ],
      "dayOfMonth": ["*" ],
      "month": [ "*” ]
  },
  "enabled": true
}
```

---
