Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

DataRobot API resources > API reference documentation > DataRobot REST API > Monitoring > Accuracy

Observability Accuracy

This page outlines the operations, endpoints, parameters, and example requests and responses for the Observability Accuracy.

GET /api/v2/deployments/{deploymentId}/accuracy/

Retrieve accuracy metric for a certain time period.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/accuracy/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
start query string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
end query string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
modelId query any false The ID of the models for which metrics are being retrieved.
batchId query any false The id of the batch for which metrics are being retrieved.
segmentAttribute query string,null false The name of the segment on which segment analysis is being performed.
segmentValue query string,null false The value of the segmentAttribute to segment on.
targetClass query any false Target class to filter out results.
metric query string false Name of the metric to retrieve. Must be provided when using multiple modelId query params.
deploymentId path string true Unique identifier of the deployment.

Enumerated Values

Parameter Value
metric [AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE]

Example responses

200 Response

{
  "batchIds": [
    "string"
  ],
  "data": [
    {
      "baselineValue": 0,
      "metricName": "string",
      "modelId": "string",
      "percentChange": 0,
      "value": 0
    }
  ],
  "metrics": "\n                {\"metrics\": {\n                    \"LogLoss\": {\n                        \"baselineValue\": 0.454221484838069,\n                        \"value\": 0.880778024500618,\n                        \"percentChange\": -93.91\n                    },\n                    \"AUC\": {\n                        \"baselineValue\": 0.8690358459556535,\n                        \"value\": 0.5294117647058824,\n                        \"percentChange\": -39.08\n                    },\n                    \"Kolmogorov-Smirnov\": {\n                        \"baselineValue\": 0.5753202944706626,\n                        \"value\": 0.4117647058823529,\n                        \"percentChange\": -28.43\n                    },\n                    \"Rate@Top10%\": {\n                        \"baselineValue\": 0.9603223806571606,\n                        \"value\": 1.0,\n                        \"percentChange\": 4.13\n                    },\n                    \"Gini Norm\": {\n                        \"baselineValue\": 0.7380716919113071,\n                        \"value\": 0.05882352941176472,\n                        \"percentChange\": -92.03\n                    }\n                }\n                ",
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "segmentAttribute": "string",
  "segmentValue": ""
}

Responses

Status Meaning Description Schema
200 OK Deployment accuracy metrics are retrieved. AccuracyResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/deployments/{deploymentId}/accuracyOverBatch/

Retrieve accuracy metric baseline, and metric value calculated for each batch.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/accuracyOverBatch/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
segmentAttribute query string,null false The name of the segment on which segment analysis is being performed.
segmentValue query string,null false The value of the segmentAttribute to segment on.
modelId query string false The id of the model for which metrics are being retrieved.
batchId query any false The id of the batch for which metrics are being retrieved.
metric query string false Accuracy metric being requested.
deploymentId path string true Unique identifier of the deployment.

Enumerated Values

Parameter Value
metric [AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance]

Example responses

200 Response

{
  "baselines": [
    {
      "metric": "string",
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "sampleSize": 0,
      "value": 0
    }
  ],
  "buckets": [
    {
      "batchId": "string",
      "batchName": "string",
      "metric": "string",
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "sampleSize": 0,
      "value": 0
    }
  ]
}

Responses

Status Meaning Description Schema
200 OK Retrieve accuracy metrics over batches for a deployment. AccuracyOverBatchResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/deployments/{deploymentId}/accuracyOverSpace/

Retrieve accuracy metric baseline, and metric value calculated for each geospatial h3 hexagon.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/accuracyOverSpace/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
start query string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
end query string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
modelId query string false The ID of the model for which metrics are being retrieved.
metric query string false Name of the metric.
geoFeatureName query string false The name of the geospatial feature.
deploymentId path string true Unique identifier of the deployment.

Enumerated Values

Parameter Value
metric [AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAD, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE]

Example responses

200 Response

{
  "baselines": [
    {
      "hexagon": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "sampleSize": 0,
      "value": 0
    }
  ],
  "buckets": [
    {
      "hexagon": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "value": 0
    }
  ],
  "geoFeatureName": "string",
  "metric": "AUC",
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  }
}

Responses

Status Meaning Description Schema
200 OK Accuracy over space AccuracyOverSpaceResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/deployments/{deploymentId}/accuracyOverTime/

Retrieve accuracy metric baseline, and metric value calculated for each time bucket dividing a longer time period.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/accuracyOverTime/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
start query string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
end query string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
bucketSize query string(duration) false The time duration of a bucket. Needs to be multiple of one hour. Can not be longer than the total length of the period. If not set, a default value will be calculated based on the start and end time.
modelId query any false The ID of the models for which metrics are being retrieved.
metric query string false Name of the metric.
segmentAttribute query string,null false The name of the segment on which segment analysis is being performed.
segmentValue query string,null false The value of the segmentAttribute to segment on.
targetClass query any false Target class to filter out results.
deploymentId path string true Unique identifier of the deployment.

Enumerated Values

Parameter Value
metric [AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAD, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE]

Example responses

200 Response

{
  "baseline": {
    "period": {
      "end": "2019-08-24T14:15:22Z",
      "start": "2019-08-24T14:15:22Z"
    },
    "sampleSize": 0,
    "value": 0,
    "valuePerClass": {}
  },
  "baselines": [
    {
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "sampleSize": 0,
      "value": 0
    }
  ],
  "buckets": [
    {
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "sampleSize": 0,
      "value": 0,
      "valuePerClass": {}
    }
  ],
  "metric": "AUC",
  "modelId": "string",
  "segmentAttribute": "string",
  "segmentValue": "",
  "summary": {
    "period": {
      "end": "2019-08-24T14:15:22Z",
      "start": "2019-08-24T14:15:22Z"
    },
    "sampleSize": 0,
    "value": 0,
    "valuePerClass": {}
  }
}

Responses

Status Meaning Description Schema
200 OK Accuracy over time. AccuracyOverTimeResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverBatch/

Retrieve metrics about predictions and actuals, such as mean predicted & actual value, predicted & actual class distribution, over a specific set of batches.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/predictionsVsActualsOverBatch/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
segmentAttribute query string,null false The name of the segment on which segment analysis is being performed.
segmentValue query string,null false The value of the segmentAttribute to segment on.
modelId query string false The id of the model for which metrics are being retrieved.
batchId query any false The id of the batch for which metrics are being retrieved.
targetClass query any false Target class to filter out results.
deploymentId path string true Unique identifier of the deployment.

Example responses

200 Response

{
  "baselines": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "buckets": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "batchId": "string",
      "batchName": "string",
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "segmentAttribute": "string",
  "segmentValue": ""
}

Responses

Status Meaning Description Schema
200 OK Predictions vs actuals over batch info. PredictionsVsActualsOverBatchResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverSpace/

Retrieve predictions vs. actuals over space.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/predictionsVsActualsOverSpace/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
start query string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
end query string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
modelId query string false The ID of the model for which metrics are being retrieved.
geoFeatureName query string false The name of the geospatial feature.
targetClass query any false Target class to filter out results.
deploymentId path string true Unique identifier of the deployment.

Example responses

200 Response

{
  "baselines": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "hexagon": "string",
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "buckets": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "hexagon": "string",
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "geoFeatureName": "string",
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "summary": {
    "rowCountTotal": 0,
    "rowCountWithActual": 0
  }
}

Responses

Status Meaning Description Schema
200 OK Predictions vs. actuals over space. PredictionsVsActualsOverSpaceResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverTime/

Retrieve metrics about predictions and actuals, such as mean predicted & actual value, predicted & actual class distribution, over a specific time range.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/predictionsVsActualsOverTime/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
start query string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
end query string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
bucketSize query string false Time duration of buckets
segmentAttribute query string,null false The name of the segment on which segment analysis is being performed.
segmentValue query string,null false The value of the segmentAttribute to segment on.
modelId query any false The ID of the models for which metrics are being retrieved.
targetClass query any false Target class to filter out results.
deploymentId path string true Unique identifier of the deployment.

Enumerated Values

Parameter Value
bucketSize [PT1H, P1D, P7D, P1M]

Example responses

200 Response

{
  "baselines": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "buckets": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "segmentAttribute": "string",
  "segmentValue": "",
  "summary": {
    "rowCountTotal": 0,
    "rowCountWithActual": 0
  }
}

Responses

Status Meaning Description Schema
200 OK Predictions vs actuals over time info. PredictionsVsActualsOverTimeResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas

AccuracyBatchBaselineBucket

{
  "metric": "string",
  "modelId": "string",
  "perClass": [
    {
      "className": "string",
      "value": 0
    }
  ],
  "sampleSize": 0,
  "value": 0
}

Properties

Name Type Required Restrictions Description
metric string true Name of the metric.
modelId string false The id of the model for which metrics are being retrieved.
perClass [AccuracyPerClass] false Accuracy metric for selected classes.
sampleSize integer true Number of rows used to calculate the metric.
value number,null true Accuracy metric value.

AccuracyBatchBucket

{
  "batchId": "string",
  "batchName": "string",
  "metric": "string",
  "modelId": "string",
  "perClass": [
    {
      "className": "string",
      "value": 0
    }
  ],
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "sampleSize": 0,
  "value": 0
}

Properties

Name Type Required Restrictions Description
batchId string true ID of the batch.
batchName string true Name of the batch.
metric string true Name of the metric.
modelId string false The id of the model for which metrics are being retrieved.
perClass [AccuracyPerClass] false Accuracy metric for selected classes.
period BatchPeriod true Time period of the batch.
sampleSize integer true Number of rows used to calculate the metric.
value number,null true Accuracy metric value.

AccuracyLegacyTimeBucket

{
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "sampleSize": 0,
  "value": 0,
  "valuePerClass": {}
}

A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.

Properties

Name Type Required Restrictions Description
period TimeRange true An object with the keys "start" and "end" defining the period.
sampleSize integer,null true Number of predictions used to calculate the metric.
value number,null true Value of the metric, null if no value.
valuePerClass object,null false A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.

AccuracyMetric

{
  "baselineValue": 0,
  "metricName": "string",
  "modelId": "string",
  "percentChange": 0,
  "value": 0
}

Properties

Name Type Required Restrictions Description
baselineValue number,null true Baseline value of the metric.
metricName string true Name of the metric.
modelId string false The id of the model for which metrics are being retrieved.
percentChange number,null true Percent of change by comparing metric value to baseline, with metric direction taken into account.
value number,null true Value of the metric.

AccuracyOverBatchResponse

{
  "baselines": [
    {
      "metric": "string",
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "sampleSize": 0,
      "value": 0
    }
  ],
  "buckets": [
    {
      "batchId": "string",
      "batchName": "string",
      "metric": "string",
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "sampleSize": 0,
      "value": 0
    }
  ]
}

Properties

Name Type Required Restrictions Description
baselines [AccuracyBatchBaselineBucket] true Accuracy metric training baseline.
buckets [AccuracyBatchBucket] true Accuracy metric for batches that are requested, with non-existing batches omitted.

AccuracyOverSpaceBaselineBucket

{
  "hexagon": "string",
  "perClass": [
    {
      "className": "string",
      "value": 0
    }
  ],
  "sampleSize": 0,
  "value": 0
}

Properties

Name Type Required Restrictions Description
hexagon string true h3 hexagon.
perClass [AccuracyPerClass] false maxItems: 100
 Accuracy metric values for selected classes, only available for multiclass deployments.
sampleSize integer true Number of rows used to calculate the metric.
value number,null true Accuracy metric value.

AccuracyOverSpaceBucket

{
  "hexagon": "string",
  "perClass": [
    {
      "className": "string",
      "value": 0
    }
  ],
  "value": 0
}

Properties

Name Type Required Restrictions Description
hexagon string true h3 hexagon.
perClass [AccuracyPerClass] false maxItems: 100
 Accuracy metric values for selected classes, only available for multiclass deployments.
value number,null true Accuracy metric value.

AccuracyOverSpaceResponse

{
  "baselines": [
    {
      "hexagon": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "sampleSize": 0,
      "value": 0
    }
  ],
  "buckets": [
    {
      "hexagon": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "value": 0
    }
  ],
  "geoFeatureName": "string",
  "metric": "AUC",
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  }
}

Properties

Name Type Required Restrictions Description
baselines [AccuracyOverSpaceBaselineBucket] true maxItems: 1000
 Baseline accuracy per geospatial hexagon.
buckets [AccuracyOverSpaceBucket] true maxItems: 1000
 Accuracy per geospatial hexagon.
geoFeatureName string true The name of the geospatial feature. Segmented analysis must be enabled for the feature specified.
metric string true The metric being retrieved.
modelId string false The ID of the model for which metrics are being retrieved.
period TimeRange false An object with the keys "start" and "end" defining the period.

Enumerated Values

Property Value
metric [AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE]

AccuracyOverTimeResponse

{
  "baseline": {
    "period": {
      "end": "2019-08-24T14:15:22Z",
      "start": "2019-08-24T14:15:22Z"
    },
    "sampleSize": 0,
    "value": 0,
    "valuePerClass": {}
  },
  "baselines": [
    {
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "sampleSize": 0,
      "value": 0
    }
  ],
  "buckets": [
    {
      "modelId": "string",
      "perClass": [
        {
          "className": "string",
          "value": 0
        }
      ],
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "sampleSize": 0,
      "value": 0,
      "valuePerClass": {}
    }
  ],
  "metric": "AUC",
  "modelId": "string",
  "segmentAttribute": "string",
  "segmentValue": "",
  "summary": {
    "period": {
      "end": "2019-08-24T14:15:22Z",
      "start": "2019-08-24T14:15:22Z"
    },
    "sampleSize": 0,
    "value": 0,
    "valuePerClass": {}
  }
}

Properties

Name Type Required Restrictions Description
baseline AccuracyLegacyTimeBucket true A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.
baselines [AccuracyTimeBaselineBucket] true Accuracy metric training baseline.
buckets [AccuracyTimeBucket] true Accuracy metric for requested models and time buckets.
metric string true The metric being retrieved.
modelId string false The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.35, use modelId in each baseline or bucket object.
segmentAttribute string,null false The name of the segment on which segment analysis is being performed.
segmentValue string,null false The value of the segmentAttribute to segment on.
summary AccuracyLegacyTimeBucket true A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.

Enumerated Values

Property Value
metric [AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE]

AccuracyPerClass

{
  "className": "string",
  "value": 0
}

Properties

Name Type Required Restrictions Description
className string true Name of the class.
value number,null true Value of the metric.

AccuracyResponse

{
  "batchIds": [
    "string"
  ],
  "data": [
    {
      "baselineValue": 0,
      "metricName": "string",
      "modelId": "string",
      "percentChange": 0,
      "value": 0
    }
  ],
  "metrics": "\n                {\"metrics\": {\n                    \"LogLoss\": {\n                        \"baselineValue\": 0.454221484838069,\n                        \"value\": 0.880778024500618,\n                        \"percentChange\": -93.91\n                    },\n                    \"AUC\": {\n                        \"baselineValue\": 0.8690358459556535,\n                        \"value\": 0.5294117647058824,\n                        \"percentChange\": -39.08\n                    },\n                    \"Kolmogorov-Smirnov\": {\n                        \"baselineValue\": 0.5753202944706626,\n                        \"value\": 0.4117647058823529,\n                        \"percentChange\": -28.43\n                    },\n                    \"Rate@Top10%\": {\n                        \"baselineValue\": 0.9603223806571606,\n                        \"value\": 1.0,\n                        \"percentChange\": 4.13\n                    },\n                    \"Gini Norm\": {\n                        \"baselineValue\": 0.7380716919113071,\n                        \"value\": 0.05882352941176472,\n                        \"percentChange\": -92.03\n                    }\n                }\n                ",
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "segmentAttribute": "string",
  "segmentValue": ""
}

Properties

Name Type Required Restrictions Description
batchIds [string] false ID of the batches used to calculate accuracy metrics.
data [AccuracyMetric] true maxItems: 100
 Accuracy metric data.
metrics object false Accuracy metrics of the deployment. Deprecated and to be removed in v2.40; use data objects.
modelId string false The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.40; use modelId in each data object.
period TimeRange false An object with the keys "start" and "end" defining the period.
segmentAttribute string,null false The name of the segment on which segment analysis is being performed.
segmentValue string,null false The value of the segmentAttribute to segment on.

AccuracyTimeBaselineBucket

{
  "modelId": "string",
  "perClass": [
    {
      "className": "string",
      "value": 0
    }
  ],
  "sampleSize": 0,
  "value": 0
}

Properties

Name Type Required Restrictions Description
modelId string false The id of the model for which metrics are being retrieved.
perClass [AccuracyPerClass] false Accuracy metric values for selected classes, only available for multiclass deployments.
sampleSize integer true Number of rows used to calculate the metric.
value number,null true Accuracy metric value.

AccuracyTimeBucket

{
  "modelId": "string",
  "perClass": [
    {
      "className": "string",
      "value": 0
    }
  ],
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "sampleSize": 0,
  "value": 0,
  "valuePerClass": {}
}

Properties

Name Type Required Restrictions Description
modelId string false The id of the model for which metrics are being retrieved.
perClass [AccuracyPerClass] false Accuracy metric values for selected classes, only available for multiclass deployments.
period TimeRange true An object with the keys "start" and "end" defining the period.
sampleSize integer true Number of rows used to calculate the metric.
value number,null true Accuracy metric value.
valuePerClass object,null false A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.

ActualClassDistribution

{
  "className": "string",
  "count": 0,
  "percent": 0
}

Properties

Name Type Required Restrictions Description
className string true Name of the class.
count integer true Count of actual rows labeled with a class in the bucket.
percent number true Percent of actual rows labeled with a class in the bucket.

BatchPeriod

{
  "end": "2019-08-24T14:15:22Z",
  "start": "2019-08-24T14:15:22Z"
}

Time period of the batch.

Properties

Name Type Required Restrictions Description
end string,null(date-time) true End time of the bucket
start string,null(date-time) true Start time of the bucket

PredictedClassDistribution

{
  "className": "string",
  "count": 0,
  "percent": 0
}

Properties

Name Type Required Restrictions Description
className string true Name of the class.
count integer true Count of prediction rows labeled with a class in the bucket.
percent number true Percent of prediction rows labeled with a class in the bucket.

PredictionsVsActualsBaseline

{
  "actualClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "meanActualValue": 0,
  "meanPredictedValue": 0,
  "modelId": "string",
  "predictedClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "rowCountTotal": 0,
  "rowCountWithActual": 0
}

Properties

Name Type Required Restrictions Description
actualClassDistribution [ActualClassDistribution] false Class distribution for all actuals in the bucket, only for classification deployments.
meanActualValue number,null false Mean actual value for all rows in the bucket, only for regression deployments.
meanPredictedValue number,null false Mean predicted value for all rows in the bucket, only for regression deployments.
modelId string true ID of the model.
predictedClassDistribution [PredictedClassDistribution] false Class distribution for all rows with actual in the bucket, only for classification deployments.
rowCountTotal integer true Number of rows in the bucket.
rowCountWithActual integer true Number of rows with actual in the bucket.

PredictionsVsActualsBatchBucket

{
  "actualClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "batchId": "string",
  "batchName": "string",
  "meanActualValue": 0,
  "meanPredictedValue": 0,
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "predictedClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "rowCountTotal": 0,
  "rowCountWithActual": 0
}

Properties

Name Type Required Restrictions Description
actualClassDistribution [ActualClassDistribution] false Class distribution for all actuals in the bucket, only for classification deployments.
batchId string true ID of the batch.
batchName string true Name of the batch.
meanActualValue number,null false Mean actual value for all rows in the bucket, only for regression deployments.
meanPredictedValue number,null false Mean predicted value for all rows in the bucket, only for regression deployments.
modelId string true ID of the model.
period BatchPeriod true Time period of the batch.
predictedClassDistribution [PredictedClassDistribution] false Class distribution for all rows with actual in the bucket, only for classification deployments.
rowCountTotal integer true Number of rows in the bucket.
rowCountWithActual integer true Number of rows with actual in the bucket.

PredictionsVsActualsOverBatchResponse

{
  "baselines": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "buckets": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "batchId": "string",
      "batchName": "string",
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "segmentAttribute": "string",
  "segmentValue": ""
}

Properties

Name Type Required Restrictions Description
baselines [PredictionsVsActualsBaseline] true Predictions vs actuals baselines.
buckets [PredictionsVsActualsBatchBucket] true Predictions vs actuals buckets.
segmentAttribute string,null false The name of the segment on which segment analysis is being performed.
segmentValue string,null false The value of the segmentAttribute to segment on.

PredictionsVsActualsOverSpaceBucket

{
  "actualClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "hexagon": "string",
  "meanActualValue": 0,
  "meanPredictedValue": 0,
  "predictedClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "rowCountTotal": 0,
  "rowCountWithActual": 0
}

Properties

Name Type Required Restrictions Description
actualClassDistribution [ActualClassDistribution] false maxItems: 100
 For classification deployments, the class distribution for all actuals in the bucket.
hexagon string true The H3 geospatial indexing hexagon.
meanActualValue number,null false For regression deployments, the mean actual value of all rows in the bucket.
meanPredictedValue number,null false For regression deployments, the mean predicted value of all rows in the bucket.
predictedClassDistribution [PredictedClassDistribution] false maxItems: 100
 For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.
rowCountTotal integer true The number of rows in the bucket.
rowCountWithActual integer true The number of prediction rows in the bucket with associated actuals.

PredictionsVsActualsOverSpaceResponse

{
  "baselines": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "hexagon": "string",
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "buckets": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "hexagon": "string",
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "geoFeatureName": "string",
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "summary": {
    "rowCountTotal": 0,
    "rowCountWithActual": 0
  }
}

Properties

Name Type Required Restrictions Description
baselines [PredictionsVsActualsOverSpaceBucket] true maxItems: 1000
 The predictions vs. actuals over space baselines.
buckets [PredictionsVsActualsOverSpaceBucket] true maxItems: 1000
 The predictions vs. actuals over space buckets.
geoFeatureName string true The name of the geospatial feature.
modelId string false The ID of the model for which metrics are being retrieved.
period TimeRange false An object with the keys "start" and "end" defining the period.
summary PredictionsVsActualsSummaryBucket true Predictions vs actuals summary.

PredictionsVsActualsOverTimeResponse

{
  "baselines": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "buckets": [
    {
      "actualClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "meanActualValue": 0,
      "meanPredictedValue": 0,
      "modelId": "string",
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "predictedClassDistribution": [
        {
          "className": "string",
          "count": 0,
          "percent": 0
        }
      ],
      "rowCountTotal": 0,
      "rowCountWithActual": 0
    }
  ],
  "segmentAttribute": "string",
  "segmentValue": "",
  "summary": {
    "rowCountTotal": 0,
    "rowCountWithActual": 0
  }
}

Properties

Name Type Required Restrictions Description
baselines [PredictionsVsActualsBaseline] true Predictions vs actuals baselines.
buckets [PredictionsVsActualsTimeBucket] true Predictions vs actuals buckets.
segmentAttribute string,null false The name of the segment on which segment analysis is being performed.
segmentValue string,null false The value of the segmentAttribute to segment on.
summary PredictionsVsActualsSummaryBucket true Predictions vs actuals summary.

PredictionsVsActualsSummaryBucket

{
  "rowCountTotal": 0,
  "rowCountWithActual": 0
}

Predictions vs actuals summary.

Properties

Name Type Required Restrictions Description
rowCountTotal integer true Number of rows for all buckets.
rowCountWithActual integer true Number of rows with actual for all buckets.

PredictionsVsActualsTimeBucket

{
  "actualClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "meanActualValue": 0,
  "meanPredictedValue": 0,
  "modelId": "string",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "predictedClassDistribution": [
    {
      "className": "string",
      "count": 0,
      "percent": 0
    }
  ],
  "rowCountTotal": 0,
  "rowCountWithActual": 0
}

Properties

Name Type Required Restrictions Description
actualClassDistribution [ActualClassDistribution] false Class distribution for all actuals in the bucket, only for classification deployments.
meanActualValue number,null false Mean actual value for all rows in the bucket, only for regression deployments.
meanPredictedValue number,null false Mean predicted value for all rows in the bucket, only for regression deployments.
modelId string true ID of the model.
period BatchPeriod true Time period of the batch.
predictedClassDistribution [PredictedClassDistribution] false Class distribution for all rows with actual in the bucket, only for classification deployments.
rowCountTotal integer true Number of rows in the bucket.
rowCountWithActual integer true Number of rows with actual in the bucket.

TimeRange

{
  "end": "2019-08-24T14:15:22Z",
  "start": "2019-08-24T14:15:22Z"
}

An object with the keys "start" and "end" defining the period.

Properties

Name Type Required Restrictions Description
end string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
start string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
Updated March 19, 2025