Observability Accuracy
This page outlines the operations, endpoints, parameters, and example requests and responses for the Observability Accuracy.
GET /api/v2/deployments/{deploymentId}/accuracy/
Retrieve accuracy metric for a certain time period.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /accuracy/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
start
query
string,null(date-time)
false
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
end
query
string,null(date-time)
false
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
modelId
query
any
false
The ID of the models for which metrics are being retrieved.
batchId
query
any
false
The id of the batch for which metrics are being retrieved.
segmentAttribute
query
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
query
string,null
false
The value of the segmentAttribute
to segment on.
targetClass
query
any
false
Target class to filter out results.
metric
query
string
false
Name of the metric to retrieve. Must be provided when using multiple modelId query params.
deploymentId
path
string
true
Unique identifier of the deployment.
Enumerated Values
Parameter
Value
metric
[AUC
, Accuracy
, Balanced Accuracy
, F1
, FPR
, FVE Binomial
, FVE Gamma
, FVE Multinomial
, FVE Poisson
, FVE Tweedie
, Gamma Deviance
, Gini Norm
, Kolmogorov-Smirnov
, LogLoss
, MAE
, MAPE
, MCC
, NPV
, PPV
, Poisson Deviance
, R Squared
, RMSE
, RMSLE
, Rate@Top10%
, Rate@Top5%
, TNR
, TPR
, Tweedie Deviance
, WGS84 MAE
, WGS84 RMSE
]
Example responses
200 Response
{
"batchIds" : [
"string"
],
"data" : [
{
"baselineValue" : 0 ,
"metricName" : "string" ,
"modelId" : "string" ,
"percentChange" : 0 ,
"value" : 0
}
],
"metrics" : "\n {\"metrics\": {\n \"LogLoss\": {\n \"baselineValue\": 0.454221484838069,\n \"value\": 0.880778024500618,\n \"percentChange\": -93.91\n },\n \"AUC\": {\n \"baselineValue\": 0.8690358459556535,\n \"value\": 0.5294117647058824,\n \"percentChange\": -39.08\n },\n \"Kolmogorov-Smirnov\": {\n \"baselineValue\": 0.5753202944706626,\n \"value\": 0.4117647058823529,\n \"percentChange\": -28.43\n },\n \"Rate@Top10%\": {\n \"baselineValue\": 0.9603223806571606,\n \"value\": 1.0,\n \"percentChange\": 4.13\n },\n \"Gini Norm\": {\n \"baselineValue\": 0.7380716919113071,\n \"value\": 0.05882352941176472,\n \"percentChange\": -92.03\n }\n }\n " ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"segmentAttribute" : "string" ,
"segmentValue" : ""
}
Responses
Status
Meaning
Description
Schema
200
OK
Deployment accuracy metrics are retrieved.
AccuracyResponse
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyOverBatch/
Retrieve accuracy metric baseline, and metric value calculated for each batch.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /accuracyOverBatch/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
segmentAttribute
query
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
query
string,null
false
The value of the segmentAttribute
to segment on.
modelId
query
string
false
The id of the model for which metrics are being retrieved.
batchId
query
any
false
The id of the batch for which metrics are being retrieved.
metric
query
string
false
Accuracy metric being requested.
deploymentId
path
string
true
Unique identifier of the deployment.
Enumerated Values
Parameter
Value
metric
[AUC
, Accuracy
, Balanced Accuracy
, F1
, FPR
, FVE Binomial
, FVE Gamma
, FVE Poisson
, FVE Tweedie
, Gamma Deviance
, Gini Norm
, Kolmogorov-Smirnov
, LogLoss
, MAE
, MAPE
, MCC
, NPV
, PPV
, Poisson Deviance
, R Squared
, RMSE
, RMSLE
, Rate@Top10%
, Rate@Top5%
, TNR
, TPR
, Tweedie Deviance
]
Example responses
200 Response
{
"baselines" : [
{
"metric" : "string" ,
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
],
"buckets" : [
{
"batchId" : "string" ,
"batchName" : "string" ,
"metric" : "string" ,
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0
}
]
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyOverSpace/
Retrieve accuracy metric baseline, and metric value calculated for each geospatial h3 hexagon.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /accuracyOverSpace/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
start
query
string,null(date-time)
false
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
end
query
string,null(date-time)
false
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
modelId
query
string
false
The ID of the model for which metrics are being retrieved.
metric
query
string
false
Name of the metric.
geoFeatureName
query
string
false
The name of the geospatial feature.
deploymentId
path
string
true
Unique identifier of the deployment.
Enumerated Values
Parameter
Value
metric
[AUC
, Accuracy
, Balanced Accuracy
, F1
, FPR
, FVE Binomial
, FVE Gamma
, FVE Multinomial
, FVE Poisson
, FVE Tweedie
, Gamma Deviance
, Gini Norm
, Kolmogorov-Smirnov
, LogLoss
, MAD
, MAE
, MAPE
, MCC
, NPV
, PPV
, Poisson Deviance
, R Squared
, RMSE
, RMSLE
, Rate@Top10%
, Rate@Top5%
, TNR
, TPR
, Tweedie Deviance
, WGS84 MAE
, WGS84 RMSE
]
Example responses
200 Response
{
"baselines" : [
{
"hexagon" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
],
"buckets" : [
{
"hexagon" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"value" : 0
}
],
"geoFeatureName" : "string" ,
"metric" : "AUC" ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
}
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyOverTime/
Retrieve accuracy metric baseline, and metric value calculated for each time bucket dividing a longer time period.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /accuracyOverTime/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
start
query
string,null(date-time)
false
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
end
query
string,null(date-time)
false
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
bucketSize
query
string(duration)
false
The time duration of a bucket. Needs to be multiple of one hour. Can not be longer than the total length of the period. If not set, a default value will be calculated based on the start and end time.
modelId
query
any
false
The ID of the models for which metrics are being retrieved.
metric
query
string
false
Name of the metric.
segmentAttribute
query
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
query
string,null
false
The value of the segmentAttribute
to segment on.
targetClass
query
any
false
Target class to filter out results.
deploymentId
path
string
true
Unique identifier of the deployment.
Enumerated Values
Parameter
Value
metric
[AUC
, Accuracy
, Balanced Accuracy
, F1
, FPR
, FVE Binomial
, FVE Gamma
, FVE Multinomial
, FVE Poisson
, FVE Tweedie
, Gamma Deviance
, Gini Norm
, Kolmogorov-Smirnov
, LogLoss
, MAD
, MAE
, MAPE
, MCC
, NPV
, PPV
, Poisson Deviance
, R Squared
, RMSE
, RMSLE
, Rate@Top10%
, Rate@Top5%
, TNR
, TPR
, Tweedie Deviance
, WGS84 MAE
, WGS84 RMSE
]
Example responses
200 Response
{
"baseline" : {
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
},
"baselines" : [
{
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
],
"buckets" : [
{
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
}
],
"metric" : "AUC" ,
"modelId" : "string" ,
"segmentAttribute" : "string" ,
"segmentValue" : "" ,
"summary" : {
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
}
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverBatch/
Retrieve metrics about predictions and actuals, such as mean predicted & actual value, predicted & actual class distribution, over a specific set of batches.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /predictionsVsActualsOverBatch/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
segmentAttribute
query
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
query
string,null
false
The value of the segmentAttribute
to segment on.
modelId
query
string
false
The id of the model for which metrics are being retrieved.
batchId
query
any
false
The id of the batch for which metrics are being retrieved.
targetClass
query
any
false
Target class to filter out results.
deploymentId
path
string
true
Unique identifier of the deployment.
Example responses
200 Response
{
"baselines" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"buckets" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"batchId" : "string" ,
"batchName" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"segmentAttribute" : "string" ,
"segmentValue" : ""
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverSpace/
Retrieve predictions vs. actuals over space.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /predictionsVsActualsOverSpace/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
start
query
string,null(date-time)
false
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
end
query
string,null(date-time)
false
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
modelId
query
string
false
The ID of the model for which metrics are being retrieved.
geoFeatureName
query
string
false
The name of the geospatial feature.
targetClass
query
any
false
Target class to filter out results.
deploymentId
path
string
true
Unique identifier of the deployment.
Example responses
200 Response
{
"baselines" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"hexagon" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"buckets" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"hexagon" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"geoFeatureName" : "string" ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"summary" : {
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverTime/
Retrieve metrics about predictions and actuals, such as mean predicted & actual value, predicted & actual class distribution, over a specific time range.
Code samples
curl -X GET https://app.datarobot.com/api/v2/deployments/{ deploymentId} /predictionsVsActualsOverTime/ \
-H "Accept: application/json" \
-H "Authorization: Bearer {access-token}"
Parameters
Name
In
Type
Required
Description
start
query
string,null(date-time)
false
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
end
query
string,null(date-time)
false
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
bucketSize
query
string
false
Time duration of buckets
segmentAttribute
query
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
query
string,null
false
The value of the segmentAttribute
to segment on.
modelId
query
any
false
The ID of the models for which metrics are being retrieved.
targetClass
query
any
false
Target class to filter out results.
deploymentId
path
string
true
Unique identifier of the deployment.
Enumerated Values
Parameter
Value
bucketSize
[PT1H
, P1D
, P7D
, P1M
]
Example responses
200 Response
{
"baselines" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"buckets" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"segmentAttribute" : "string" ,
"segmentValue" : "" ,
"summary" : {
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
Schemas
AccuracyBatchBaselineBucket
{
"metric" : "string" ,
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
metric
string
true
Name of the metric.
modelId
string
false
The id of the model for which metrics are being retrieved.
perClass
[AccuracyPerClass ]
false
Accuracy metric for selected classes.
sampleSize
integer
true
Number of rows used to calculate the metric.
value
number,null
true
Accuracy metric value.
AccuracyBatchBucket
{
"batchId" : "string" ,
"batchName" : "string" ,
"metric" : "string" ,
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
batchId
string
true
ID of the batch.
batchName
string
true
Name of the batch.
metric
string
true
Name of the metric.
modelId
string
false
The id of the model for which metrics are being retrieved.
perClass
[AccuracyPerClass ]
false
Accuracy metric for selected classes.
period
BatchPeriod
true
Time period of the batch.
sampleSize
integer
true
Number of rows used to calculate the metric.
value
number,null
true
Accuracy metric value.
AccuracyLegacyTimeBucket
{
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
}
A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.
Properties
Name
Type
Required
Restrictions
Description
period
TimeRange
true
An object with the keys "start" and "end" defining the period.
sampleSize
integer,null
true
Number of predictions used to calculate the metric.
value
number,null
true
Value of the metric, null if no value.
valuePerClass
object,null
false
A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.
AccuracyMetric
{
"baselineValue" : 0 ,
"metricName" : "string" ,
"modelId" : "string" ,
"percentChange" : 0 ,
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
baselineValue
number,null
true
Baseline value of the metric.
metricName
string
true
Name of the metric.
modelId
string
false
The id of the model for which metrics are being retrieved.
percentChange
number,null
true
Percent of change by comparing metric value to baseline, with metric direction taken into account.
value
number,null
true
Value of the metric.
AccuracyOverBatchResponse
{
"baselines" : [
{
"metric" : "string" ,
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
],
"buckets" : [
{
"batchId" : "string" ,
"batchName" : "string" ,
"metric" : "string" ,
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0
}
]
}
Properties
Name
Type
Required
Restrictions
Description
baselines
[AccuracyBatchBaselineBucket ]
true
Accuracy metric training baseline.
buckets
[AccuracyBatchBucket ]
true
Accuracy metric for batches that are requested, with non-existing batches omitted.
AccuracyOverSpaceBaselineBucket
{
"hexagon" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
hexagon
string
true
h3 hexagon.
perClass
[AccuracyPerClass ]
false
maxItems: 100
Accuracy metric values for selected classes, only available for multiclass deployments.
sampleSize
integer
true
Number of rows used to calculate the metric.
value
number,null
true
Accuracy metric value.
AccuracyOverSpaceBucket
{
"hexagon" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
hexagon
string
true
h3 hexagon.
perClass
[AccuracyPerClass ]
false
maxItems: 100
Accuracy metric values for selected classes, only available for multiclass deployments.
value
number,null
true
Accuracy metric value.
AccuracyOverSpaceResponse
{
"baselines" : [
{
"hexagon" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
],
"buckets" : [
{
"hexagon" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"value" : 0
}
],
"geoFeatureName" : "string" ,
"metric" : "AUC" ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
}
}
Properties
Name
Type
Required
Restrictions
Description
baselines
[AccuracyOverSpaceBaselineBucket ]
true
maxItems: 1000
Baseline accuracy per geospatial hexagon.
buckets
[AccuracyOverSpaceBucket ]
true
maxItems: 1000
Accuracy per geospatial hexagon.
geoFeatureName
string
true
The name of the geospatial feature. Segmented analysis must be enabled for the feature specified.
metric
string
true
The metric being retrieved.
modelId
string
false
The ID of the model for which metrics are being retrieved.
period
TimeRange
false
An object with the keys "start" and "end" defining the period.
Enumerated Values
Property
Value
metric
[AUC
, Accuracy
, Balanced Accuracy
, F1
, FPR
, FVE Binomial
, FVE Gamma
, FVE Multinomial
, FVE Poisson
, FVE Tweedie
, Gamma Deviance
, Gini Norm
, Kolmogorov-Smirnov
, LogLoss
, MAE
, MAPE
, MCC
, NPV
, PPV
, Poisson Deviance
, R Squared
, RMSE
, RMSLE
, Rate@Top10%
, Rate@Top5%
, TNR
, TPR
, Tweedie Deviance
, WGS84 MAE
, WGS84 RMSE
]
AccuracyOverTimeResponse
{
"baseline" : {
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
},
"baselines" : [
{
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
],
"buckets" : [
{
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
}
],
"metric" : "AUC" ,
"modelId" : "string" ,
"segmentAttribute" : "string" ,
"segmentValue" : "" ,
"summary" : {
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
}
}
Properties
Name
Type
Required
Restrictions
Description
baseline
AccuracyLegacyTimeBucket
true
A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.
baselines
[AccuracyTimeBaselineBucket ]
true
Accuracy metric training baseline.
buckets
[AccuracyTimeBucket ]
true
Accuracy metric for requested models and time buckets.
metric
string
true
The metric being retrieved.
modelId
string
false
The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.35, use modelId in each baseline or bucket object.
segmentAttribute
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
string,null
false
The value of the segmentAttribute
to segment on.
summary
AccuracyLegacyTimeBucket
true
A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.
Enumerated Values
Property
Value
metric
[AUC
, Accuracy
, Balanced Accuracy
, F1
, FPR
, FVE Binomial
, FVE Gamma
, FVE Multinomial
, FVE Poisson
, FVE Tweedie
, Gamma Deviance
, Gini Norm
, Kolmogorov-Smirnov
, LogLoss
, MAE
, MAPE
, MCC
, NPV
, PPV
, Poisson Deviance
, R Squared
, RMSE
, RMSLE
, Rate@Top10%
, Rate@Top5%
, TNR
, TPR
, Tweedie Deviance
, WGS84 MAE
, WGS84 RMSE
]
AccuracyPerClass
{
"className" : "string" ,
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
className
string
true
Name of the class.
value
number,null
true
Value of the metric.
AccuracyResponse
{
"batchIds" : [
"string"
],
"data" : [
{
"baselineValue" : 0 ,
"metricName" : "string" ,
"modelId" : "string" ,
"percentChange" : 0 ,
"value" : 0
}
],
"metrics" : "\n {\"metrics\": {\n \"LogLoss\": {\n \"baselineValue\": 0.454221484838069,\n \"value\": 0.880778024500618,\n \"percentChange\": -93.91\n },\n \"AUC\": {\n \"baselineValue\": 0.8690358459556535,\n \"value\": 0.5294117647058824,\n \"percentChange\": -39.08\n },\n \"Kolmogorov-Smirnov\": {\n \"baselineValue\": 0.5753202944706626,\n \"value\": 0.4117647058823529,\n \"percentChange\": -28.43\n },\n \"Rate@Top10%\": {\n \"baselineValue\": 0.9603223806571606,\n \"value\": 1.0,\n \"percentChange\": 4.13\n },\n \"Gini Norm\": {\n \"baselineValue\": 0.7380716919113071,\n \"value\": 0.05882352941176472,\n \"percentChange\": -92.03\n }\n }\n " ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"segmentAttribute" : "string" ,
"segmentValue" : ""
}
Properties
Name
Type
Required
Restrictions
Description
batchIds
[string]
false
ID of the batches used to calculate accuracy metrics.
data
[AccuracyMetric ]
true
maxItems: 100
Accuracy metric data.
metrics
object
false
Accuracy metrics of the deployment. Deprecated and to be removed in v2.40; use data objects.
modelId
string
false
The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.40; use modelId in each data object.
period
TimeRange
false
An object with the keys "start" and "end" defining the period.
segmentAttribute
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
string,null
false
The value of the segmentAttribute
to segment on.
AccuracyTimeBaselineBucket
{
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"sampleSize" : 0 ,
"value" : 0
}
Properties
Name
Type
Required
Restrictions
Description
modelId
string
false
The id of the model for which metrics are being retrieved.
perClass
[AccuracyPerClass ]
false
Accuracy metric values for selected classes, only available for multiclass deployments.
sampleSize
integer
true
Number of rows used to calculate the metric.
value
number,null
true
Accuracy metric value.
AccuracyTimeBucket
{
"modelId" : "string" ,
"perClass" : [
{
"className" : "string" ,
"value" : 0
}
],
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"sampleSize" : 0 ,
"value" : 0 ,
"valuePerClass" : {}
}
Properties
Name
Type
Required
Restrictions
Description
modelId
string
false
The id of the model for which metrics are being retrieved.
perClass
[AccuracyPerClass ]
false
Accuracy metric values for selected classes, only available for multiclass deployments.
period
TimeRange
true
An object with the keys "start" and "end" defining the period.
sampleSize
integer
true
Number of rows used to calculate the metric.
value
number,null
true
Accuracy metric value.
valuePerClass
object,null
false
A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.
ActualClassDistribution
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
Properties
Name
Type
Required
Restrictions
Description
className
string
true
Name of the class.
count
integer
true
Count of actual rows labeled with a class in the bucket.
percent
number
true
Percent of actual rows labeled with a class in the bucket.
BatchPeriod
{
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
}
Time period of the batch.
Properties
Name
Type
Required
Restrictions
Description
end
string,null(date-time)
true
End time of the bucket
start
string,null(date-time)
true
Start time of the bucket
PredictedClassDistribution
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
Properties
Name
Type
Required
Restrictions
Description
className
string
true
Name of the class.
count
integer
true
Count of prediction rows labeled with a class in the bucket.
percent
number
true
Percent of prediction rows labeled with a class in the bucket.
PredictionsVsActualsBaseline
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
Properties
Name
Type
Required
Restrictions
Description
actualClassDistribution
[ActualClassDistribution ]
false
Class distribution for all actuals in the bucket, only for classification deployments.
meanActualValue
number,null
false
Mean actual value for all rows in the bucket, only for regression deployments.
meanPredictedValue
number,null
false
Mean predicted value for all rows in the bucket, only for regression deployments.
modelId
string
true
ID of the model.
predictedClassDistribution
[PredictedClassDistribution ]
false
Class distribution for all rows with actual in the bucket, only for classification deployments.
rowCountTotal
integer
true
Number of rows in the bucket.
rowCountWithActual
integer
true
Number of rows with actual in the bucket.
PredictionsVsActualsBatchBucket
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"batchId" : "string" ,
"batchName" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
Properties
Name
Type
Required
Restrictions
Description
actualClassDistribution
[ActualClassDistribution ]
false
Class distribution for all actuals in the bucket, only for classification deployments.
batchId
string
true
ID of the batch.
batchName
string
true
Name of the batch.
meanActualValue
number,null
false
Mean actual value for all rows in the bucket, only for regression deployments.
meanPredictedValue
number,null
false
Mean predicted value for all rows in the bucket, only for regression deployments.
modelId
string
true
ID of the model.
period
BatchPeriod
true
Time period of the batch.
predictedClassDistribution
[PredictedClassDistribution ]
false
Class distribution for all rows with actual in the bucket, only for classification deployments.
rowCountTotal
integer
true
Number of rows in the bucket.
rowCountWithActual
integer
true
Number of rows with actual in the bucket.
PredictionsVsActualsOverBatchResponse
{
"baselines" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"buckets" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"batchId" : "string" ,
"batchName" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"segmentAttribute" : "string" ,
"segmentValue" : ""
}
Properties
Name
Type
Required
Restrictions
Description
baselines
[PredictionsVsActualsBaseline ]
true
Predictions vs actuals baselines.
buckets
[PredictionsVsActualsBatchBucket ]
true
Predictions vs actuals buckets.
segmentAttribute
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
string,null
false
The value of the segmentAttribute
to segment on.
PredictionsVsActualsOverSpaceBucket
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"hexagon" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
Properties
Name
Type
Required
Restrictions
Description
actualClassDistribution
[ActualClassDistribution ]
false
maxItems: 100
For classification deployments, the class distribution for all actuals in the bucket.
hexagon
string
true
The H3 geospatial indexing hexagon.
meanActualValue
number,null
false
For regression deployments, the mean actual value of all rows in the bucket.
meanPredictedValue
number,null
false
For regression deployments, the mean predicted value of all rows in the bucket.
predictedClassDistribution
[PredictedClassDistribution ]
false
maxItems: 100
For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.
rowCountTotal
integer
true
The number of rows in the bucket.
rowCountWithActual
integer
true
The number of prediction rows in the bucket with associated actuals.
PredictionsVsActualsOverSpaceResponse
{
"baselines" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"hexagon" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"buckets" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"hexagon" : "string" ,
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"geoFeatureName" : "string" ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"summary" : {
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
}
Properties
Name
Type
Required
Restrictions
Description
baselines
[PredictionsVsActualsOverSpaceBucket ]
true
maxItems: 1000
The predictions vs. actuals over space baselines.
buckets
[PredictionsVsActualsOverSpaceBucket ]
true
maxItems: 1000
The predictions vs. actuals over space buckets.
geoFeatureName
string
true
The name of the geospatial feature.
modelId
string
false
The ID of the model for which metrics are being retrieved.
period
TimeRange
false
An object with the keys "start" and "end" defining the period.
summary
PredictionsVsActualsSummaryBucket
true
Predictions vs actuals summary.
PredictionsVsActualsOverTimeResponse
{
"baselines" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"buckets" : [
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
],
"segmentAttribute" : "string" ,
"segmentValue" : "" ,
"summary" : {
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
}
Properties
Name
Type
Required
Restrictions
Description
baselines
[PredictionsVsActualsBaseline ]
true
Predictions vs actuals baselines.
buckets
[PredictionsVsActualsTimeBucket ]
true
Predictions vs actuals buckets.
segmentAttribute
string,null
false
The name of the segment on which segment analysis is being performed.
segmentValue
string,null
false
The value of the segmentAttribute
to segment on.
summary
PredictionsVsActualsSummaryBucket
true
Predictions vs actuals summary.
PredictionsVsActualsSummaryBucket
{
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
Predictions vs actuals summary.
Properties
Name
Type
Required
Restrictions
Description
rowCountTotal
integer
true
Number of rows for all buckets.
rowCountWithActual
integer
true
Number of rows with actual for all buckets.
PredictionsVsActualsTimeBucket
{
"actualClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"meanActualValue" : 0 ,
"meanPredictedValue" : 0 ,
"modelId" : "string" ,
"period" : {
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
},
"predictedClassDistribution" : [
{
"className" : "string" ,
"count" : 0 ,
"percent" : 0
}
],
"rowCountTotal" : 0 ,
"rowCountWithActual" : 0
}
Properties
Name
Type
Required
Restrictions
Description
actualClassDistribution
[ActualClassDistribution ]
false
Class distribution for all actuals in the bucket, only for classification deployments.
meanActualValue
number,null
false
Mean actual value for all rows in the bucket, only for regression deployments.
meanPredictedValue
number,null
false
Mean predicted value for all rows in the bucket, only for regression deployments.
modelId
string
true
ID of the model.
period
BatchPeriod
true
Time period of the batch.
predictedClassDistribution
[PredictedClassDistribution ]
false
Class distribution for all rows with actual in the bucket, only for classification deployments.
rowCountTotal
integer
true
Number of rows in the bucket.
rowCountWithActual
integer
true
Number of rows with actual in the bucket.
TimeRange
{
"end" : "2019-08-24T14:15:22Z" ,
"start" : "2019-08-24T14:15:22Z"
}
An object with the keys "start" and "end" defining the period.
Properties
Name
Type
Required
Restrictions
Description
end
string,null(date-time)
false
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
start
string,null(date-time)
false
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z
.
Updated March 19, 2025
Submit
Thanks for your feedback!