Accuracy
Use the endpoints described below to manage accuracy. Accuracy allows you to analyze the performance of model deployments over time using standard statistical measures and exportable visualizations. Use this tool to determine whether a model's quality is decaying and if you should consider replacing it.
GET /api/v2/deployments/{deploymentId}/accuracy/
Retrieve accuracy metric for a certain time period.
Parameters
| Name |
In |
Type |
Required |
Description |
| start |
query |
string,null(date-time) |
false |
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| end |
query |
string,null(date-time) |
false |
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| modelId |
query |
any |
false |
The ID of the models for which metrics are being retrieved. |
| batchId |
query |
any |
false |
The id of the batch for which metrics are being retrieved. |
| segmentAttribute |
query |
string,null |
false |
The name of the segment on which segment analysis is being performed. |
| segmentValue |
query |
string,null |
false |
The value of the segmentAttribute to segment on. |
| targetClass |
query |
any |
false |
Target class to filter out results. |
| metric |
query |
string |
false |
Name of the metric to retrieve. Must be provided when using multiple modelId query params. |
| baselineModelId |
query |
string |
false |
ID of the model whose metric value to use as baseline. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Enumerated Values
| Parameter |
Value |
| metric |
[AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE] |
Example responses
200 Response
{
"properties": {
"batchIds": {
"description": "ID of the batches used to calculate accuracy metrics.",
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"description": "Accuracy metric data.",
"items": {
"properties": {
"baselineSampleSize": {
"description": "Number of predictions used to calculate the baseline metric value.",
"type": "integer",
"x-versionadded": "v2.37"
},
"baselineValue": {
"description": "Baseline value of the metric.",
"type": [
"number",
"null"
]
},
"metricName": {
"description": "Name of the metric.",
"type": "string"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"percentChange": {
"description": "Percent of change by comparing metric value to baseline, with metric direction taken into account.",
"type": [
"number",
"null"
]
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric value.",
"type": "integer",
"x-versionadded": "v2.37"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"baselineSampleSize",
"baselineValue",
"metricName",
"percentChange",
"sampleSize",
"value"
],
"type": "object",
"x-versionadded": "v2.36"
},
"maxItems": 100,
"type": "array",
"x-versionadded": "v2.36"
},
"metrics": {
"description": "Accuracy metrics of the deployment. Deprecated and to be removed in v2.40; use data objects.",
"example": "\n {\"metrics\": {\n \"LogLoss\": {\n \"baselineValue\": 0.454221484838069,\n \"value\": 0.880778024500618,\n \"percentChange\": -93.91\n },\n \"AUC\": {\n \"baselineValue\": 0.8690358459556535,\n \"value\": 0.5294117647058824,\n \"percentChange\": -39.08\n },\n \"Kolmogorov-Smirnov\": {\n \"baselineValue\": 0.5753202944706626,\n \"value\": 0.4117647058823529,\n \"percentChange\": -28.43\n },\n \"Rate@Top10%\": {\n \"baselineValue\": 0.9603223806571606,\n \"value\": 1.0,\n \"percentChange\": 4.13\n },\n \"Gini Norm\": {\n \"baselineValue\": 0.7380716919113071,\n \"value\": 0.05882352941176472,\n \"percentChange\": -92.03\n }\n }\n ",
"type": "object",
"x-versiondeprecated": "v2.36"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.40; use modelId in each data object.",
"type": "string",
"x-versiondeprecated": "v2.36"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
}
},
"required": [
"data"
],
"type": "object"
}
Responses
| Status |
Meaning |
Description |
Schema |
| 200 |
OK |
Deployment accuracy metrics are retrieved. |
AccuracyResponse |
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyMetrics/
Retrieve information about which accuracy metrics will be displayed and in what order.
Parameters
| Name |
In |
Type |
Required |
Description |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Example responses
200 Response
{
"properties": {
"data": {
"description": "List of Accuracy Metrics.",
"items": {
"description": "Name of the accuracy metric.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"maxItems": 15,
"type": "array"
}
},
"required": [
"data"
],
"type": "object",
"x-versionadded": "v2.36"
}
Responses
| Status |
Meaning |
Description |
Schema |
| 200 |
OK |
Information on which accuracy metrics will be displayed and in what order. |
AccuracyMetricList |
| 404 |
Not Found |
Deployment not found. |
None |
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
PUT /api/v2/deployments/{deploymentId}/accuracyMetrics/
Update accuracy metrics being returned in accuracy endpoint.
Body parameter
{
"properties": {
"data": {
"description": "List of Accuracy Metrics.",
"items": {
"description": "Name of the accuracy metric.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"maxItems": 15,
"type": "array"
}
},
"required": [
"data"
],
"type": "object",
"x-versionadded": "v2.36"
}
Parameters
| Name |
In |
Type |
Required |
Description |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
| body |
body |
AccuracyMetricUpdate |
false |
none |
Example responses
200 Response
{
"properties": {
"data": {
"description": "List of Accuracy Metrics.",
"items": {
"description": "Name of the accuracy metric.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"maxItems": 15,
"type": "array"
}
},
"required": [
"data"
],
"type": "object",
"x-versionadded": "v2.36"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyOverBatch/
Retrieve accuracy metric baseline, and metric value calculated for each batch.
Parameters
| Name |
In |
Type |
Required |
Description |
| segmentAttribute |
query |
string,null |
false |
The name of the segment on which segment analysis is being performed. |
| segmentValue |
query |
string,null |
false |
The value of the segmentAttribute to segment on. |
| modelId |
query |
string |
false |
The id of the model for which metrics are being retrieved. |
| batchId |
query |
any |
false |
The id of the batch for which metrics are being retrieved. |
| metric |
query |
string |
false |
Accuracy metric being requested. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Enumerated Values
| Parameter |
Value |
| metric |
[AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance] |
Example responses
200 Response
{
"properties": {
"baselines": {
"description": "Accuracy metric training baseline.",
"items": {
"properties": {
"metric": {
"description": "Name of the metric.",
"type": "string",
"x-versionadded": "v2.33"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string",
"x-versionadded": "v2.33"
},
"perClass": {
"description": "Accuracy metric for selected classes.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer",
"x-versionadded": "v2.33"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
}
},
"required": [
"metric",
"sampleSize",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"buckets": {
"description": "Accuracy metric for batches that are requested, with non-existing batches omitted.",
"items": {
"properties": {
"batchId": {
"description": "ID of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"batchName": {
"description": "Name of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"metric": {
"description": "Name of the metric.",
"type": "string",
"x-versionadded": "v2.33"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string",
"x-versionadded": "v2.33"
},
"perClass": {
"description": "Accuracy metric for selected classes.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer",
"x-versionadded": "v2.33"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
}
},
"required": [
"batchId",
"batchName",
"metric",
"period",
"sampleSize",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
}
},
"required": [
"baselines",
"buckets"
],
"type": "object"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyOverSpace/
Retrieve accuracy metric baseline, and metric value calculated for each geospatial h3 hexagon.
Parameters
| Name |
In |
Type |
Required |
Description |
| start |
query |
string,null(date-time) |
false |
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| end |
query |
string,null(date-time) |
false |
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| modelId |
query |
string |
false |
The ID of the model for which metrics are being retrieved. |
| metric |
query |
string |
false |
Name of the metric. |
| geoFeatureName |
query |
string |
false |
The name of the geospatial feature. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Enumerated Values
| Parameter |
Value |
| metric |
[AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAD, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE] |
Example responses
200 Response
{
"properties": {
"baselines": {
"description": "Baseline accuracy per geospatial hexagon.",
"items": {
"properties": {
"hexagon": {
"description": "h3 hexagon.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"hexagon",
"sampleSize",
"value"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"buckets": {
"description": "Accuracy per geospatial hexagon.",
"items": {
"properties": {
"hexagon": {
"description": "h3 hexagon.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"hexagon",
"value"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"geoFeatureName": {
"description": "The name of the geospatial feature. Segmented analysis must be enabled for the feature specified.",
"type": "string"
},
"metric": {
"description": "The metric being retrieved.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"modelId": {
"description": "The ID of the model for which metrics are being retrieved. ",
"type": "string"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
}
},
"required": [
"baselines",
"buckets",
"geoFeatureName",
"metric"
],
"type": "object",
"x-versionadded": "v2.35"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/accuracyOverTime/
Retrieve accuracy metric baseline, and metric value calculated for each time bucket dividing a longer time period.
Parameters
| Name |
In |
Type |
Required |
Description |
| start |
query |
string,null(date-time) |
false |
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| end |
query |
string,null(date-time) |
false |
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| bucketSize |
query |
string(duration) |
false |
The time duration of a bucket. Needs to be multiple of one hour. Can not be longer than the total length of the period. If not set, a default value will be calculated based on the start and end time. |
| modelId |
query |
any |
false |
The ID of the models for which metrics are being retrieved. |
| metric |
query |
string |
false |
Name of the metric. |
| segmentAttribute |
query |
string,null |
false |
The name of the segment on which segment analysis is being performed. |
| segmentValue |
query |
string,null |
false |
The value of the segmentAttribute to segment on. |
| targetClass |
query |
any |
false |
Target class to filter out results. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Enumerated Values
| Parameter |
Value |
| metric |
[AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAD, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE] |
Example responses
200 Response
{
"properties": {
"baseline": {
"description": "A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.",
"properties": {
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric.",
"type": [
"integer",
"null"
]
},
"value": {
"description": "Value of the metric, null if no value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
]
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
},
"baselines": {
"description": "Accuracy metric training baseline.",
"items": {
"properties": {
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"sampleSize",
"value"
],
"type": "object"
},
"type": "array"
},
"buckets": {
"description": "Accuracy metric for requested models and time buckets.",
"items": {
"properties": {
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
],
"x-versiondeprecated": "v2.33"
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
},
"type": "array"
},
"metric": {
"description": "The metric being retrieved.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.35, use modelId in each baseline or bucket object.",
"type": "string",
"x-versiondeprecated": "v2.33"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"summary": {
"description": "A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.",
"properties": {
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric.",
"type": [
"integer",
"null"
]
},
"value": {
"description": "Value of the metric, null if no value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
]
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
}
},
"required": [
"baseline",
"baselines",
"buckets",
"metric",
"summary"
],
"type": "object"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverBatch/
Retrieve metrics about predictions and actuals, such as mean predicted & actual value, predicted & actual class distribution, over a specific set of batches.
Parameters
| Name |
In |
Type |
Required |
Description |
| segmentAttribute |
query |
string,null |
false |
The name of the segment on which segment analysis is being performed. |
| segmentValue |
query |
string,null |
false |
The value of the segmentAttribute to segment on. |
| modelId |
query |
string |
false |
The id of the model for which metrics are being retrieved. |
| batchId |
query |
any |
false |
The id of the batch for which metrics are being retrieved. |
| targetClass |
query |
any |
false |
Target class to filter out results. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Example responses
200 Response
{
"properties": {
"baselines": {
"description": "Predictions vs actuals baselines.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"buckets": {
"description": "Predictions vs actuals buckets.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"batchId": {
"description": "ID of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"batchName": {
"description": "Name of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
},
"modelId": {
"description": "ID of the model.",
"type": "string",
"x-versionadded": "v2.33"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer",
"x-versionadded": "v2.33"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer",
"x-versionadded": "v2.33"
}
},
"required": [
"batchId",
"batchName",
"modelId",
"period",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
}
},
"required": [
"baselines",
"buckets"
],
"type": "object"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverSpace/
Retrieve predictions vs. actuals over space.
Parameters
| Name |
In |
Type |
Required |
Description |
| start |
query |
string,null(date-time) |
false |
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| end |
query |
string,null(date-time) |
false |
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| modelId |
query |
string |
false |
The ID of the model for which metrics are being retrieved. |
| geoFeatureName |
query |
string |
false |
The name of the geospatial feature. |
| targetClass |
query |
any |
false |
Target class to filter out results. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Example responses
200 Response
{
"properties": {
"baselines": {
"description": "The predictions vs. actuals over space baselines.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "For classification deployments, the class distribution for all actuals in the bucket.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"hexagon": {
"description": "The H3 geospatial indexing hexagon.",
"type": "string"
},
"meanActualValue": {
"description": "For regression deployments, the mean actual value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "For regression deployments, the mean predicted value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"predictedClassDistribution": {
"description": "For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"rowCountTotal": {
"description": "The number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "The number of prediction rows in the bucket with associated actuals.",
"type": "integer"
}
},
"required": [
"hexagon",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"buckets": {
"description": "The predictions vs. actuals over space buckets.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "For classification deployments, the class distribution for all actuals in the bucket.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"hexagon": {
"description": "The H3 geospatial indexing hexagon.",
"type": "string"
},
"meanActualValue": {
"description": "For regression deployments, the mean actual value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "For regression deployments, the mean predicted value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"predictedClassDistribution": {
"description": "For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"rowCountTotal": {
"description": "The number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "The number of prediction rows in the bucket with associated actuals.",
"type": "integer"
}
},
"required": [
"hexagon",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"geoFeatureName": {
"description": "The name of the geospatial feature.",
"type": "string"
},
"modelId": {
"description": "The ID of the model for which metrics are being retrieved. ",
"type": "string"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"summary": {
"description": "Predictions vs actuals summary.",
"properties": {
"rowCountTotal": {
"description": "Number of rows for all buckets.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual for all buckets.",
"type": "integer"
}
},
"required": [
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
},
"required": [
"baselines",
"buckets",
"geoFeatureName",
"summary"
],
"type": "object",
"x-versionadded": "v2.35"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
GET /api/v2/deployments/{deploymentId}/predictionsVsActualsOverTime/
Retrieve metrics about predictions and actuals, such as mean predicted & actual value, predicted & actual class distribution, over a specific time range.
Parameters
| Name |
In |
Type |
Required |
Description |
| start |
query |
string,null(date-time) |
false |
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| end |
query |
string,null(date-time) |
false |
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| bucketSize |
query |
string |
false |
Time duration of buckets |
| segmentAttribute |
query |
string,null |
false |
The name of the segment on which segment analysis is being performed. |
| segmentValue |
query |
string,null |
false |
The value of the segmentAttribute to segment on. |
| modelId |
query |
any |
false |
The ID of the models for which metrics are being retrieved. |
| targetClass |
query |
any |
false |
Target class to filter out results. |
| deploymentId |
path |
string |
true |
Unique identifier of the deployment. |
Enumerated Values
| Parameter |
Value |
| bucketSize |
[PT1H, P1D, P7D, P1M] |
Example responses
200 Response
{
"properties": {
"baselines": {
"description": "Predictions vs actuals baselines.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array"
},
"buckets": {
"description": "Predictions vs actuals buckets.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"period",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"summary": {
"description": "Predictions vs actuals summary.",
"properties": {
"rowCountTotal": {
"description": "Number of rows for all buckets.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual for all buckets.",
"type": "integer"
}
},
"required": [
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
},
"required": [
"baselines",
"buckets",
"summary"
],
"type": "object"
}
Responses
To perform this operation, you must be authenticated by means of one of the following methods:
BearerAuth
Schemas
AccuracyBatchBaselineBucket
{
"properties": {
"metric": {
"description": "Name of the metric.",
"type": "string",
"x-versionadded": "v2.33"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string",
"x-versionadded": "v2.33"
},
"perClass": {
"description": "Accuracy metric for selected classes.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer",
"x-versionadded": "v2.33"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
}
},
"required": [
"metric",
"sampleSize",
"value"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| metric |
string |
true |
|
Name of the metric. |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. |
| perClass |
[AccuracyPerClass] |
false |
|
Accuracy metric for selected classes. |
| sampleSize |
integer |
true |
|
Number of rows used to calculate the metric. |
| value |
number,null |
true |
|
Accuracy metric value. |
AccuracyBatchBucket
{
"properties": {
"batchId": {
"description": "ID of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"batchName": {
"description": "Name of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"metric": {
"description": "Name of the metric.",
"type": "string",
"x-versionadded": "v2.33"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string",
"x-versionadded": "v2.33"
},
"perClass": {
"description": "Accuracy metric for selected classes.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer",
"x-versionadded": "v2.33"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
}
},
"required": [
"batchId",
"batchName",
"metric",
"period",
"sampleSize",
"value"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| batchId |
string |
true |
|
ID of the batch. |
| batchName |
string |
true |
|
Name of the batch. |
| metric |
string |
true |
|
Name of the metric. |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. |
| perClass |
[AccuracyPerClass] |
false |
|
Accuracy metric for selected classes. |
| period |
BatchPeriod |
true |
|
Time period of the batch. |
| sampleSize |
integer |
true |
|
Number of rows used to calculate the metric. |
| value |
number,null |
true |
|
Accuracy metric value. |
AccuracyLegacyTimeBucket
{
"description": "A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.",
"properties": {
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric.",
"type": [
"integer",
"null"
]
},
"value": {
"description": "Value of the metric, null if no value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
]
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
}
A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| period |
TimeRange |
true |
|
An object with the keys "start" and "end" defining the period. |
| sampleSize |
integer,null |
true |
|
Number of predictions used to calculate the metric. |
| value |
number,null |
true |
|
Value of the metric, null if no value. |
| valuePerClass |
object,null |
false |
|
A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead. |
AccuracyMetric
{
"properties": {
"baselineSampleSize": {
"description": "Number of predictions used to calculate the baseline metric value.",
"type": "integer",
"x-versionadded": "v2.37"
},
"baselineValue": {
"description": "Baseline value of the metric.",
"type": [
"number",
"null"
]
},
"metricName": {
"description": "Name of the metric.",
"type": "string"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"percentChange": {
"description": "Percent of change by comparing metric value to baseline, with metric direction taken into account.",
"type": [
"number",
"null"
]
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric value.",
"type": "integer",
"x-versionadded": "v2.37"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"baselineSampleSize",
"baselineValue",
"metricName",
"percentChange",
"sampleSize",
"value"
],
"type": "object",
"x-versionadded": "v2.36"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baselineSampleSize |
integer |
true |
|
Number of predictions used to calculate the baseline metric value. |
| baselineValue |
number,null |
true |
|
Baseline value of the metric. |
| metricName |
string |
true |
|
Name of the metric. |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. |
| percentChange |
number,null |
true |
|
Percent of change by comparing metric value to baseline, with metric direction taken into account. |
| sampleSize |
integer |
true |
|
Number of predictions used to calculate the metric value. |
| value |
number,null |
true |
|
Value of the metric. |
AccuracyMetricList
{
"properties": {
"data": {
"description": "List of Accuracy Metrics.",
"items": {
"description": "Name of the accuracy metric.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"maxItems": 15,
"type": "array"
}
},
"required": [
"data"
],
"type": "object",
"x-versionadded": "v2.36"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| data |
[string] |
true |
maxItems: 15
|
List of Accuracy Metrics. |
AccuracyMetricUpdate
{
"properties": {
"data": {
"description": "List of Accuracy Metrics.",
"items": {
"description": "Name of the accuracy metric.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"maxItems": 15,
"type": "array"
}
},
"required": [
"data"
],
"type": "object",
"x-versionadded": "v2.36"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| data |
[string] |
true |
maxItems: 15
|
List of Accuracy Metrics. |
AccuracyOverBatchResponse
{
"properties": {
"baselines": {
"description": "Accuracy metric training baseline.",
"items": {
"properties": {
"metric": {
"description": "Name of the metric.",
"type": "string",
"x-versionadded": "v2.33"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string",
"x-versionadded": "v2.33"
},
"perClass": {
"description": "Accuracy metric for selected classes.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer",
"x-versionadded": "v2.33"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
}
},
"required": [
"metric",
"sampleSize",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"buckets": {
"description": "Accuracy metric for batches that are requested, with non-existing batches omitted.",
"items": {
"properties": {
"batchId": {
"description": "ID of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"batchName": {
"description": "Name of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"metric": {
"description": "Name of the metric.",
"type": "string",
"x-versionadded": "v2.33"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string",
"x-versionadded": "v2.33"
},
"perClass": {
"description": "Accuracy metric for selected classes.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer",
"x-versionadded": "v2.33"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
}
},
"required": [
"batchId",
"batchName",
"metric",
"period",
"sampleSize",
"value"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
}
},
"required": [
"baselines",
"buckets"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baselines |
[AccuracyBatchBaselineBucket] |
true |
|
Accuracy metric training baseline. |
| buckets |
[AccuracyBatchBucket] |
true |
|
Accuracy metric for batches that are requested, with non-existing batches omitted. |
AccuracyOverSpaceBaselineBucket
{
"properties": {
"hexagon": {
"description": "h3 hexagon.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"hexagon",
"sampleSize",
"value"
],
"type": "object",
"x-versionadded": "v2.35"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| hexagon |
string |
true |
|
h3 hexagon. |
| perClass |
[AccuracyPerClass] |
false |
maxItems: 125
|
Accuracy metric values for selected classes, only available for multiclass deployments. |
| sampleSize |
integer |
true |
|
Number of rows used to calculate the metric. |
| value |
number,null |
true |
|
Accuracy metric value. |
AccuracyOverSpaceBucket
{
"properties": {
"hexagon": {
"description": "h3 hexagon.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"hexagon",
"value"
],
"type": "object",
"x-versionadded": "v2.35"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| hexagon |
string |
true |
|
h3 hexagon. |
| perClass |
[AccuracyPerClass] |
false |
maxItems: 125
|
Accuracy metric values for selected classes, only available for multiclass deployments. |
| value |
number,null |
true |
|
Accuracy metric value. |
AccuracyOverSpaceResponse
{
"properties": {
"baselines": {
"description": "Baseline accuracy per geospatial hexagon.",
"items": {
"properties": {
"hexagon": {
"description": "h3 hexagon.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"hexagon",
"sampleSize",
"value"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"buckets": {
"description": "Accuracy per geospatial hexagon.",
"items": {
"properties": {
"hexagon": {
"description": "h3 hexagon.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"hexagon",
"value"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"geoFeatureName": {
"description": "The name of the geospatial feature. Segmented analysis must be enabled for the feature specified.",
"type": "string"
},
"metric": {
"description": "The metric being retrieved.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"modelId": {
"description": "The ID of the model for which metrics are being retrieved. ",
"type": "string"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
}
},
"required": [
"baselines",
"buckets",
"geoFeatureName",
"metric"
],
"type": "object",
"x-versionadded": "v2.35"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baselines |
[AccuracyOverSpaceBaselineBucket] |
true |
maxItems: 1000
|
Baseline accuracy per geospatial hexagon. |
| buckets |
[AccuracyOverSpaceBucket] |
true |
maxItems: 1000
|
Accuracy per geospatial hexagon. |
| geoFeatureName |
string |
true |
|
The name of the geospatial feature. Segmented analysis must be enabled for the feature specified. |
| metric |
string |
true |
|
The metric being retrieved. |
| modelId |
string |
false |
|
The ID of the model for which metrics are being retrieved. |
| period |
TimeRange |
false |
|
An object with the keys "start" and "end" defining the period. |
Enumerated Values
| Property |
Value |
| metric |
[AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE] |
AccuracyOverTimeResponse
{
"properties": {
"baseline": {
"description": "A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.",
"properties": {
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric.",
"type": [
"integer",
"null"
]
},
"value": {
"description": "Value of the metric, null if no value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
]
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
},
"baselines": {
"description": "Accuracy metric training baseline.",
"items": {
"properties": {
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"sampleSize",
"value"
],
"type": "object"
},
"type": "array"
},
"buckets": {
"description": "Accuracy metric for requested models and time buckets.",
"items": {
"properties": {
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
],
"x-versiondeprecated": "v2.33"
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
},
"type": "array"
},
"metric": {
"description": "The metric being retrieved.",
"enum": [
"AUC",
"Accuracy",
"Balanced Accuracy",
"F1",
"FPR",
"FVE Binomial",
"FVE Gamma",
"FVE Multinomial",
"FVE Poisson",
"FVE Tweedie",
"Gamma Deviance",
"Gini Norm",
"Kolmogorov-Smirnov",
"LogLoss",
"MAE",
"MAPE",
"MCC",
"NPV",
"PPV",
"Poisson Deviance",
"R Squared",
"RMSE",
"RMSLE",
"Rate@Top10%",
"Rate@Top5%",
"TNR",
"TPR",
"Tweedie Deviance",
"WGS84 MAE",
"WGS84 RMSE"
],
"type": "string"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.35, use modelId in each baseline or bucket object.",
"type": "string",
"x-versiondeprecated": "v2.33"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"summary": {
"description": "A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range.",
"properties": {
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric.",
"type": [
"integer",
"null"
]
},
"value": {
"description": "Value of the metric, null if no value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
]
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
}
},
"required": [
"baseline",
"baselines",
"buckets",
"metric",
"summary"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baseline |
AccuracyLegacyTimeBucket |
true |
|
A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range. |
| baselines |
[AccuracyTimeBaselineBucket] |
true |
|
Accuracy metric training baseline. |
| buckets |
[AccuracyTimeBucket] |
true |
|
Accuracy metric for requested models and time buckets. |
| metric |
string |
true |
|
The metric being retrieved. |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.35, use modelId in each baseline or bucket object. |
| segmentAttribute |
string,null |
false |
|
The name of the segment on which segment analysis is being performed. |
| segmentValue |
string,null |
false |
|
The value of the segmentAttribute to segment on. |
| summary |
AccuracyLegacyTimeBucket |
true |
|
A bucket object containing metric info calculated. Deprecated and to be removed in v2.35, use summaries field with includeSummaries query param to get accuracy metrics computed over the whole time range. |
Enumerated Values
| Property |
Value |
| metric |
[AUC, Accuracy, Balanced Accuracy, F1, FPR, FVE Binomial, FVE Gamma, FVE Multinomial, FVE Poisson, FVE Tweedie, Gamma Deviance, Gini Norm, Kolmogorov-Smirnov, LogLoss, MAE, MAPE, MCC, NPV, PPV, Poisson Deviance, R Squared, RMSE, RMSLE, Rate@Top10%, Rate@Top5%, TNR, TPR, Tweedie Deviance, WGS84 MAE, WGS84 RMSE] |
AccuracyPerClass
{
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| className |
string |
true |
|
Name of the class. |
| value |
number,null |
true |
|
Value of the metric. |
AccuracyResponse
{
"properties": {
"batchIds": {
"description": "ID of the batches used to calculate accuracy metrics.",
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"description": "Accuracy metric data.",
"items": {
"properties": {
"baselineSampleSize": {
"description": "Number of predictions used to calculate the baseline metric value.",
"type": "integer",
"x-versionadded": "v2.37"
},
"baselineValue": {
"description": "Baseline value of the metric.",
"type": [
"number",
"null"
]
},
"metricName": {
"description": "Name of the metric.",
"type": "string"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"percentChange": {
"description": "Percent of change by comparing metric value to baseline, with metric direction taken into account.",
"type": [
"number",
"null"
]
},
"sampleSize": {
"description": "Number of predictions used to calculate the metric value.",
"type": "integer",
"x-versionadded": "v2.37"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"baselineSampleSize",
"baselineValue",
"metricName",
"percentChange",
"sampleSize",
"value"
],
"type": "object",
"x-versionadded": "v2.36"
},
"maxItems": 100,
"type": "array",
"x-versionadded": "v2.36"
},
"metrics": {
"description": "Accuracy metrics of the deployment. Deprecated and to be removed in v2.40; use data objects.",
"example": "\n {\"metrics\": {\n \"LogLoss\": {\n \"baselineValue\": 0.454221484838069,\n \"value\": 0.880778024500618,\n \"percentChange\": -93.91\n },\n \"AUC\": {\n \"baselineValue\": 0.8690358459556535,\n \"value\": 0.5294117647058824,\n \"percentChange\": -39.08\n },\n \"Kolmogorov-Smirnov\": {\n \"baselineValue\": 0.5753202944706626,\n \"value\": 0.4117647058823529,\n \"percentChange\": -28.43\n },\n \"Rate@Top10%\": {\n \"baselineValue\": 0.9603223806571606,\n \"value\": 1.0,\n \"percentChange\": 4.13\n },\n \"Gini Norm\": {\n \"baselineValue\": 0.7380716919113071,\n \"value\": 0.05882352941176472,\n \"percentChange\": -92.03\n }\n }\n ",
"type": "object",
"x-versiondeprecated": "v2.36"
},
"modelId": {
"description": "The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.40; use modelId in each data object.",
"type": "string",
"x-versiondeprecated": "v2.36"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
}
},
"required": [
"data"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| batchIds |
[string] |
false |
|
ID of the batches used to calculate accuracy metrics. |
| data |
[AccuracyMetric] |
true |
maxItems: 100
|
Accuracy metric data. |
| metrics |
object |
false |
|
Accuracy metrics of the deployment. Deprecated and to be removed in v2.40; use data objects. |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. Deprecated and to be removed in v2.40; use modelId in each data object. |
| period |
TimeRange |
false |
|
An object with the keys "start" and "end" defining the period. |
| segmentAttribute |
string,null |
false |
|
The name of the segment on which segment analysis is being performed. |
| segmentValue |
string,null |
false |
|
The value of the segmentAttribute to segment on. |
AccuracyTimeBaselineBucket
{
"properties": {
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
}
},
"required": [
"sampleSize",
"value"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. |
| perClass |
[AccuracyPerClass] |
false |
|
Accuracy metric values for selected classes, only available for multiclass deployments. |
| sampleSize |
integer |
true |
|
Number of rows used to calculate the metric. |
| value |
number,null |
true |
|
Accuracy metric value. |
AccuracyTimeBucket
{
"properties": {
"modelId": {
"description": "The id of the model for which metrics are being retrieved.",
"type": "string"
},
"perClass": {
"description": "Accuracy metric values for selected classes, only available for multiclass deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"value": {
"description": "Value of the metric.",
"type": [
"number",
"null"
]
}
},
"required": [
"className",
"value"
],
"type": "object"
},
"type": "array"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"sampleSize": {
"description": "Number of rows used to calculate the metric.",
"type": "integer"
},
"value": {
"description": "Accuracy metric value.",
"type": [
"number",
"null"
]
},
"valuePerClass": {
"description": "A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead.",
"type": [
"object",
"null"
],
"x-versiondeprecated": "v2.33"
}
},
"required": [
"period",
"sampleSize",
"value"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| modelId |
string |
false |
|
The id of the model for which metrics are being retrieved. |
| perClass |
[AccuracyPerClass] |
false |
|
Accuracy metric values for selected classes, only available for multiclass deployments. |
| period |
TimeRange |
true |
|
An object with the keys "start" and "end" defining the period. |
| sampleSize |
integer |
true |
|
Number of rows used to calculate the metric. |
| value |
number,null |
true |
|
Accuracy metric value. |
| valuePerClass |
object,null |
false |
|
A dict keyed by class names with metric calculated for specific classes as values, if targetClass is set. Deprecated and to be removed in v2.35, use perClass instead. |
ActualClassDistribution
{
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| className |
string |
true |
|
Name of the class. |
| count |
integer |
true |
|
Count of actual rows labeled with a class in the bucket. |
| percent |
number |
true |
|
Percent of actual rows labeled with a class in the bucket. |
BatchPeriod
{
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
}
Time period of the batch.
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| end |
string,null(date-time) |
true |
|
End time of the bucket |
| start |
string,null(date-time) |
true |
|
Start time of the bucket |
PredictedClassDistribution
{
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| className |
string |
true |
|
Name of the class. |
| count |
integer |
true |
|
Count of prediction rows labeled with a class in the bucket. |
| percent |
number |
true |
|
Percent of prediction rows labeled with a class in the bucket. |
PredictionsVsActualsBaseline
{
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| actualClassDistribution |
[ActualClassDistribution] |
false |
|
Class distribution for all actuals in the bucket, only for classification deployments. |
| meanActualValue |
number,null |
false |
|
Mean actual value for all rows in the bucket, only for regression deployments. |
| meanPredictedValue |
number,null |
false |
|
Mean predicted value for all rows in the bucket, only for regression deployments. |
| modelId |
string |
true |
|
ID of the model. |
| predictedClassDistribution |
[PredictedClassDistribution] |
false |
|
Class distribution for all rows with actual in the bucket, only for classification deployments. |
| rowCountTotal |
integer |
true |
|
Number of rows in the bucket. |
| rowCountWithActual |
integer |
true |
|
Number of rows with actual in the bucket. |
PredictionsVsActualsBatchBucket
{
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"batchId": {
"description": "ID of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"batchName": {
"description": "Name of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
},
"modelId": {
"description": "ID of the model.",
"type": "string",
"x-versionadded": "v2.33"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer",
"x-versionadded": "v2.33"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer",
"x-versionadded": "v2.33"
}
},
"required": [
"batchId",
"batchName",
"modelId",
"period",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| actualClassDistribution |
[ActualClassDistribution] |
false |
|
Class distribution for all actuals in the bucket, only for classification deployments. |
| batchId |
string |
true |
|
ID of the batch. |
| batchName |
string |
true |
|
Name of the batch. |
| meanActualValue |
number,null |
false |
|
Mean actual value for all rows in the bucket, only for regression deployments. |
| meanPredictedValue |
number,null |
false |
|
Mean predicted value for all rows in the bucket, only for regression deployments. |
| modelId |
string |
true |
|
ID of the model. |
| period |
BatchPeriod |
true |
|
Time period of the batch. |
| predictedClassDistribution |
[PredictedClassDistribution] |
false |
|
Class distribution for all rows with actual in the bucket, only for classification deployments. |
| rowCountTotal |
integer |
true |
|
Number of rows in the bucket. |
| rowCountWithActual |
integer |
true |
|
Number of rows with actual in the bucket. |
PredictionsVsActualsOverBatchResponse
{
"properties": {
"baselines": {
"description": "Predictions vs actuals baselines.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"buckets": {
"description": "Predictions vs actuals buckets.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"batchId": {
"description": "ID of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"batchName": {
"description": "Name of the batch.",
"type": "string",
"x-versionadded": "v2.33"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
],
"x-versionadded": "v2.33"
},
"modelId": {
"description": "ID of the model.",
"type": "string",
"x-versionadded": "v2.33"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer",
"x-versionadded": "v2.33"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer",
"x-versionadded": "v2.33"
}
},
"required": [
"batchId",
"batchName",
"modelId",
"period",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array",
"x-versionadded": "v2.33"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
}
},
"required": [
"baselines",
"buckets"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baselines |
[PredictionsVsActualsBaseline] |
true |
|
Predictions vs actuals baselines. |
| buckets |
[PredictionsVsActualsBatchBucket] |
true |
|
Predictions vs actuals buckets. |
| segmentAttribute |
string,null |
false |
|
The name of the segment on which segment analysis is being performed. |
| segmentValue |
string,null |
false |
|
The value of the segmentAttribute to segment on. |
PredictionsVsActualsOverSpaceBucket
{
"properties": {
"actualClassDistribution": {
"description": "For classification deployments, the class distribution for all actuals in the bucket.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"hexagon": {
"description": "The H3 geospatial indexing hexagon.",
"type": "string"
},
"meanActualValue": {
"description": "For regression deployments, the mean actual value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "For regression deployments, the mean predicted value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"predictedClassDistribution": {
"description": "For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"rowCountTotal": {
"description": "The number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "The number of prediction rows in the bucket with associated actuals.",
"type": "integer"
}
},
"required": [
"hexagon",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object",
"x-versionadded": "v2.35"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| actualClassDistribution |
[ActualClassDistribution] |
false |
maxItems: 125
|
For classification deployments, the class distribution for all actuals in the bucket. |
| hexagon |
string |
true |
|
The H3 geospatial indexing hexagon. |
| meanActualValue |
number,null |
false |
|
For regression deployments, the mean actual value of all rows in the bucket. |
| meanPredictedValue |
number,null |
false |
|
For regression deployments, the mean predicted value of all rows in the bucket. |
| predictedClassDistribution |
[PredictedClassDistribution] |
false |
maxItems: 125
|
For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals. |
| rowCountTotal |
integer |
true |
|
The number of rows in the bucket. |
| rowCountWithActual |
integer |
true |
|
The number of prediction rows in the bucket with associated actuals. |
PredictionsVsActualsOverSpaceResponse
{
"properties": {
"baselines": {
"description": "The predictions vs. actuals over space baselines.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "For classification deployments, the class distribution for all actuals in the bucket.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"hexagon": {
"description": "The H3 geospatial indexing hexagon.",
"type": "string"
},
"meanActualValue": {
"description": "For regression deployments, the mean actual value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "For regression deployments, the mean predicted value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"predictedClassDistribution": {
"description": "For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"rowCountTotal": {
"description": "The number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "The number of prediction rows in the bucket with associated actuals.",
"type": "integer"
}
},
"required": [
"hexagon",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"buckets": {
"description": "The predictions vs. actuals over space buckets.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "For classification deployments, the class distribution for all actuals in the bucket.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"hexagon": {
"description": "The H3 geospatial indexing hexagon.",
"type": "string"
},
"meanActualValue": {
"description": "For regression deployments, the mean actual value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "For regression deployments, the mean predicted value of all rows in the bucket.",
"type": [
"number",
"null"
]
},
"predictedClassDistribution": {
"description": "For classification deployments, the class distribution for all prediction rows in the bucket with associated actuals.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"maxItems": 125,
"type": "array"
},
"rowCountTotal": {
"description": "The number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "The number of prediction rows in the bucket with associated actuals.",
"type": "integer"
}
},
"required": [
"hexagon",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object",
"x-versionadded": "v2.35"
},
"maxItems": 1000,
"type": "array"
},
"geoFeatureName": {
"description": "The name of the geospatial feature.",
"type": "string"
},
"modelId": {
"description": "The ID of the model for which metrics are being retrieved. ",
"type": "string"
},
"period": {
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"summary": {
"description": "Predictions vs actuals summary.",
"properties": {
"rowCountTotal": {
"description": "Number of rows for all buckets.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual for all buckets.",
"type": "integer"
}
},
"required": [
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
},
"required": [
"baselines",
"buckets",
"geoFeatureName",
"summary"
],
"type": "object",
"x-versionadded": "v2.35"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baselines |
[PredictionsVsActualsOverSpaceBucket] |
true |
maxItems: 1000
|
The predictions vs. actuals over space baselines. |
| buckets |
[PredictionsVsActualsOverSpaceBucket] |
true |
maxItems: 1000
|
The predictions vs. actuals over space buckets. |
| geoFeatureName |
string |
true |
|
The name of the geospatial feature. |
| modelId |
string |
false |
|
The ID of the model for which metrics are being retrieved. |
| period |
TimeRange |
false |
|
An object with the keys "start" and "end" defining the period. |
| summary |
PredictionsVsActualsSummaryBucket |
true |
|
Predictions vs actuals summary. |
PredictionsVsActualsOverTimeResponse
{
"properties": {
"baselines": {
"description": "Predictions vs actuals baselines.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array"
},
"buckets": {
"description": "Predictions vs actuals buckets.",
"items": {
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"period",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
},
"type": "array"
},
"segmentAttribute": {
"description": "The name of the segment on which segment analysis is being performed.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"segmentValue": {
"default": "",
"description": "The value of the `segmentAttribute` to segment on.",
"type": [
"string",
"null"
],
"x-versionadded": "v2.20"
},
"summary": {
"description": "Predictions vs actuals summary.",
"properties": {
"rowCountTotal": {
"description": "Number of rows for all buckets.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual for all buckets.",
"type": "integer"
}
},
"required": [
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
},
"required": [
"baselines",
"buckets",
"summary"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| baselines |
[PredictionsVsActualsBaseline] |
true |
|
Predictions vs actuals baselines. |
| buckets |
[PredictionsVsActualsTimeBucket] |
true |
|
Predictions vs actuals buckets. |
| segmentAttribute |
string,null |
false |
|
The name of the segment on which segment analysis is being performed. |
| segmentValue |
string,null |
false |
|
The value of the segmentAttribute to segment on. |
| summary |
PredictionsVsActualsSummaryBucket |
true |
|
Predictions vs actuals summary. |
PredictionsVsActualsSummaryBucket
{
"description": "Predictions vs actuals summary.",
"properties": {
"rowCountTotal": {
"description": "Number of rows for all buckets.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual for all buckets.",
"type": "integer"
}
},
"required": [
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
Predictions vs actuals summary.
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| rowCountTotal |
integer |
true |
|
Number of rows for all buckets. |
| rowCountWithActual |
integer |
true |
|
Number of rows with actual for all buckets. |
PredictionsVsActualsTimeBucket
{
"properties": {
"actualClassDistribution": {
"description": "Class distribution for all actuals in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of actual rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of actual rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"meanActualValue": {
"description": "Mean actual value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"meanPredictedValue": {
"description": "Mean predicted value for all rows in the bucket, only for regression deployments.",
"type": [
"number",
"null"
]
},
"modelId": {
"description": "ID of the model.",
"type": "string"
},
"period": {
"description": "Time period of the batch.",
"properties": {
"end": {
"description": "End time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start time of the bucket",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"required": [
"end",
"start"
],
"type": "object"
},
"predictedClassDistribution": {
"description": "Class distribution for all rows with actual in the bucket, only for classification deployments.",
"items": {
"properties": {
"className": {
"description": "Name of the class.",
"type": "string"
},
"count": {
"description": "Count of prediction rows labeled with a class in the bucket.",
"type": "integer"
},
"percent": {
"description": "Percent of prediction rows labeled with a class in the bucket.",
"type": "number"
}
},
"required": [
"className",
"count",
"percent"
],
"type": "object"
},
"type": "array"
},
"rowCountTotal": {
"description": "Number of rows in the bucket.",
"type": "integer"
},
"rowCountWithActual": {
"description": "Number of rows with actual in the bucket.",
"type": "integer"
}
},
"required": [
"modelId",
"period",
"rowCountTotal",
"rowCountWithActual"
],
"type": "object"
}
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| actualClassDistribution |
[ActualClassDistribution] |
false |
|
Class distribution for all actuals in the bucket, only for classification deployments. |
| meanActualValue |
number,null |
false |
|
Mean actual value for all rows in the bucket, only for regression deployments. |
| meanPredictedValue |
number,null |
false |
|
Mean predicted value for all rows in the bucket, only for regression deployments. |
| modelId |
string |
true |
|
ID of the model. |
| period |
BatchPeriod |
true |
|
Time period of the batch. |
| predictedClassDistribution |
[PredictedClassDistribution] |
false |
|
Class distribution for all rows with actual in the bucket, only for classification deployments. |
| rowCountTotal |
integer |
true |
|
Number of rows in the bucket. |
| rowCountWithActual |
integer |
true |
|
Number of rows with actual in the bucket. |
TimeRange
{
"description": "An object with the keys \"start\" and \"end\" defining the period.",
"properties": {
"end": {
"description": "End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
},
"start": {
"description": "Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: ``2019-08-01T00:00:00Z``.",
"format": "date-time",
"type": [
"string",
"null"
]
}
},
"type": "object"
}
An object with the keys "start" and "end" defining the period.
Properties
| Name |
Type |
Required |
Restrictions |
Description |
| end |
string,null(date-time) |
false |
|
End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |
| start |
string,null(date-time) |
false |
|
Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z. |