Skip to content
For Managed AI Platform users, see the cloud platform documentation
DataRobot APIのリソース > APIリファレンスドキュメント > DataRobot REST API > Monitoring > Bias and fairness

Observability Bias Fairness

This page outlines the operations, endpoints, parameters, and example requests and responses for the Observability Bias Fairness.

GET /api/v2/deployments/{deploymentId}/fairnessScoresOverTime/

Retrieve fairness scores for all features, or for different classes of a single feature changes over a specific time period.

Code samples

curl -X GET https://app.datarobot.com/api/v2/deployments/{deploymentId}/fairnessScoresOverTime/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
start query string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
end query string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
bucketSize query string false Time duration of buckets
modelId query string false The ID of the model to retrieve the statistics for.
fairnessMetric query string,null false The name of the fairness metric to compute fairness over time.
protectedFeature query string false Name of the protected feature. When present, will return per class fairness score of the protected feature, otherwise returning fairness score of all protected features.
orderBy query string false The type of FairnessScore to order the response by.
includePrivilegedClass query string false Always include privileged class in response.
onlyStatisticallySignificant query string false Include only statistically significant attributes.
limit query integer false Max number of features in response.
offset query integer false Number of features to offset in response.
deploymentId path string true Unique identifier of the deployment.

Enumerated Values

Parameter Value
bucketSize [PT1H, P1D, P7D, P1M]
fairnessMetric [proportionalParity, equalParity, favorableClassBalance, unfavorableClassBalance, trueUnfavorableRateParity, trueFavorableRateParity, favorablePredictiveValueParity, unfavorablePredictiveValueParity]
orderBy [absoluteFairnessScore, relativeFairnessScore, -absoluteFairnessScore, -relativeFairnessScore]
includePrivilegedClass [false, False, true, True]
onlyStatisticallySignificant [false, False, true, True]

Example responses

200 Response

{
  "buckets": [
    {
      "metricName": "proportionalParity",
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "scores": [
        {
          "absoluteValue": 0,
          "classesCount": 0,
          "healthyClassesCount": 0,
          "healthyCount": 0,
          "isStatisticallySignificant": true,
          "label": "string",
          "message": "string",
          "privilegedClass": "string",
          "sampleSize": 0,
          "totalCount": 1,
          "value": 1
        }
      ]
    }
  ],
  "fairnessThreshold": 1,
  "favorableTargetOutcome": true,
  "modelId": "string",
  "modelPackageId": "string",
  "protectedFeature": "string",
  "summary": {
    "metricName": "proportionalParity",
    "period": {
      "end": "2019-08-24T14:15:22Z",
      "start": "2019-08-24T14:15:22Z"
    },
    "scores": [
      {
        "absoluteValue": 0,
        "classesCount": 0,
        "healthyClassesCount": 0,
        "healthyCount": 0,
        "isStatisticallySignificant": true,
        "label": "string",
        "message": "string",
        "privilegedClass": "string",
        "sampleSize": 0,
        "totalCount": 1,
        "value": 1
      }
    ]
  }
}

Responses

Status Meaning Description Schema
200 OK Fairness summary and over time. FairnessOverTimeResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas

FairnessOverTimeBucket

{
  "metricName": "proportionalParity",
  "period": {
    "end": "2019-08-24T14:15:22Z",
    "start": "2019-08-24T14:15:22Z"
  },
  "scores": [
    {
      "absoluteValue": 0,
      "classesCount": 0,
      "healthyClassesCount": 0,
      "healthyCount": 0,
      "isStatisticallySignificant": true,
      "label": "string",
      "message": "string",
      "privilegedClass": "string",
      "sampleSize": 0,
      "totalCount": 1,
      "value": 1
    }
  ]
}

Summary of per feature fairness.

Properties

Name Type Required Restrictions Description
metricName string true Name of the metric.
period TimeRange true An object with the keys "start" and "end" defining the period.
scores [FairnessOverTimeScore] true List of per-feature fairness details.

Enumerated Values

Property Value
metricName [proportionalParity, equalParity, favorableClassBalance, unfavorableClassBalance, trueUnfavorableRateParity, trueFavorableRateParity, favorablePredictiveValueParity, unfavorablePredictiveValueParity]

FairnessOverTimeResponse

{
  "buckets": [
    {
      "metricName": "proportionalParity",
      "period": {
        "end": "2019-08-24T14:15:22Z",
        "start": "2019-08-24T14:15:22Z"
      },
      "scores": [
        {
          "absoluteValue": 0,
          "classesCount": 0,
          "healthyClassesCount": 0,
          "healthyCount": 0,
          "isStatisticallySignificant": true,
          "label": "string",
          "message": "string",
          "privilegedClass": "string",
          "sampleSize": 0,
          "totalCount": 1,
          "value": 1
        }
      ]
    }
  ],
  "fairnessThreshold": 1,
  "favorableTargetOutcome": true,
  "modelId": "string",
  "modelPackageId": "string",
  "protectedFeature": "string",
  "summary": {
    "metricName": "proportionalParity",
    "period": {
      "end": "2019-08-24T14:15:22Z",
      "start": "2019-08-24T14:15:22Z"
    },
    "scores": [
      {
        "absoluteValue": 0,
        "classesCount": 0,
        "healthyClassesCount": 0,
        "healthyCount": 0,
        "isStatisticallySignificant": true,
        "label": "string",
        "message": "string",
        "privilegedClass": "string",
        "sampleSize": 0,
        "totalCount": 1,
        "value": 1
      }
    ]
  }
}

Properties

Name Type Required Restrictions Description
buckets [FairnessOverTimeBucket] true Per bucket summary.
fairnessThreshold number true maximum: 1
minimum: 0
 Threshold used to compute fairness results.
favorableTargetOutcome any true Preferable class of target.

oneOf

Name Type Required Restrictions Description
» anonymous boolean false none

xor

Name Type Required Restrictions Description
» anonymous string false none

xor

Name Type Required Restrictions Description
» anonymous integer false none

continued

Name Type Required Restrictions Description
modelId string true Model Id for which fairness is computed.
modelPackageId string true Model package Id.
protectedFeature string false Name of the protected feature.
summary FairnessOverTimeBucket true Summary of per feature fairness.

FairnessOverTimeScore

{
  "absoluteValue": 0,
  "classesCount": 0,
  "healthyClassesCount": 0,
  "healthyCount": 0,
  "isStatisticallySignificant": true,
  "label": "string",
  "message": "string",
  "privilegedClass": "string",
  "sampleSize": 0,
  "totalCount": 1,
  "value": 1
}

Properties

Name Type Required Restrictions Description
absoluteValue number false Absolute fairness score.
classesCount integer false Count of occurences of the class within the data the fairness is calculated on.
healthyClassesCount integer false Count of statistically significant classes above fairness threshold.
healthyCount integer false minimum: 0
 Total number of classes with fairness score above the threshold.
isStatisticallySignificant boolean false Class is statistically significant.
label string false Name of the feature.
message string false Explanation message.
privilegedClass string false Name of the privileged class (the one with the highest fairness score) within the feature.
sampleSize integer false Sample size used for fairness status calculation
totalCount integer false minimum: 1
 Total number of classes.
value number false maximum: 1
minimum: 0
 Fairness score in relation to the privileged class fairness score.

TimeRange

{
  "end": "2019-08-24T14:15:22Z",
  "start": "2019-08-24T14:15:22Z"
}

An object with the keys "start" and "end" defining the period.

Properties

Name Type Required Restrictions Description
end string,null(date-time) false End of the period to retrieve monitoring data, defaults to the next top of the hour. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.
start string,null(date-time) false Start of the period to retrieve monitoring data, defaults to 7 days ago from the end of the period. Note: this field only accepts top of the hour RFC3339 datetime strings, for example: 2019-08-01T00:00:00Z.