Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

DataRobot API resources > API reference documentation > DataRobot REST API > Comparison Prompts (GenAI)

Comparison Prompts (GenAI)

This page outlines the operations, endpoints, parameters, and example requests and responses for the Comparison Prompts (GenAI).

GET /api/v2/genai/comparisonPrompts/

List the comparison prompts associated with a comparison chat or a set of LLM blueprints.

Code samples

# You can also use wget
curl -X GET https://app.datarobot.com/api/v2/genai/comparisonPrompts/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
llmBlueprintIds query any false Only retrieve the comparison prompts associated with the specified LLM blueprint IDs. Either this parameter or comparisonChatId must be specified, but not both.
comparisonChatId query string false Only retrieve the comparison prompts associated with the specified comparison chat ID. Either this parameter or llmBlueprintIds must be specified, but not both.
offset query integer false Skip the specified number of values.
limit query integer false Retrieve only the specified number of values.

Example responses

200 Response

{
  "count": 0,
  "data": [
    {
      "comparisonChatId": "string",
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "results": [
        {
          "chatContextId": "string",
          "citations": [
            {
              "page": 0,
              "source": "string",
              "text": "string"
            }
          ],
          "comparisonPromptResultIdsIncludedInHistory": [
            "string"
          ],
          "confidenceScores": {
            "bleu": 0,
            "meteor": 0,
            "rouge": 0
          },
          "executionStatus": "NEW",
          "id": "string",
          "llmBlueprintId": "string",
          "resultMetadata": {
            "cost": 0,
            "errorMessage": "string",
            "estimatedDocsTokenCount": 0,
            "feedbackResult": {
              "negativeUserIds": [],
              "positiveUserIds": []
            },
            "finalPrompt": "string",
            "inputTokenCount": 0,
            "latencyMilliseconds": 0,
            "metrics": [],
            "outputTokenCount": 0,
            "totalTokenCount": 0
          },
          "resultText": "string"
        }
      ],
      "text": "string",
      "userName": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses

Status Meaning Description Schema
200 OK Successful Response ListComparisonPromptsResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/comparisonPrompts/

Request the execution of a new comparison prompt.

Code samples

# You can also use wget
curl -X POST https://app.datarobot.com/api/v2/genai/comparisonPrompts/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Body parameter

{
  "comparisonChatId": "string",
  "llmBlueprintIds": [
    "string"
  ],
  "text": "string"
}

Parameters

Name In Type Required Description
body body CreateComparisonPromptRequest true none

Example responses

202 Response

{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "page": 0,
          "source": "string",
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

Responses

Status Meaning Description Schema
202 Accepted Successful Response ComparisonPromptResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/comparisonPrompts/{comparisonPromptId}/

Delete an existing comparison prompt.

Code samples

# You can also use wget
curl -X DELETE https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
comparisonPromptId path string true The ID of the comparison prompt to delete.

Example responses

422 Response

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses

Status Meaning Description Schema
204 No Content Successful Response None
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/comparisonPrompts/{comparisonPromptId}/

Retrieve an existing comparison prompt.

Code samples

# You can also use wget
curl -X GET https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
comparisonPromptId path string true The ID of the comparison prompt to retrieve.

Example responses

200 Response

{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "page": 0,
          "source": "string",
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

Responses

Status Meaning Description Schema
200 OK Successful Response ComparisonPromptResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/comparisonPrompts/{comparisonPromptId}/

Edit an existing comparison prompt. Editing may involve adding new prompt result metadata or executing this comparison prompt on new LLM blueprints.

Code samples

# You can also use wget
curl -X PATCH https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Body parameter

{
  "additionalLLMBlueprintIds": []
}

Parameters

Name In Type Required Description
comparisonPromptId path string true The ID of the comparison prompt to edit.
body body EditComparisonPromptRequest true none

Example responses

202 Response

"string"

Responses

Status Meaning Description Schema
202 Accepted Successful Response string
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas

Citation

{
  "page": 0,
  "source": "string",
  "text": "string"
}

Citation

Properties

Name Type Required Restrictions Description
page integer¦null false The source page number where the citation was found.
source string¦null true The source of the citation (e.g., a filename in the original dataset).
text string true The text of the citation.

ComparisonPromptResponse

{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "page": 0,
          "source": "string",
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

ComparisonPromptResponse

Properties

Name Type Required Restrictions Description
comparisonChatId string¦null false The ID of the comparison chat associated with the comparison prompt.
creationDate string(date-time) true The creation date of the comparison prompt (ISO 8601 formatted).
creationUserId string true The ID of the user that created the comparison prompt.
id string true The ID of the comparison prompt.
results [ComparisonPromptResultApiFormatted] true The list of comparison prompt results.
text string true The text of the user prompt.
userName string true The name of the user that created the comparison prompt.

ComparisonPromptResultApiFormatted

{
  "chatContextId": "string",
  "citations": [
    {
      "page": 0,
      "source": "string",
      "text": "string"
    }
  ],
  "comparisonPromptResultIdsIncludedInHistory": [
    "string"
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "resultMetadata": {
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "totalTokenCount": 0
  },
  "resultText": "string"
}

ComparisonPromptResultApiFormatted

Properties

Name Type Required Restrictions Description
chatContextId string¦null false The ID of the chat context for this prompt.
citations [Citation] true The list of relevant vector database citations (in case of using a vector database).
comparisonPromptResultIdsIncludedInHistory [string]¦null false The list of IDs of the comparison prompt results included in this prompt's history.
confidenceScores ConfidenceScores¦null true The confidence scores that measure the similarity between the prompt context and the prompt completion.
executionStatus ExecutionStatus true The execution status of the comparison prompt by this LLM blueprint.
id string true The ID of the comparison prompt result.
llmBlueprintId string true The ID of the LLM blueprint that produced the result.
resultMetadata ResultMetadata¦null false The additional information about the prompt result.
resultText string¦null true The text of the prompt completion.

ConfidenceScores

{
  "bleu": 0,
  "meteor": 0,
  "rouge": 0
}

ConfidenceScores

Properties

Name Type Required Restrictions Description
bleu number true BLEU score.
meteor number true METEOR score.
rouge number true ROUGE score.

CreateComparisonPromptRequest

{
  "comparisonChatId": "string",
  "llmBlueprintIds": [
    "string"
  ],
  "text": "string"
}

CreateComparisonPromptRequest

Properties

Name Type Required Restrictions Description
comparisonChatId string¦null false The ID of the comparison chat to associate the comparison prompt with.
llmBlueprintIds [string] true maxItems: 10
 The list of LLM blueprint IDs that should execute the comparison prompt.
text string true maxLength: 500000
 The text of the user prompt.

EditComparisonPromptRequest

{
  "additionalLLMBlueprintIds": []
}

EditComparisonPromptRequest

Properties

Name Type Required Restrictions Description
additionalLLMBlueprintIds [string] false maxItems: 10
 The list of additional LLM blueprint IDs that should execute this comparison prompt.

ExecutionStatus

"NEW"

ExecutionStatus

Properties

Name Type Required Restrictions Description
ExecutionStatus string false Job execution status.

Enumerated Values

Property Value
ExecutionStatus [NEW, RUNNING, COMPLETED, ERROR]

FeedbackResult

{
  "negativeUserIds": [],
  "positiveUserIds": []
}

FeedbackResult

Properties

Name Type Required Restrictions Description
negativeUserIds [string] false The list of user IDs whose feedback is negative.
positiveUserIds [string] false The list of user IDs whose feedback is positive.

HTTPValidationErrorResponse

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

HTTPValidationErrorResponse

Properties

Name Type Required Restrictions Description
detail [ValidationError] false none

ListComparisonPromptsResponse

{
  "count": 0,
  "data": [
    {
      "comparisonChatId": "string",
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "results": [
        {
          "chatContextId": "string",
          "citations": [
            {
              "page": 0,
              "source": "string",
              "text": "string"
            }
          ],
          "comparisonPromptResultIdsIncludedInHistory": [
            "string"
          ],
          "confidenceScores": {
            "bleu": 0,
            "meteor": 0,
            "rouge": 0
          },
          "executionStatus": "NEW",
          "id": "string",
          "llmBlueprintId": "string",
          "resultMetadata": {
            "cost": 0,
            "errorMessage": "string",
            "estimatedDocsTokenCount": 0,
            "feedbackResult": {
              "negativeUserIds": [],
              "positiveUserIds": []
            },
            "finalPrompt": "string",
            "inputTokenCount": 0,
            "latencyMilliseconds": 0,
            "metrics": [],
            "outputTokenCount": 0,
            "totalTokenCount": 0
          },
          "resultText": "string"
        }
      ],
      "text": "string",
      "userName": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListComparisonPromptsResponse

Properties

Name Type Required Restrictions Description
count integer true The number of records on this page.
data [ComparisonPromptResponse] true The list of records.
next string¦null true The URL to the next page, or null if there is no such page.
previous string¦null true The URL to the previous page, or null if there is no such page.
totalCount integer true The total number of records.

MetricMetadata

{
  "costConfigurationId": "string",
  "customModelId": "string",
  "evaluationDatasetConfigurationId": "string",
  "formattedName": "string",
  "formattedValue": "string",
  "name": "string",
  "sidecarModelMetricValidationId": "string",
  "value": null
}

MetricMetadata

Properties

Name Type Required Restrictions Description
costConfigurationId string¦null false The ID of the cost configuration.
customModelId string¦null false The ID of the custom model used for the metric.
evaluationDatasetConfigurationId string¦null false The ID of the evaluation dataset configuration.
formattedName string¦null false The formatted name of the metric.
formattedValue string¦null false The formatted value of the metric.
name string true The name of the metric.
sidecarModelMetricValidationId string¦null false The validation ID of the sidecar model validation(in case of using a sidecar model deployment for the metric).
value any true The value of the metric.

ResultMetadata

{
  "cost": 0,
  "errorMessage": "string",
  "estimatedDocsTokenCount": 0,
  "feedbackResult": {
    "negativeUserIds": [],
    "positiveUserIds": []
  },
  "finalPrompt": "string",
  "inputTokenCount": 0,
  "latencyMilliseconds": 0,
  "metrics": [],
  "outputTokenCount": 0,
  "totalTokenCount": 0
}

ResultMetadata

Properties

Name Type Required Restrictions Description
cost number¦null false The estimated cost of executing the prompt.
errorMessage string¦null false The error message for the prompt (in case of an errored prompt).
estimatedDocsTokenCount integer false The estimated number of tokens in the documents retrieved from the vector database.
feedbackResult FeedbackResult false The user feedback associated with the prompt.
finalPrompt any false The final representation of the prompt that was submitted to the LLM.

anyOf

Name Type Required Restrictions Description
» anonymous string false none

or

Name Type Required Restrictions Description
» anonymous [object] false none
»» additionalProperties string false none

or

Name Type Required Restrictions Description
» anonymous object false none
»» additionalProperties any false none

anyOf

Name Type Required Restrictions Description
»»» anonymous string false none

or

Name Type Required Restrictions Description
»»» anonymous [object] false none
»»»» additionalProperties string false none

continued

Name Type Required Restrictions Description
inputTokenCount integer false The number of tokens in the LLM input. This number includes the tokens in the system prompt, the user prompt, the chat history (for history-aware chats) and the documents retrieved from the vector database (in case of using a vector database).
latencyMilliseconds integer true The latency of the LLM response (in milliseconds).
metrics [MetricMetadata] false The evaluation metrics for the prompt.
outputTokenCount integer false The number of tokens in the LLM output.
totalTokenCount integer false The combined number of tokens in the LLM input and output.

ValidationError

{
  "loc": [
    "string"
  ],
  "msg": "string",
  "type": "string"
}

ValidationError

Properties

Name Type Required Restrictions Description
loc [anyOf] true none

anyOf

Name Type Required Restrictions Description
» anonymous string false none

or

Name Type Required Restrictions Description
» anonymous integer false none

continued

Name Type Required Restrictions Description
msg string true none
type string true none
Updated October 8, 2024