DataRobot API resources > API reference documentation > DataRobot REST API > Generative AI > Prompting

Prompting (GenAI)¶

This page outlines the operations, endpoints, parameters, and example requests and responses for the Prompting (GenAI).

GET /api/v2/genai/chatPrompts/¶

List chat prompts.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/chatPrompts/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
playgroundId	query	any	false	Only retrieve the chat prompts associated with this playground ID.
llmBlueprintId	query	any	false	Only retrieve the chat prompts associated with this LLM blueprint ID. If specified, will retrieve the chat prompts for the oldest chat in this LLM blueprint.
chatId	query	any	false	Only retrieve the chat prompts associated with this chat ID.
offset	query	integer	false	Skip the specified number of values.
limit	query	integer	false	Retrieve only the specified number of values.

Example responses¶


{
  "count": 0,
  "data": [
    {
      "chatContextId": "string",
      "chatId": "string",
      "chatPromptIdsIncludedInHistory": [
        "string"
      ],
      "citations": [
        {
          "chunkId": 0,
          "metadata": {},
          "page": 0,
          "similarityScore": 0,
          "source": "string",
          "startIndex": 0,
          "text": "string"
        }
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "llmId": "azure-openai-gpt-3.5-turbo",
      "llmSettings": {
        "maxCompletionLength": 0,
        "systemPrompt": "string",
        "temperature": 0,
        "topP": 0
      },
      "metadataFilter": {},
      "resultMetadata": {
        "blockedResultText": "string",
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "providerLLMGuards": [
          {
            "name": "string",
            "satisfyCriteria": true,
            "stage": "prompt",
            "value": "string"
          }
        ],
        "totalTokenCount": 0
      },
      "resultText": "string",
      "text": "string",
      "userName": "string",
      "vectorDatabaseFamilyId": "string",
      "vectorDatabaseId": "string",
      "vectorDatabaseSettings": {
        "addNeighborChunks": false,
        "maxDocumentsRetrievedPerPrompt": 1,
        "maxTokens": 1,
        "retriever": "SINGLE_LOOKUP_RETRIEVER"
      }
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ListChatPromptsResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/chatPrompts/¶

Request the execution of a new prompt within a chat or an LLM blueprint.

Code samples¶


curl -X POST https://app.datarobot.com/api/v2/genai/chatPrompts/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{CreateChatPromptRequest}'

Body parameter¶


{
  "chatId": "string",
  "llmBlueprintId": "string",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "llmSettings": {
    "maxCompletionLength": 0,
    "systemPrompt": "string",
    "temperature": 0,
    "topP": 0
  },
  "metadataFilter": {},
  "text": "string",
  "vectorDatabaseId": "string",
  "vectorDatabaseSettings": {
    "addNeighborChunks": false,
    "maxDocumentsRetrievedPerPrompt": 1,
    "maxTokens": 1,
    "retriever": "SINGLE_LOOKUP_RETRIEVER"
  }
}

Parameters

Name	In	Type	Required	Description
body	body	CreateChatPromptRequest	true	none

Example responses¶


{
  "chatContextId": "string",
  "chatId": "string",
  "chatPromptIdsIncludedInHistory": [
    "string"
  ],
  "citations": [
    {
      "chunkId": 0,
      "metadata": {},
      "page": 0,
      "similarityScore": 0,
      "source": "string",
      "startIndex": 0,
      "text": "string"
    }
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "llmSettings": {
    "maxCompletionLength": 0,
    "systemPrompt": "string",
    "temperature": 0,
    "topP": 0
  },
  "metadataFilter": {},
  "resultMetadata": {
    "blockedResultText": "string",
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "providerLLMGuards": [
      {
        "name": "string",
        "satisfyCriteria": true,
        "stage": "prompt",
        "value": "string"
      }
    ],
    "totalTokenCount": 0
  },
  "resultText": "string",
  "text": "string",
  "userName": "string",
  "vectorDatabaseFamilyId": "string",
  "vectorDatabaseId": "string",
  "vectorDatabaseSettings": {
    "addNeighborChunks": false,
    "maxDocumentsRetrievedPerPrompt": 1,
    "maxTokens": 1,
    "retriever": "SINGLE_LOOKUP_RETRIEVER"
  }
}

Responses¶

Status	Meaning	Description	Schema
202	Accepted	Successful Response	ChatPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/chatPrompts/{chatPromptId}/¶

Delete an existing chat prompt.

Code samples¶


curl -X DELETE https://app.datarobot.com/api/v2/genai/chatPrompts/{chatPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
chatPromptId	path	string	true	The ID of the chat prompt to delete.

Example responses¶


{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses¶

Status	Meaning	Description	Schema
204	No Content	Successful Response	None
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/chatPrompts/{chatPromptId}/¶

Retrieve an existing chat prompt.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/chatPrompts/{chatPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
chatPromptId	path	string	true	The ID of the chat prompt to retrieve.

Example responses¶


{
  "chatContextId": "string",
  "chatId": "string",
  "chatPromptIdsIncludedInHistory": [
    "string"
  ],
  "citations": [
    {
      "chunkId": 0,
      "metadata": {},
      "page": 0,
      "similarityScore": 0,
      "source": "string",
      "startIndex": 0,
      "text": "string"
    }
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "llmSettings": {
    "maxCompletionLength": 0,
    "systemPrompt": "string",
    "temperature": 0,
    "topP": 0
  },
  "metadataFilter": {},
  "resultMetadata": {
    "blockedResultText": "string",
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "providerLLMGuards": [
      {
        "name": "string",
        "satisfyCriteria": true,
        "stage": "prompt",
        "value": "string"
      }
    ],
    "totalTokenCount": 0
  },
  "resultText": "string",
  "text": "string",
  "userName": "string",
  "vectorDatabaseFamilyId": "string",
  "vectorDatabaseId": "string",
  "vectorDatabaseSettings": {
    "addNeighborChunks": false,
    "maxDocumentsRetrievedPerPrompt": 1,
    "maxTokens": 1,
    "retriever": "SINGLE_LOOKUP_RETRIEVER"
  }
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ChatPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/chatPrompts/{chatPromptId}/¶

Edit an existing chat prompt.

Code samples¶


curl -X PATCH https://app.datarobot.com/api/v2/genai/chatPrompts/{chatPromptId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{undefined}'

Body parameter¶


{
  "customMetrics": [
    {
      "costConfigurationId": "string",
      "customModelId": "string",
      "errorMessage": "string",
      "evaluationDatasetConfigurationId": "string",
      "executionStatus": "NEW",
      "formattedName": "string",
      "formattedValue": "string",
      "llmIsDeprecated": true,
      "name": "string",
      "nemoMetricId": "string",
      "ootbMetricId": "string",
      "sidecarModelMetricValidationId": "string",
      "stage": "prompt_pipeline",
      "value": null
    }
  ],
  "feedbackMetadata": {
    "feedback": "1"
  }
}

Parameters

Name	In	Type	Required	Description
chatPromptId	path	string	true	The ID of the chat prompt to edit.
body	body	EditChatPromptRequest	true	none

Example responses¶


{
  "chatContextId": "string",
  "chatId": "string",
  "chatPromptIdsIncludedInHistory": [
    "string"
  ],
  "citations": [
    {
      "chunkId": 0,
      "metadata": {},
      "page": 0,
      "similarityScore": 0,
      "source": "string",
      "startIndex": 0,
      "text": "string"
    }
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "llmSettings": {
    "maxCompletionLength": 0,
    "systemPrompt": "string",
    "temperature": 0,
    "topP": 0
  },
  "metadataFilter": {},
  "resultMetadata": {
    "blockedResultText": "string",
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "providerLLMGuards": [
      {
        "name": "string",
        "satisfyCriteria": true,
        "stage": "prompt",
        "value": "string"
      }
    ],
    "totalTokenCount": 0
  },
  "resultText": "string",
  "text": "string",
  "userName": "string",
  "vectorDatabaseFamilyId": "string",
  "vectorDatabaseId": "string",
  "vectorDatabaseSettings": {
    "addNeighborChunks": false,
    "maxDocumentsRetrievedPerPrompt": 1,
    "maxTokens": 1,
    "retriever": "SINGLE_LOOKUP_RETRIEVER"
  }
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ChatPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/chats/¶

List chats.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/chats/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
llmBlueprintId	query	any	false	Only retrieve the chats associated with this LLM blueprint ID.
offset	query	integer	false	Skip the specified number of values.
limit	query	integer	false	Retrieve only the specified number of values.
sort	query	any	false	Apply this sort order to the results. Valid options are "name" and "creationDate". Prefix the attribute name with a dash to sort in descending order, e.g., sort=-creationDate.

Example responses¶


{
  "count": 0,
  "data": [
    {
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "isFrozen": true,
      "llmBlueprintId": "string",
      "name": "string",
      "promptsCount": 0,
      "warning": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ListChatsResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/chats/¶

Create a new chat.

Code samples¶


curl -X POST https://app.datarobot.com/api/v2/genai/chats/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{CreateChatRequest}'

Body parameter¶


{
  "llmBlueprintId": "string",
  "name": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	CreateChatRequest	true	none

Example responses¶


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "isFrozen": true,
  "llmBlueprintId": "string",
  "name": "string",
  "promptsCount": 0,
  "warning": "string"
}

Responses¶

Status	Meaning	Description	Schema
201	Created	Successful Response	ChatResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/chats/{chatId}/¶

Delete an existing chat.

Code samples¶


curl -X DELETE https://app.datarobot.com/api/v2/genai/chats/{chatId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
chatId	path	string	true	The ID of the chat to delete.

Example responses¶


{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses¶

Status	Meaning	Description	Schema
204	No Content	Successful Response	None
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/chats/{chatId}/¶

Retrieve an existing chat.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/chats/{chatId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
chatId	path	string	true	The ID of the chat to retrieve.

Example responses¶


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "isFrozen": true,
  "llmBlueprintId": "string",
  "name": "string",
  "promptsCount": 0,
  "warning": "string"
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ChatResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/chats/{chatId}/¶

Edit an existing chat.

Code samples¶


curl -X PATCH https://app.datarobot.com/api/v2/genai/chats/{chatId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{undefined}'

Body parameter¶


{
  "name": "string"
}

Parameters

Name	In	Type	Required	Description
chatId	path	string	true	The ID of the chat to edit.
body	body	EditChatRequest	true	none

Example responses¶


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "isFrozen": true,
  "llmBlueprintId": "string",
  "name": "string",
  "promptsCount": 0,
  "warning": "string"
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ChatResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/comparisonChats/¶

List comparison chats.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/comparisonChats/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
playgroundId	query	any	false	Only retrieve the comparison chats associated with this playground ID.
offset	query	integer	false	Skip the specified number of values.
limit	query	integer	false	Retrieve only the specified number of values.
sort	query	any	false	Apply this sort order to the results. Valid options are "name" and "creationDate". Prefix the attribute name with a dash to sort in descending order, e.g., sort=-creationDate.

Example responses¶


{
  "count": 0,
  "data": [
    {
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "name": "string",
      "playgroundId": "string",
      "promptsCount": 0
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ListComparisonChatsResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/comparisonChats/¶

Create a new comparison chat.

Code samples¶


curl -X POST https://app.datarobot.com/api/v2/genai/comparisonChats/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{CreateComparisonChatRequest}'

Body parameter¶


{
  "name": "string",
  "playgroundId": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	CreateComparisonChatRequest	true	none

Example responses¶


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "name": "string",
  "playgroundId": "string",
  "promptsCount": 0
}

Responses¶

Status	Meaning	Description	Schema
201	Created	Successful Response	ComparisonChatResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/comparisonChats/{comparisonChatId}/¶

Delete an existing comparison chat.

Code samples¶


curl -X DELETE https://app.datarobot.com/api/v2/genai/comparisonChats/{comparisonChatId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
comparisonChatId	path	string	true	The ID of the comparison chat to delete.

Example responses¶


{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses¶

Status	Meaning	Description	Schema
204	No Content	Successful Response	None
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/comparisonChats/{comparisonChatId}/¶

Retrieve an existing comparison chat.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/comparisonChats/{comparisonChatId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
comparisonChatId	path	string	true	The ID of the comparison chat to retrieve.

Example responses¶


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "name": "string",
  "playgroundId": "string",
  "promptsCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ComparisonChatResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/comparisonChats/{comparisonChatId}/¶

Edit an existing comparison chat.

Code samples¶


curl -X PATCH https://app.datarobot.com/api/v2/genai/comparisonChats/{comparisonChatId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{undefined}'

Body parameter¶


{
  "name": "string"
}

Parameters

Name	In	Type	Required	Description
comparisonChatId	path	string	true	The ID of the comparison chat to edit.
body	body	EditComparisonChatRequest	true	none

Example responses¶


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "name": "string",
  "playgroundId": "string",
  "promptsCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ComparisonChatResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/comparisonPrompts/¶

List the comparison prompts associated with a comparison chat or a set of LLM blueprints.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/comparisonPrompts/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
llmBlueprintIds	query	any	false	Only retrieve the comparison prompts associated with the specified LLM blueprint IDs. Either this parameter or `comparisonChatId` must be specified, but not both.
comparisonChatId	query	any	false	Only retrieve the comparison prompts associated with the specified comparison chat ID. Either this parameter or `llmBlueprintIds` must be specified, but not both.
offset	query	integer	false	Skip the specified number of values.
limit	query	integer	false	Retrieve only the specified number of values.

Example responses¶


{
  "count": 0,
  "data": [
    {
      "comparisonChatId": "string",
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "executionStatus": "NEW",
      "id": "string",
      "metadataFilter": {},
      "results": [
        {
          "chatContextId": "string",
          "citations": [
            {
              "chunkId": 0,
              "metadata": {},
              "page": 0,
              "similarityScore": 0,
              "source": "string",
              "startIndex": 0,
              "text": "string"
            }
          ],
          "comparisonPromptResultIdsIncludedInHistory": [
            "string"
          ],
          "confidenceScores": {
            "bleu": 0,
            "meteor": 0,
            "rouge": 0
          },
          "executionStatus": "NEW",
          "id": "string",
          "llmBlueprintId": "string",
          "resultMetadata": {
            "blockedResultText": "string",
            "cost": 0,
            "errorMessage": "string",
            "estimatedDocsTokenCount": 0,
            "feedbackResult": {
              "negativeUserIds": [],
              "positiveUserIds": []
            },
            "finalPrompt": "string",
            "inputTokenCount": 0,
            "latencyMilliseconds": 0,
            "metrics": [],
            "outputTokenCount": 0,
            "providerLLMGuards": [
              {
                "name": "string",
                "satisfyCriteria": true,
                "stage": "prompt",
                "value": "string"
              }
            ],
            "totalTokenCount": 0
          },
          "resultText": "string"
        }
      ],
      "text": "string",
      "userName": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ListComparisonPromptsResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/comparisonPrompts/¶

Request the execution of a new comparison prompt.

Code samples¶


curl -X POST https://app.datarobot.com/api/v2/genai/comparisonPrompts/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{CreateComparisonPromptRequest}'

Body parameter¶


{
  "comparisonChatId": "string",
  "llmBlueprintIds": [
    "string"
  ],
  "metadataFilter": {},
  "text": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	CreateComparisonPromptRequest	true	none

Example responses¶


{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "metadataFilter": {},
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "chunkId": 0,
          "metadata": {},
          "page": 0,
          "similarityScore": 0,
          "source": "string",
          "startIndex": 0,
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "blockedResultText": "string",
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "providerLLMGuards": [
          {
            "name": "string",
            "satisfyCriteria": true,
            "stage": "prompt",
            "value": "string"
          }
        ],
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

Responses¶

Status	Meaning	Description	Schema
202	Accepted	Successful Response	ComparisonPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/comparisonPrompts/{comparisonPromptId}/¶

Delete an existing comparison prompt.

Code samples¶


curl -X DELETE https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
comparisonPromptId	path	string	true	The ID of the comparison prompt to delete.

Example responses¶


{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses¶

Status	Meaning	Description	Schema
204	No Content	Successful Response	None
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/comparisonPrompts/{comparisonPromptId}/¶

Retrieve an existing comparison prompt.

Code samples¶


curl -X GET https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
comparisonPromptId	path	string	true	The ID of the comparison prompt to retrieve.

Example responses¶


{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "metadataFilter": {},
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "chunkId": 0,
          "metadata": {},
          "page": 0,
          "similarityScore": 0,
          "source": "string",
          "startIndex": 0,
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "blockedResultText": "string",
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "providerLLMGuards": [
          {
            "name": "string",
            "satisfyCriteria": true,
            "stage": "prompt",
            "value": "string"
          }
        ],
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ComparisonPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/comparisonPrompts/{comparisonPromptId}/¶

Edit an existing comparison prompt. Editing may involve adding new prompt result metadata or executing this comparison prompt on new LLM blueprints.

Code samples¶


curl -X PATCH https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{undefined}'

Body parameter¶


{
  "additionalLLMBlueprintIds": [],
  "feedbackResult": {
    "comparisonPromptResultId": "string",
    "feedbackMetadata": {
      "feedback": "1"
    }
  }
}

Parameters

Name	In	Type	Required	Description
comparisonPromptId	path	string	true	The ID of the comparison prompt to edit.
body	body	EditComparisonPromptRequest	true	none

Example responses¶


"string"

Responses¶

Status	Meaning	Description	Schema
202	Accepted	Successful Response	string
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas¶

ChatPromptResponse


{
  "chatContextId": "string",
  "chatId": "string",
  "chatPromptIdsIncludedInHistory": [
    "string"
  ],
  "citations": [
    {
      "chunkId": 0,
      "metadata": {},
      "page": 0,
      "similarityScore": 0,
      "source": "string",
      "startIndex": 0,
      "text": "string"
    }
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "llmSettings": {
    "maxCompletionLength": 0,
    "systemPrompt": "string",
    "temperature": 0,
    "topP": 0
  },
  "metadataFilter": {},
  "resultMetadata": {
    "blockedResultText": "string",
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "providerLLMGuards": [
      {
        "name": "string",
        "satisfyCriteria": true,
        "stage": "prompt",
        "value": "string"
      }
    ],
    "totalTokenCount": 0
  },
  "resultText": "string",
  "text": "string",
  "userName": "string",
  "vectorDatabaseFamilyId": "string",
  "vectorDatabaseId": "string",
  "vectorDatabaseSettings": {
    "addNeighborChunks": false,
    "maxDocumentsRetrievedPerPrompt": 1,
    "maxTokens": 1,
    "retriever": "SINGLE_LOOKUP_RETRIEVER"
  }
}

ChatPromptResponse

Properties¶

Name	Type	Required	Restrictions	Description
chatContextId	any	false		The ID of the chat context for this prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
chatId	any	false		The ID of the chat this chat prompt belongs to.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
chatPromptIdsIncludedInHistory	any	false		The list of IDs of the chat prompts included in this prompt's history.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	[string]	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
citations	[Citation]	true		The list of relevant vector database citations (in case of using a vector database).
confidenceScores	any	true		The confidence scores that measure the similarity between the prompt context and the prompt completion.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	ConfidenceScores	false		API response object for confidence scores.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
creationDate	string(date-time)	true	The creation date of the chat prompt (ISO 8601 formatted).
creationUserId	string	true	The ID of the user that created the chat prompt.
executionStatus	ExecutionStatus	true	The execution status of the chat prompt.
id	string	true	The ID of the chat prompt.
llmBlueprintId	string	true	The ID of the LLM blueprint the chat prompt belongs to.
llmId	LanguageModelTypeId	true	The ID of the LLM used by the chat prompt.
llmSettings	any	false	A key/value dictionary of LLM settings.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	CommonLLMSettings	false		The settings that are available for all non-custom LLMs.

or

Name	Type	Required	Restrictions	Description
» anonymous	CustomModelLLMSettings	false		The settings that are available for custom model LLMs.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
metadataFilter	any	false		The metadata dictionary defining the filters that documents must match in order to be retrieved.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
resultMetadata	any	true		The additional information about the chat prompt results.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	ResultMetadata	false		The additional information about prompt execution results.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
resultText	any	true		The text of the prompt completion.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
text	string	true	The text of the user prompt.
userName	string	true	The name of the user that created the chat prompt.
vectorDatabaseFamilyId	any	false	The ID of the vector database family this chat prompt belongs to.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
vectorDatabaseId	any	false		The ID of the vector database linked to this LLM blueprint.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
vectorDatabaseSettings	any	false		A key/value dictionary of vector database settings.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	VectorDatabaseSettings	false		Vector database retrieval settings.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

ChatResponse


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "isFrozen": true,
  "llmBlueprintId": "string",
  "name": "string",
  "promptsCount": 0,
  "warning": "string"
}

ChatResponse

Properties¶

Name	Type	Required	Description
creationDate	string(date-time)	true	The creation date of the chat (ISO 8601 formatted).
creationUserId	string	true	The ID of the user that created the chat.
id	string	true	The ID of the chat.
isFrozen	boolean	true	Whether the chat is frozen (e.g., an evaluation chat). If the chat is frozen, it does not accept new prompts.
llmBlueprintId	string	true	The ID of the LLM blueprint associated with the chat.
name	string	true	The name of the chat.
promptsCount	integer	true	The number of chat prompts in the chat.
warning	any	true	Warning about the contents of the chat.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

ChatsSortQueryParam


"name"

ChatsSortQueryParam

Properties¶

Name	Type	Required	Restrictions	Description
ChatsSortQueryParam	string	false		Sort order values for listing chats.

Enumerated Values¶

Property	Value
ChatsSortQueryParam	[`name`, `-name`, `creationDate`, `-creationDate`]

Citation


{
  "chunkId": 0,
  "metadata": {},
  "page": 0,
  "similarityScore": 0,
  "source": "string",
  "startIndex": 0,
  "text": "string"
}

Citation

Properties¶

Name	Type	Required	Restrictions	Description
chunkId	any	false		The ID of the chunk in the vector database index.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
metadata	any	false		LangChain Document metadata information holder.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
page	any	false		The source page number where the citation was found.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
similarityScore	any	false		The similarity score between the citation and the user prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	number	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
source	any	true		The source of the citation (e.g., a filename in the original dataset).

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
startIndex	any	false		The chunk's start character index in the source document.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
text	string	true		The text of the citation.

CommonLLMSettings


{
  "maxCompletionLength": 0,
  "systemPrompt": "string",
  "temperature": 0,
  "topP": 0
}

CommonLLMSettings

Properties¶

Name	Type	Required	Restrictions	Description
maxCompletionLength	any	false		Maximum number of tokens allowed in the completion. The combined count of this value and prompt tokens must be below the model's maximum context size, where prompt token count is comprised of system prompt, user prompt, recent chat history, and vector database citations.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
systemPrompt	any	false		System prompt guides the style of the LLM response. It is a "universal" prompt, prepended to all individual prompts.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false	maxLength: 500000	none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
temperature	any	false		Temperature controls the randomness of model output, where higher values return more diverse output and lower values return more deterministic results.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	number	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
topP	any	false		Top P sets a threshold that controls the selection of words included in the response, based on a cumulative probability cutoff for token selection. For example, 0.2 considers only the top 20% probability mass. Higher numbers return more diverse options for outputs.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	number	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

ComparisonChatResponse


{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "name": "string",
  "playgroundId": "string",
  "promptsCount": 0
}

ComparisonChatResponse

Properties¶

Name	Type	Required	Description
creationDate	string(date-time)	true	The creation date of the comparison chat (ISO 8601 formatted).
creationUserId	string	true	The ID of the user that created the comparison chat.
id	string	true	The ID of the comparison chat.
name	string	true	The name of the comparison chat.
playgroundId	string	true	The ID of the playground associated with the comparison chat.
promptsCount	integer	true	The number of comparison prompts in the comparison chat.

ComparisonChatsSortQueryParam


"name"

ComparisonChatsSortQueryParam

Properties¶

Name	Type	Required	Restrictions	Description
ComparisonChatsSortQueryParam	string	false		Sort order values for listing comparison chats.

Enumerated Values¶

Property	Value
ComparisonChatsSortQueryParam	[`name`, `-name`, `creationDate`, `-creationDate`]

ComparisonPromptFeedbackResult


{
  "comparisonPromptResultId": "string",
  "feedbackMetadata": {
    "feedback": "1"
  }
}

ComparisonPromptFeedbackResult

Properties¶

Name	Type	Required	Restrictions	Description
comparisonPromptResultId	string	true		The ID of the comparison prompt result associated with this feedback.
feedbackMetadata	FeedbackMetadata	true		The feedback metadata for the comparison prompt result.

ComparisonPromptResponse


{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "executionStatus": "NEW",
  "id": "string",
  "metadataFilter": {},
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "chunkId": 0,
          "metadata": {},
          "page": 0,
          "similarityScore": 0,
          "source": "string",
          "startIndex": 0,
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "blockedResultText": "string",
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "providerLLMGuards": [
          {
            "name": "string",
            "satisfyCriteria": true,
            "stage": "prompt",
            "value": "string"
          }
        ],
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

ComparisonPromptResponse

Properties¶

Name	Type	Required	Restrictions	Description
comparisonChatId	any	false		The ID of the comparison chat associated with the comparison prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
creationDate	string(date-time)	true	The creation date of the comparison prompt (ISO 8601 formatted).
creationUserId	string	true	The ID of the user that created the comparison prompt.
executionStatus	ExecutionStatus	true	The execution status of the entire comparison prompt.
id	string	true	The ID of the comparison prompt.
metadataFilter	any	false	The metadata filters applied to the comparison prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
results	[ComparisonPromptResult]	true	The list of comparison prompt results.
text	string	true	The text of the user prompt.
userName	string	true	The name of the user that created the comparison prompt.

ComparisonPromptResult


{
  "chatContextId": "string",
  "citations": [
    {
      "chunkId": 0,
      "metadata": {},
      "page": 0,
      "similarityScore": 0,
      "source": "string",
      "startIndex": 0,
      "text": "string"
    }
  ],
  "comparisonPromptResultIdsIncludedInHistory": [
    "string"
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "resultMetadata": {
    "blockedResultText": "string",
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "providerLLMGuards": [
      {
        "name": "string",
        "satisfyCriteria": true,
        "stage": "prompt",
        "value": "string"
      }
    ],
    "totalTokenCount": 0
  },
  "resultText": "string"
}

ComparisonPromptResult

Properties¶

Name	Type	Required	Restrictions	Description
chatContextId	any	false		The ID of the chat context for this prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
citations	[Citation]	true		The list of relevant vector database citations (in case of using a vector database).
comparisonPromptResultIdsIncludedInHistory	any	false		The list of IDs of the comparison prompt results included in this prompt's history.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	[string]	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
confidenceScores	any	true		The confidence scores that measure the similarity between the prompt context and the prompt completion.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	ConfidenceScores	false		API response object for confidence scores.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
executionStatus	ExecutionStatus	true	The execution status of the comparison prompt by this LLM blueprint.
id	string	true	The ID of the comparison prompt result.
llmBlueprintId	string	true	The ID of the LLM blueprint that produced the result.
resultMetadata	any	false	The additional information about the prompt result.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	ResultMetadata	false		The additional information about prompt execution results.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
resultText	any	true		The text of the prompt completion.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

ConfidenceScores


{
  "bleu": 0,
  "meteor": 0,
  "rouge": 0
}

ConfidenceScores

Properties¶

Name	Type	Required	Description
bleu	number	true	BLEU score.
meteor	number	true	METEOR score.
rouge	number	true	ROUGE score.

CreateChatPromptRequest


{
  "chatId": "string",
  "llmBlueprintId": "string",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "llmSettings": {
    "maxCompletionLength": 0,
    "systemPrompt": "string",
    "temperature": 0,
    "topP": 0
  },
  "metadataFilter": {},
  "text": "string",
  "vectorDatabaseId": "string",
  "vectorDatabaseSettings": {
    "addNeighborChunks": false,
    "maxDocumentsRetrievedPerPrompt": 1,
    "maxTokens": 1,
    "retriever": "SINGLE_LOOKUP_RETRIEVER"
  }
}

CreateChatPromptRequest

Properties¶

Name	Type	Required	Restrictions	Description
chatId	any	false		The ID of the chat this prompt belongs to. If LLM and vector database settings are not specified in the request, then the prompt will use the current settings of the chat.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
llmBlueprintId	any	false		The ID of the LLM blueprint this prompt belongs to. If LLM and vector database settings are not specified in the request, then the prompt will use the current settings of the LLM blueprint.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
llmId	any	false		If specified, uses this LLM ID for the prompt and updates the settings of the corresponding chat or LLM blueprint to use this LLM ID.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	LanguageModelTypeId	false		The ID that defines the type of the LLM.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
llmSettings	any	false		If specified, uses these LLM settings for the prompt and updates the settings of the corresponding chat or LLM blueprint to use these LLM settings.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	CommonLLMSettings	false		The settings that are available for all non-custom LLMs.

or

Name	Type	Required	Restrictions	Description
» anonymous	CustomModelLLMSettings	false		The settings that are available for custom model LLMs.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
metadataFilter	any	false		The metadata fields to add to the chat prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
text	string	true	maxLength: 500000	The text of the user prompt.
vectorDatabaseId	any	false		If specified, uses this vector database ID for the prompt and updates the settings of the corresponding chat or LLM blueprint to use this vector database ID.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
vectorDatabaseSettings	any	false		If specified, uses these vector database settings for the prompt and updates the settings of the corresponding chat or LLM blueprint to use these vector database settings.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	VectorDatabaseSettings	false		Vector database retrieval settings.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

CreateChatRequest


{
  "llmBlueprintId": "string",
  "name": "string"
}

CreateChatRequest

Properties¶

Name	Type	Required	Restrictions	Description
llmBlueprintId	string	true		The ID of the LLM blueprint to associate with the chat.
name	string	true	maxLength: 5000 minLength: 1 minLength: 1	The name of the chat.

CreateComparisonChatRequest


{
  "name": "string",
  "playgroundId": "string"
}

CreateComparisonChatRequest

Properties¶

Name	Type	Required	Restrictions	Description
name	string	true	maxLength: 5000 minLength: 1 minLength: 1	The name of the comparison chat.
playgroundId	string	true		The ID of the playground to associate with the comparison chat.

CreateComparisonPromptRequest


{
  "comparisonChatId": "string",
  "llmBlueprintIds": [
    "string"
  ],
  "metadataFilter": {},
  "text": "string"
}

CreateComparisonPromptRequest

Properties¶

Name	Type	Required	Restrictions	Description
comparisonChatId	any	false		The ID of the comparison chat to associate the comparison prompt with.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
llmBlueprintIds	[string]	true	maxItems: 10	The list of LLM blueprint IDs that should execute the comparison prompt.
metadataFilter	any	false		The metadata dict that defines filters that the retrieved documents need to match.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
text	string	true	maxLength: 500000	The text of the user prompt.

CustomModelLLMSettings


{
  "externalLlmContextSize": 128,
  "systemPrompt": "string",
  "validationId": "string"
}

CustomModelLLMSettings

Properties¶

Name	Type	Required	Restrictions	Description
externalLlmContextSize	any	false		The external LLM's context size, in tokens. This value is only used for pruning documents supplied to the LLM when a vector database is associated with the LLM blueprint. It does not affect the external LLM's actual context size in any way and is not supplied to the LLM.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false	maximum: 128000 minimum: 128	none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
systemPrompt	any	false		System prompt guides the style of the LLM response. It is a "universal" prompt, prepended to all individual prompts.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false	maxLength: 500000	none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
validationId	any	false		The validation ID of the custom model LLM.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

EditChatPromptRequest


{
  "customMetrics": [
    {
      "costConfigurationId": "string",
      "customModelId": "string",
      "errorMessage": "string",
      "evaluationDatasetConfigurationId": "string",
      "executionStatus": "NEW",
      "formattedName": "string",
      "formattedValue": "string",
      "llmIsDeprecated": true,
      "name": "string",
      "nemoMetricId": "string",
      "ootbMetricId": "string",
      "sidecarModelMetricValidationId": "string",
      "stage": "prompt_pipeline",
      "value": null
    }
  ],
  "feedbackMetadata": {
    "feedback": "1"
  }
}

EditChatPromptRequest

Properties¶

Name	Type	Required	Restrictions	Description
customMetrics	any	false		The list of metric results to add to the chat prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	[MetricMetadata]	false		[Prompt metric metadata.]

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
feedbackMetadata	any	false		The feedback metadata to add to the chat prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	FeedbackMetadata	false		Prompt feedback metadata.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

EditChatRequest


{
  "name": "string"
}

EditChatRequest

Properties¶

Name	Type	Required	Restrictions	Description
name	string	true	maxLength: 5000 minLength: 1 minLength: 1	The new name of the chat.

EditComparisonChatRequest


{
  "name": "string"
}

EditComparisonChatRequest

Properties¶

Name	Type	Required	Restrictions	Description
name	string	true	maxLength: 5000 minLength: 1 minLength: 1	The new name of the comparison chat.

EditComparisonPromptRequest


{
  "additionalLLMBlueprintIds": [],
  "feedbackResult": {
    "comparisonPromptResultId": "string",
    "feedbackMetadata": {
      "feedback": "1"
    }
  }
}

EditComparisonPromptRequest

Properties¶

Name	Type	Required	Restrictions	Description
additionalLLMBlueprintIds	[string]	false	maxItems: 10	The list of additional LLM blueprint IDs that should execute this comparison prompt.
feedbackResult	any	false		The feedback information to add to the comparison prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	ComparisonPromptFeedbackResult	false		Feedback metadata for a comparison prompt result.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

ExecutionStatus


"NEW"

ExecutionStatus

Properties¶

Name	Type	Required	Restrictions	Description
ExecutionStatus	string	false		Job and entity execution status.

Enumerated Values¶

Property	Value
ExecutionStatus	[`NEW`, `RUNNING`, `COMPLETED`, `REQUIRES_USER_INPUT`, `SKIPPED`, `ERROR`]

FeedbackMetadata


{
  "feedback": "1"
}

FeedbackMetadata

Properties¶

Name	Type	Required	Restrictions	Description
feedback	any	true		The sentiment of the feedback.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	FeedbackSentiment	false		The sentiment of the feedback.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

FeedbackResult


{
  "negativeUserIds": [],
  "positiveUserIds": []
}

FeedbackResult

Properties¶

Name	Type	Required	Restrictions	Description
negativeUserIds	[string]	false		The list of user IDs whose feedback is negative.
positiveUserIds	[string]	false		The list of user IDs whose feedback is positive.

FeedbackSentiment

"1"

FeedbackSentiment

Properties¶

Name	Type	Required	Restrictions	Description
FeedbackSentiment	string	false		The sentiment of the feedback.

Enumerated Values¶

Property	Value
FeedbackSentiment	[`1`, `0`]

HTTPValidationErrorResponse


{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

HTTPValidationErrorResponse

Properties¶

Name	Type	Required	Restrictions	Description
detail	[ValidationError]	false		none

LanguageModelTypeId


"azure-openai-gpt-3.5-turbo"

LanguageModelTypeId

Properties¶

Name	Type	Required	Restrictions	Description
LanguageModelTypeId	string	false		The ID that defines the type of the LLM.

Enumerated Values¶

Property Value

LanguageModelTypeId [azure-openai-gpt-3.5-turbo, azure-openai-gpt-3.5-turbo-16k, azure-openai-gpt-4, azure-openai-gpt-4-32k, azure-openai-gpt-4-turbo, azure-openai-gpt-4-o, azure-openai-gpt-4-o-mini, amazon-titan, amazon-nova-micro, amazon-nova-lite, amazon-nova-pro, anthropic-claude-2, anthropic-claude-3-haiku, anthropic-claude-3-sonnet, anthropic-claude-3-opus, anthropic-claude-3.5-sonnet-v1, amazon-anthropic-claude-3.5-sonnet-v2, google-bison, google-gemini-1.5-flash, google-gemini-1.5-pro, custom-model]

ListChatPromptsResponse


{
  "count": 0,
  "data": [
    {
      "chatContextId": "string",
      "chatId": "string",
      "chatPromptIdsIncludedInHistory": [
        "string"
      ],
      "citations": [
        {
          "chunkId": 0,
          "metadata": {},
          "page": 0,
          "similarityScore": 0,
          "source": "string",
          "startIndex": 0,
          "text": "string"
        }
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "llmId": "azure-openai-gpt-3.5-turbo",
      "llmSettings": {
        "maxCompletionLength": 0,
        "systemPrompt": "string",
        "temperature": 0,
        "topP": 0
      },
      "metadataFilter": {},
      "resultMetadata": {
        "blockedResultText": "string",
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "providerLLMGuards": [
          {
            "name": "string",
            "satisfyCriteria": true,
            "stage": "prompt",
            "value": "string"
          }
        ],
        "totalTokenCount": 0
      },
      "resultText": "string",
      "text": "string",
      "userName": "string",
      "vectorDatabaseFamilyId": "string",
      "vectorDatabaseId": "string",
      "vectorDatabaseSettings": {
        "addNeighborChunks": false,
        "maxDocumentsRetrievedPerPrompt": 1,
        "maxTokens": 1,
        "retriever": "SINGLE_LOOKUP_RETRIEVER"
      }
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListChatPromptsResponse

Properties¶

Name	Type	Required	Description
count	integer	true	The number of records on this page.
data	[ChatPromptResponse]	true	The list of records.
next	any	true	The URL to the next page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
previous	any	true		The URL to the previous page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
totalCount	integer	true		The total number of records.

ListChatsResponse


{
  "count": 0,
  "data": [
    {
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "isFrozen": true,
      "llmBlueprintId": "string",
      "name": "string",
      "promptsCount": 0,
      "warning": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListChatsResponse

Properties¶

Name	Type	Required	Description
count	integer	true	The number of records on this page.
data	[ChatResponse]	true	The list of records.
next	any	true	The URL to the next page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
previous	any	true		The URL to the previous page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
totalCount	integer	true		The total number of records.

ListComparisonChatsResponse


{
  "count": 0,
  "data": [
    {
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "name": "string",
      "playgroundId": "string",
      "promptsCount": 0
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListComparisonChatsResponse

Properties¶

Name	Type	Required	Description
count	integer	true	The number of records on this page.
data	[ComparisonChatResponse]	true	The list of records.
next	any	true	The URL to the next page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
previous	any	true		The URL to the previous page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
totalCount	integer	true		The total number of records.

ListComparisonPromptsResponse


{
  "count": 0,
  "data": [
    {
      "comparisonChatId": "string",
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "executionStatus": "NEW",
      "id": "string",
      "metadataFilter": {},
      "results": [
        {
          "chatContextId": "string",
          "citations": [
            {
              "chunkId": 0,
              "metadata": {},
              "page": 0,
              "similarityScore": 0,
              "source": "string",
              "startIndex": 0,
              "text": "string"
            }
          ],
          "comparisonPromptResultIdsIncludedInHistory": [
            "string"
          ],
          "confidenceScores": {
            "bleu": 0,
            "meteor": 0,
            "rouge": 0
          },
          "executionStatus": "NEW",
          "id": "string",
          "llmBlueprintId": "string",
          "resultMetadata": {
            "blockedResultText": "string",
            "cost": 0,
            "errorMessage": "string",
            "estimatedDocsTokenCount": 0,
            "feedbackResult": {
              "negativeUserIds": [],
              "positiveUserIds": []
            },
            "finalPrompt": "string",
            "inputTokenCount": 0,
            "latencyMilliseconds": 0,
            "metrics": [],
            "outputTokenCount": 0,
            "providerLLMGuards": [
              {
                "name": "string",
                "satisfyCriteria": true,
                "stage": "prompt",
                "value": "string"
              }
            ],
            "totalTokenCount": 0
          },
          "resultText": "string"
        }
      ],
      "text": "string",
      "userName": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListComparisonPromptsResponse

Properties¶

Name	Type	Required	Description
count	integer	true	The number of records on this page.
data	[ComparisonPromptResponse]	true	The list of records.
next	any	true	The URL to the next page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
previous	any	true		The URL to the previous page, or `null` if there is no such page.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
totalCount	integer	true		The total number of records.

MetricMetadata


{
  "costConfigurationId": "string",
  "customModelId": "string",
  "errorMessage": "string",
  "evaluationDatasetConfigurationId": "string",
  "executionStatus": "NEW",
  "formattedName": "string",
  "formattedValue": "string",
  "llmIsDeprecated": true,
  "name": "string",
  "nemoMetricId": "string",
  "ootbMetricId": "string",
  "sidecarModelMetricValidationId": "string",
  "stage": "prompt_pipeline",
  "value": null
}

MetricMetadata

Properties¶

Name	Type	Required	Restrictions	Description
costConfigurationId	any	false		The ID of the cost configuration.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
customModelId	any	false		The ID of the custom model used for the metric.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
errorMessage	any	false		The error message associated with the metric computation.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
evaluationDatasetConfigurationId	any	false		The ID of the evaluation dataset configuration.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
executionStatus	any	false		The creation status of the vector database.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	ExecutionStatus	false		Job and entity execution status.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
formattedName	any	false		The formatted name of the metric.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
formattedValue	any	false		The formatted value of the metric.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
llmIsDeprecated	any	false		Whether the LLM is deprecated and will be removed in a future release.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	boolean	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
name	string	true		The name of the metric.
nemoMetricId	any	false		The id of the NeMo Pipeline configuration.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
ootbMetricId	any	false		The id of the OOTB metric configuration.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
sidecarModelMetricValidationId	any	false		The validation ID of the sidecar model validation(in case of using a sidecar model deployment for the metric).

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
stage	any	false		The stage (prompt or response) that the metric applies to.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	PipelineStage	false		Enum that describes at which stage the metric may be calculated.

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
value	any	true		The value of the metric.

PipelineStage


"prompt_pipeline"

PipelineStage

Properties¶

Name	Type	Required	Restrictions	Description
PipelineStage	string	false		Enum that describes at which stage the metric may be calculated.

Enumerated Values¶

Property	Value
PipelineStage	[`prompt_pipeline`, `response_pipeline`]

ProviderGuardStage


"prompt"

ProviderGuardStage

Properties¶

Name	Type	Required	Restrictions	Description
ProviderGuardStage	string	false		The data stage where the provider guard metric is acting upon.

Enumerated Values¶

Property	Value
ProviderGuardStage	[`prompt`, `response`]

ProviderGuardsMetadata


{
  "name": "string",
  "satisfyCriteria": true,
  "stage": "prompt",
  "value": "string"
}

ProviderGuardsMetadata

Properties¶

Name	Type	Required	Description
name	string	true	The name of the provider guard metric.
satisfyCriteria	boolean	true	Whether the configured provider guard metric satisfied its hidden internal guard criteria.
stage	ProviderGuardStage	true	The data stage where the provider guard metric is acting upon.
value	any	true	The value of the provider guard metric.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	number	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

ResultMetadata


{
  "blockedResultText": "string",
  "cost": 0,
  "errorMessage": "string",
  "estimatedDocsTokenCount": 0,
  "feedbackResult": {
    "negativeUserIds": [],
    "positiveUserIds": []
  },
  "finalPrompt": "string",
  "inputTokenCount": 0,
  "latencyMilliseconds": 0,
  "metrics": [],
  "outputTokenCount": 0,
  "providerLLMGuards": [
    {
      "name": "string",
      "satisfyCriteria": true,
      "stage": "prompt",
      "value": "string"
    }
  ],
  "totalTokenCount": 0
}

ResultMetadata

Properties¶

Name	Type	Required	Restrictions	Description
blockedResultText	any	false		The message to replace the result text if it is non empty, which represents a blocked response.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
cost	any	false		The estimated cost of executing the prompt.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	number	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
errorMessage	any	false		The error message for the prompt (in case of an errored prompt).

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
estimatedDocsTokenCount	integer	false	The estimated number of tokens in the documents retrieved from the vector database.
feedbackResult	FeedbackResult	false	The user feedback associated with the prompt.
finalPrompt	any	false	The final representation of the prompt that was submitted to the LLM.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	[object]	false		none
»» additionalProperties	any	false		none

anyOf

Name	Type	Required	Restrictions	Description
»»» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
»»» anonymous	[object]	false		none

or

Name	Type	Required	Restrictions	Description
»»» anonymous	null	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	[object]	false		none
»» additionalProperties	any	false		none

anyOf

Name	Type	Required	Restrictions	Description
»»» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
»»» anonymous	[object]	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none
»» additionalProperties	any	false		none

anyOf

Name	Type	Required	Restrictions	Description
»»» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
»»» anonymous	[object]	false		none
»»»» additionalProperties	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Description
inputTokenCount	integer	false	The number of tokens in the LLM input. This number includes the tokens in the system prompt, the user prompt, the chat history (for history-aware chats) and the documents retrieved from the vector database (in case of using a vector database).
latencyMilliseconds	integer	true	The latency of the LLM response (in milliseconds).
metrics	[MetricMetadata]	false	The evaluation metrics for the prompt.
outputTokenCount	integer	false	The number of tokens in the LLM output.
providerLLMGuards	any	false	The provider llm guards metadata.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	[ProviderGuardsMetadata]	false		[Info on the provider guard metrics.]

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
totalTokenCount	integer	false		The combined number of tokens in the LLM input and output.

ValidationError


{
  "loc": [
    "string"
  ],
  "msg": "string",
  "type": "string"
}

ValidationError

Properties¶

Name	Type	Required	Restrictions	Description
loc	[anyOf]	true		none

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

continued

Name	Type	Required	Restrictions	Description
msg	string	true		none
type	string	true		none

VectorDatabaseRetrievers


"SINGLE_LOOKUP_RETRIEVER"

VectorDatabaseRetrievers

Properties¶

Name	Type	Required	Restrictions	Description
VectorDatabaseRetrievers	string	false		The method used to retrieve relevant chunks from the vector database.

Enumerated Values¶

Property	Value
VectorDatabaseRetrievers	[`SINGLE_LOOKUP_RETRIEVER`, `CONVERSATIONAL_RETRIEVER`, `MULTI_STEP_RETRIEVER`]

VectorDatabaseSettings


{
  "addNeighborChunks": false,
  "maxDocumentsRetrievedPerPrompt": 1,
  "maxTokens": 1,
  "retriever": "SINGLE_LOOKUP_RETRIEVER"
}

VectorDatabaseSettings

Properties¶

Name	Type	Required	Restrictions	Description
addNeighborChunks	boolean	false		Add neighboring chunks to those that the similarity search retrieves, such that when selected, search returns i, i-1, and i+1.
maxDocumentsRetrievedPerPrompt	any	false		The maximum number of chunks to retrieve from the vector database.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false	maximum: 10 minimum: 1	none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
maxTokens	any	false		The maximum number of tokens to retrieve from the vector database.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	integer	false	maximum: 51200 minimum: 1	none

or

Name	Type	Required	Restrictions	Description
» anonymous	null	false		none

continued

Name	Type	Required	Restrictions	Description
retriever	VectorDatabaseRetrievers	false		The method used to retrieve relevant chunks from the vector database.