Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

DataRobot API resources > API reference documentation > DataRobot REST API > Develop > Generative AI > Custom Model LLMs (GenAI) > LLM Test Suites (GenAI)

LLM Test Suites (GenAI)

This page outlines the operations, endpoints, parameters, and example requests and responses for the LLM Test Suites (GenAI).

GET /api/v2/genai/llmTestSuites/

List LLM test suites.

Code samples

# You can also use wget
curl -X GET https://app.datarobot.com/api/v2/genai/llmTestSuites/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
useCaseId query string false Only retrieve the LLM test suites associated with this use case ID.
offset query integer false Skip the specified number of values.
limit query integer false Retrieve only the specified number of values.
sort query any false Apply this sort order to the results. Valid options are "name" and "creationDate". Prefix the attribute name with a dash to sort in descending order, e.g., sort=-creationDate.

Example responses

200 Response

{
  "count": 0,
  "data": [
    {
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "description": "string",
      "id": "string",
      "llmTestConfigurationIds": [
        "string"
      ],
      "name": "string",
      "useCaseId": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses

Status Meaning Description Schema
200 OK Successful Response ListLLMTestSuitesResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/llmTestSuites/

Create a new LLM test suite.

Code samples

# You can also use wget
curl -X POST https://app.datarobot.com/api/v2/genai/llmTestSuites/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{CreateLLMTestSuiteRequest}'

Body parameter

{
  "description": "",
  "llmTestConfigurationIds": [],
  "name": "string",
  "useCaseId": "string"
}

Parameters

Name In Type Required Description
body body CreateLLMTestSuiteRequest true none

Example responses

201 Response

{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "description": "string",
  "id": "string",
  "llmTestConfigurationIds": [
    "string"
  ],
  "name": "string",
  "useCaseId": "string"
}

Responses

Status Meaning Description Schema
201 Created Successful Response LLMTestSuiteResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/llmTestSuites/{llmTestSuiteId}/

Delete an existing LLM test suite.

Code samples

# You can also use wget
curl -X DELETE https://app.datarobot.com/api/v2/genai/llmTestSuites/{llmTestSuiteId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
llmTestSuiteId path string true The ID of the LLM test suite to delete.

Example responses

422 Response

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses

Status Meaning Description Schema
204 No Content Successful Response None
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/llmTestSuites/{llmTestSuiteId}/

Retrieve an existing LLM test suite.

Code samples

# You can also use wget
curl -X GET https://app.datarobot.com/api/v2/genai/llmTestSuites/{llmTestSuiteId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name In Type Required Description
llmTestSuiteId path string true The ID of the LLM test suite to retrieve.

Example responses

200 Response

{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "description": "string",
  "id": "string",
  "llmTestConfigurationIds": [
    "string"
  ],
  "name": "string",
  "useCaseId": "string"
}

Responses

Status Meaning Description Schema
200 OK Successful Response LLMTestSuiteResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/llmTestSuites/{llmTestSuiteId}/

Edit an existing LLM test suite.

Code samples

# You can also use wget
curl -X PATCH https://app.datarobot.com/api/v2/genai/llmTestSuites/{llmTestSuiteId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{undefined}'

Body parameter

{
  "description": "string",
  "llmTestConfigurationIds": [
    "string"
  ],
  "name": "string"
}

Parameters

Name In Type Required Description
llmTestSuiteId path string true The ID of the LLM test suite to edit.
body body EditLLMTestSuiteRequest true none

Example responses

200 Response

{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "description": "string",
  "id": "string",
  "llmTestConfigurationIds": [
    "string"
  ],
  "name": "string",
  "useCaseId": "string"
}

Responses

Status Meaning Description Schema
200 OK Successful Response LLMTestSuiteResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas

CreateLLMTestSuiteRequest

{
  "description": "",
  "llmTestConfigurationIds": [],
  "name": "string",
  "useCaseId": "string"
}

CreateLLMTestSuiteRequest

Properties

Name Type Required Restrictions Description
description string false maxLength: 5000
 The description of the LLM test suite.
llmTestConfigurationIds [string] false maxItems: 100
 The IDs of the LLM test configurations in the LLM test suite.
name string true maxLength: 5000
minLength: 1
minLength: 1
 The name of the LLM test suite.
useCaseId string true The ID of the use case to associate with the LLM test suite.

EditLLMTestSuiteRequest

{
  "description": "string",
  "llmTestConfigurationIds": [
    "string"
  ],
  "name": "string"
}

EditLLMTestSuiteRequest

Properties

Name Type Required Restrictions Description
description string¦null false maxLength: 5000
 The description of the LLM test suite.
llmTestConfigurationIds [string]¦null false maxItems: 100
 The IDs of the LLM test configurations in the LLM test suite.
name string¦null false maxLength: 5000
minLength: 1
minLength: 1
 The name of the LLM test suite.

HTTPValidationErrorResponse

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

HTTPValidationErrorResponse

Properties

Name Type Required Restrictions Description
detail [ValidationError] false none

LLMTestSuiteResponse

{
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "description": "string",
  "id": "string",
  "llmTestConfigurationIds": [
    "string"
  ],
  "name": "string",
  "useCaseId": "string"
}

LLMTestSuiteResponse

Properties

Name Type Required Restrictions Description
creationDate string(date-time) true The creation date of the chat (ISO 8601 formatted).
creationUserId string true The ID of the user that created the chat.
description string true The description of the LLM test suite.
id string true The ID of the LLM test suite.
llmTestConfigurationIds [string] true The IDs of the LLM test configurations in this LLM test suite.
name string true The name of the LLM test suite.
useCaseId string true The ID of the use case associated with the LLM test suite.

ListLLMTestSuiteSortQueryParam

"name"

ListLLMTestSuiteSortQueryParam

Properties

Name Type Required Restrictions Description
ListLLMTestSuiteSortQueryParam string false Sort order values for listing chats.

Enumerated Values

Property Value
ListLLMTestSuiteSortQueryParam [name, -name, creationDate, -creationDate]

ListLLMTestSuitesResponse

{
  "count": 0,
  "data": [
    {
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "description": "string",
      "id": "string",
      "llmTestConfigurationIds": [
        "string"
      ],
      "name": "string",
      "useCaseId": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListLLMTestSuitesResponse

Properties

Name Type Required Restrictions Description
count integer true The number of records on this page.
data [LLMTestSuiteResponse] true The list of records.
next string¦null true The URL to the next page, or null if there is no such page.
previous string¦null true The URL to the previous page, or null if there is no such page.
totalCount integer true The total number of records.

ValidationError

{
  "loc": [
    "string"
  ],
  "msg": "string",
  "type": "string"
}

ValidationError

Properties

Name Type Required Restrictions Description
loc [anyOf] true none

anyOf

Name Type Required Restrictions Description
» anonymous string false none

or

Name Type Required Restrictions Description
» anonymous integer false none

continued

Name Type Required Restrictions Description
msg string true none
type string true none
Updated January 24, 2025