Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

DataRobot API resources > API reference documentation > DataRobot REST API > Custom Models (GenAI)

Custom Models (GenAI)

This page outlines the operations, endpoints, parameters, and example requests and responses for the Custom Models (GenAI).

POST /api/v2/genai/customModelVersions/

Export the specified LLM blueprint as a custom model version in Model Registry.

Code samples

# You can also use wget
curl -X POST https://app.datarobot.com/api/v2/genai/customModelVersions/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}" \
  -d '{CreateCustomModelVersionRequest}'

Body parameter

{
  "insightsConfiguration": [],
  "llmBlueprintId": "string",
  "llmTestConfigurationIds": [],
  "promptColumnName": "promptText",
  "targetColumnName": "resultText"
}

Parameters

Name In Type Required Description
body body CreateCustomModelVersionRequest true none

Example responses

202 Response

{
  "customModelId": "string"
}

Responses

Status Meaning Description Schema
202 Accepted Successful Response CreateCustomModelVersionResponse
422 Unprocessable Entity Validation Error HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas

AggregationType

"average"

AggregationType

Properties

Name Type Required Restrictions Description
AggregationType string false The type of the metric aggregation.

Enumerated Values

Property Value
AggregationType [average, percentYes, classPercentCoverage, ngramImportance, guardConditionPercentYes]

CreateCustomModelVersionRequest

{
  "insightsConfiguration": [],
  "llmBlueprintId": "string",
  "llmTestConfigurationIds": [],
  "promptColumnName": "promptText",
  "targetColumnName": "resultText"
}

CreateCustomModelVersionRequest

Properties

Name Type Required Restrictions Description
insightsConfiguration [InsightsConfigurationWithAdditionalData] false The configuration of insights to transfer to production.
llmBlueprintId string true The ID of the LLM blueprint to use for the custom model version.
llmTestConfigurationIds [string] false maxItems: 100
 The IDs of the LLM test configurations to execute.
promptColumnName string false maxLength: 5000
 The name of the column to use for prompt text input in the custom model.
targetColumnName string false maxLength: 5000
 The name of the column to use for prediction output in the custom model.

CreateCustomModelVersionResponse

{
  "customModelId": "string"
}

CreateCustomModelVersionResponse

Properties

Name Type Required Restrictions Description
customModelId string true The ID of the created custom model.

EvaluationDatasetConfigurationErrorMessage

"Unknown error during dataset retrieval"

EvaluationDatasetConfigurationErrorMessage

Properties

Name Type Required Restrictions Description
EvaluationDatasetConfigurationErrorMessage string false The error message associated with the evaluation dataset configuration.

Enumerated Values

Property Value
EvaluationDatasetConfigurationErrorMessage [Unknown error during dataset retrieval, Dataset permission denied, Dataset not found, Synthetic dataset generation not ready, Synthetic dataset failed to generate contexts, Synthetic dataset generation failed, Synthetic dataset AI Catalog upload failed, Synthetic dataset evaluation configuration failed]

ExecutionStatus

"NEW"

ExecutionStatus

Properties

Name Type Required Restrictions Description
ExecutionStatus string false Job and entity execution status.

Enumerated Values

Property Value
ExecutionStatus [NEW, RUNNING, COMPLETED, REQUIRES_USER_INPUT, SKIPPED, ERROR]

GuardCondition

{
  "comparand": 0,
  "comparator": "greaterThan"
}

GuardCondition

Properties

Name Type Required Restrictions Description
comparand any true The comparand(s) used in the guard condition.

anyOf

Name Type Required Restrictions Description
» anonymous number false none

or

Name Type Required Restrictions Description
» anonymous string false none

or

Name Type Required Restrictions Description
» anonymous boolean false none

or

Name Type Required Restrictions Description
» anonymous [string] false none

continued

Name Type Required Restrictions Description
comparator GuardConditionComparator true The comparator used in the guard condition.

GuardConditionComparator

"greaterThan"

GuardConditionComparator

Properties

Name Type Required Restrictions Description
GuardConditionComparator string false The comparator used in a guard condition.

Enumerated Values

Property Value
GuardConditionComparator [greaterThan, lessThan, equals, notEquals, is, isNot, matches, doesNotMatch, contains, doesNotContain]

GuardType

"guardModel"

GuardType

Properties

Name Type Required Restrictions Description
GuardType string false The type of the guard configuration used for moderation in production.

Enumerated Values

Property Value
GuardType [guardModel, nemo, ootb, pii, userModel]

HTTPValidationErrorResponse

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

HTTPValidationErrorResponse

Properties

Name Type Required Restrictions Description
detail [ValidationError] false none

InsightErrorResolution

"ootbMetricName"

InsightErrorResolution

Properties

Name Type Required Restrictions Description
InsightErrorResolution string false Error type linking directly to the field name that is related to the error.

Enumerated Values

Property Value
InsightErrorResolution [ootbMetricName, intervention, guardCondition, sidecarOverall, sidecarRevalidate, sidecarDeploymentId, sidecarInputColumnName, sidecarOutputColumnName, promptPipelineFiles, promptPipelineTemplateId, responsePipelineFiles, responsePipelineTemplateId]

InsightTypes

"Reference"

InsightTypes

Properties

Name Type Required Restrictions Description
InsightTypes string false The type of insight.

Enumerated Values

Property Value
InsightTypes [Reference, Quality metric, Operational metric, Evaluation deployment, Custom metric, Nemo]

InsightsConfigurationWithAdditionalData

{
  "aggregationTypes": [
    "average"
  ],
  "costConfigurationId": "string",
  "customMetricId": "string",
  "customModelLLMValidationId": "string",
  "deploymentId": "string",
  "errorMessage": "Unknown error during dataset retrieval",
  "errorResolution": [
    "ootbMetricName"
  ],
  "evaluationDatasetConfigurationId": "string",
  "executionStatus": "NEW",
  "guardConfigurationId": "string",
  "guardTemplateId": "string",
  "insightName": "string",
  "insightType": "Reference",
  "llmId": "azure-openai-gpt-3.5-turbo",
  "modelId": "string",
  "modelPackageRegisteredModelId": "string",
  "moderationConfiguration": {
    "guardConditions": [
      {
        "comparand": 0,
        "comparator": "greaterThan"
      }
    ],
    "intervention": {
      "action": "block",
      "message": "string"
    }
  },
  "nemoMetricId": "string",
  "ootbMetricId": "string",
  "ootbMetricName": "latency",
  "resultUnit": "s",
  "sidecarModelMetricMetadata": {
    "expectedResponseColumnName": "string",
    "guardType": "guardModel",
    "promptColumnName": "string",
    "responseColumnName": "string",
    "targetColumnName": "string"
  },
  "sidecarModelMetricValidationId": "string",
  "stage": "prompt_pipeline"
}

InsightsConfigurationWithAdditionalData

Properties

Name Type Required Restrictions Description
aggregationTypes [AggregationType]¦null true The aggregation types used in the insights configuration.
costConfigurationId string¦null false The ID of the cost configuration.
customMetricId string¦null false The ID of the custom metric (if using a custom metric).
customModelLLMValidationId string¦null false The ID of the custom model LLM validation if using a custom model LLM for OOTB metrics.
deploymentId string¦null false The ID of the custom model deployment associated with the insight.
errorMessage any false The error message associated with the evaluation dataset configuration or sidecar model metric validation or OOTB metric.

anyOf

Name Type Required Restrictions Description
» anonymous EvaluationDatasetConfigurationErrorMessage false The error message associated with the evaluation dataset configuration.

or

Name Type Required Restrictions Description
» anonymous SidecarModelMetricErrorMessages false Error messages for sidecar model metrics.

or

Name Type Required Restrictions Description
» anonymous NemoErrorMessages false Error messages for NeMo configuration.

or

Name Type Required Restrictions Description
» anonymous string false none

continued

Name Type Required Restrictions Description
errorResolution [InsightErrorResolution]¦null false The error type associated with the insight error status and error message as an indicator of what fields needs to be edited if any.
evaluationDatasetConfigurationId string¦null false The ID of the evaluation dataset configuration.
executionStatus ExecutionStatus¦null false The execution status of the evaluation dataset configuration.
guardConfigurationId string¦null false The ID of the guard configuration.
guardTemplateId string¦null false The ID of the guard template.
insightName string true maxLength: 5000
minLength: 1
minLength: 1
 The name of the insight.
insightType InsightTypes¦null false The type of the insight.
llmId LanguageModelTypeId¦null false The LLM ID for OOTB metrics that use LLMs.
modelId string¦null false The ID of the model associated with deploymentId.
modelPackageRegisteredModelId string¦null false The ID of the registered model package associated with deploymentId.
moderationConfiguration any false The moderation configuration associated with the insight configuration.

anyOf

Name Type Required Restrictions Description
» anonymous ModerationConfigurationWithID false Moderation Configuration associated with an insight.

or

Name Type Required Restrictions Description
» anonymous ModerationConfigurationWithoutID false Moderation Configuration associated with an insight.

continued

Name Type Required Restrictions Description
nemoMetricId string¦null false The ID of the Nemo configuration.
ootbMetricId string¦null false The ID of the ootb metric (if using an ootb metric).
ootbMetricName OOTBMetricInsightNames¦null false The OOTB metric name.
resultUnit MetricUnit¦null false The unit of measurement associated with the insight result.
sidecarModelMetricMetadata SidecarModelMetricMetadata¦null false The metadata of the sidecar model metric (if using a sidecar model metric).
sidecarModelMetricValidationId string¦null false The ID of the sidecar model metric validation (if using a sidecar model metric).
stage PipelineStage¦null false The stage (prompt or response) where insight is calculated at.

Intervention

{
  "action": "block",
  "message": "string"
}

Intervention

Properties

Name Type Required Restrictions Description
action ModerationAction true The intervention strategy.
message string true minLength: 1
minLength: 1
 The intervention message to replace the prediction when a guard condition is satisfied.

LanguageModelTypeId

"azure-openai-gpt-3.5-turbo"

LanguageModelTypeId

Properties

Name Type Required Restrictions Description
LanguageModelTypeId string false The ID that defines the type of the LLM.

Enumerated Values

Property Value
LanguageModelTypeId [azure-openai-gpt-3.5-turbo, azure-openai-gpt-3.5-turbo-16k, azure-openai-gpt-4, azure-openai-gpt-4-32k, azure-openai-gpt-4-turbo, azure-openai-gpt-4-o, amazon-titan, anthropic-claude-2, anthropic-claude-3-haiku, anthropic-claude-3-sonnet, anthropic-claude-3-opus, google-bison, google-gemini-1.5-flash, google-gemini-1.5-pro, custom-model]

MetricUnit

"s"

MetricUnit

Properties

Name Type Required Restrictions Description
MetricUnit string false The unit of measurement associated with a metric.

Enumerated Values

Property Value
MetricUnit [s, ms, %]

ModerationAction

"block"

ModerationAction

Properties

Name Type Required Restrictions Description
ModerationAction string false The moderation strategy.

Enumerated Values

Property Value
ModerationAction [block, report, reportAndBlock]

ModerationConfigurationWithID

{
  "guardConditions": [
    {
      "comparand": 0,
      "comparator": "greaterThan"
    }
  ],
  "intervention": {
    "action": "block",
    "message": "string"
  }
}

ModerationConfigurationWithID

Properties

Name Type Required Restrictions Description
guardConditions [GuardCondition] true maxItems: 1
minItems: 1
 The guard conditions associated with a metric.
intervention Intervention true The intervention specific moderation configuration.

ModerationConfigurationWithoutID

{
  "guardConditions": [
    {
      "comparand": 0,
      "comparator": "greaterThan"
    }
  ],
  "intervention": {
    "action": "block",
    "message": "string"
  }
}

ModerationConfigurationWithoutID

Properties

Name Type Required Restrictions Description
guardConditions [GuardCondition] true maxItems: 1
minItems: 1
 The guard conditions associated with a metric.
intervention Intervention true The intervention specific moderation configuration.

NemoErrorMessages

"Template ID not found."

NemoErrorMessages

Properties

Name Type Required Restrictions Description
NemoErrorMessages string false Error messages for NeMo configuration.

Enumerated Values

Property Value
NemoErrorMessages [Template ID not found., Invalid template ID., Files contents are missing.]

OOTBMetricInsightNames

"latency"

OOTBMetricInsightNames

Properties

Name Type Required Restrictions Description
OOTBMetricInsightNames string false The Out-Of-The-Box metric name that can be used in the playground.

Enumerated Values

Property Value
OOTBMetricInsightNames [latency, citations, rouge_1, faithfulness, correctness, prompt_tokens, response_tokens, document_tokens, all_tokens, jailbreak_violation, toxicity_violation, pii_violation, exact_match]

PipelineStage

"prompt_pipeline"

PipelineStage

Properties

Name Type Required Restrictions Description
PipelineStage string false Enum that describes at which stage the metric may be calculated.

Enumerated Values

Property Value
PipelineStage [prompt_pipeline, response_pipeline]

SidecarModelMetricErrorMessages

"An unknown sidecar model metric error occurred."

SidecarModelMetricErrorMessages

Properties

Name Type Required Restrictions Description
SidecarModelMetricErrorMessages string false Error messages for sidecar model metrics.

Enumerated Values

Property Value
SidecarModelMetricErrorMessages [An unknown sidecar model metric error occurred., A sidecar model metric prediction timeout error occurred., API token was deleted or revoked or user doesn't have access to the deployment., Sidecar model metric access failed., Sidecar model metric does not exist or was deleted., Sidecar model metric has been unlinked., Sidecar model metric deployment has been deleted., No access to the sidecar model metric deployment., An API token is required but was not provided in the request., Deployment prediction server did not accept the request., Deployment prediction does not comply with the expected format., Deployment prediction does not contain the specified target column name., Only unstructured, text generation, regression, binary and multiclass deployment types are supported., Only one of responseColumnName or promptColumnName may be specified., Either responseColumnName or promptColumnName may be specified.]

SidecarModelMetricMetadata

{
  "expectedResponseColumnName": "string",
  "guardType": "guardModel",
  "promptColumnName": "string",
  "responseColumnName": "string",
  "targetColumnName": "string"
}

SidecarModelMetricMetadata

Properties

Name Type Required Restrictions Description
expectedResponseColumnName string¦null false The name of the column the custom model uses for expected response text input.
guardType GuardType¦null false The matching guard template configuration type of the sidecar model.
promptColumnName string¦null false The name of the column the custom model uses for prompt text input.
responseColumnName string¦null false The name of the column the custom model uses for response text input.
targetColumnName string¦null true The name of the column the custom model uses for prediction output.

ValidationError

{
  "loc": [
    "string"
  ],
  "msg": "string",
  "type": "string"
}

ValidationError

Properties

Name Type Required Restrictions Description
loc [anyOf] true none

anyOf

Name Type Required Restrictions Description
» anonymous string false none

or

Name Type Required Restrictions Description
» anonymous integer false none

continued

Name Type Required Restrictions Description
msg string true none
type string true none
Updated November 26, 2024