# Custom model validation

> Custom model validation - If you have a custom model deployment that serves as an LLM, you need to
> validate it before using it in LLM blueprints. Validation ensures the deployment can properly handle
> LLM requests. Usedatarobot.CustomModelLLMValidationto validate your deployments.

This Markdown file sits beside the HTML page at the same path (with a `.md` suffix). It summarizes the topic and lists links for tools and LLM context.

Companion generated at `2026-04-24T16:03:56.279140+00:00` (UTC).

## Primary page

- [Custom model validation](https://docs.datarobot.com/en/docs/api/dev-learning/python/genai/custom-model-validation.html): Full documentation for this topic (HTML).

## Sections on this page

- [Start LLM validation](https://docs.datarobot.com/en/docs/api/dev-learning/python/genai/custom-model-validation.html#start-llm-validation): In-page section heading.
- [Check validation status](https://docs.datarobot.com/en/docs/api/dev-learning/python/genai/custom-model-validation.html#check-validation-status): In-page section heading.
- [Use a validated LLM in a blueprint](https://docs.datarobot.com/en/docs/api/dev-learning/python/genai/custom-model-validation.html#use-a-validated-llm-in-a-blueprint): In-page section heading.
- [Update validation settings](https://docs.datarobot.com/en/docs/api/dev-learning/python/genai/custom-model-validation.html#update-validation-settings): In-page section heading.

## Related documentation

- [Developer documentation](https://docs.datarobot.com/en/docs/api/index.html): Linked from this page.
- [Developer learning](https://docs.datarobot.com/en/docs/api/dev-learning/index.html): Linked from this page.
- [Python API client user guide](https://docs.datarobot.com/en/docs/api/dev-learning/python/index.html): Linked from this page.
- [Generative AI](https://docs.datarobot.com/en/docs/api/dev-learning/python/genai/index.html): Linked from this page.
- [datarobot.CustomModelLLMValidation](https://docs.datarobot.com/en/docs/api/reference/sdk/gen-llm-generation.html#datarobot.models.genai.custom_model_llm_validation.CustomModelLLMValidation): Linked from this page.

## Documentation content

# Validate custom model LLMs

If you have a custom model deployment that serves as an LLM, you need to validate it before using it in LLM blueprints. Validation ensures the deployment can properly handle LLM requests. Use [datarobot.CustomModelLLMValidation](https://docs.datarobot.com/en/docs/api/reference/sdk/gen-llm-generation.html#datarobot.models.genai.custom_model_llm_validation.CustomModelLLMValidation) to validate your deployments.

## Start LLM validation

Validate a deployment as a custom model LLM. Specify the following:

- deployment_id : The ID of the deployment to validate.
- name : An optional name for the validation.
- prompt_column_name : The name of the column the deployed model uses for prompt text input (required for non-chat API deployments).
- target_column_name : The name of the column the deployed model uses for prediction output (required for non-chat API deployments).
- chat_model_id : The model ID to specify when calling the chat completion API (for deployments that support the chat completion API).
- wait_for_completion : If set to True, the code will wait for the validation job to complete before returning results.

```
import datarobot as dr
deployment = dr.Deployment.get(deployment_id)
use_case = dr.UseCase.get(use_case_id)
validation = dr.genai.CustomModelLLMValidation.create(
    deployment_id=deployment.id,
    use_case=use_case.id,
    name="My Custom LLM",
    prompt_column_name="prompt",
    target_column_name="response",
    wait_for_completion=True
)
validation
```

For deployments that support the chat completion API, use `chat_model_id`:

```
validation = dr.genai.CustomModelLLMValidation.create(
    deployment_id=deployment.id,
    name="Chat API LLM",
    chat_model_id="model-id-from-deployment",
    wait_for_completion=True
)
```

## Check validation status

If you don't wait for completion, poll for status using [datarobot.CustomModelLLMValidation.get()](https://docs.datarobot.com/en/docs/api/reference/sdk/gen-llm-generation.html#datarobot.models.genai.custom_model_llm_validation.CustomModelLLMValidation.get):

```
validation = dr.genai.CustomModelLLMValidation.create(
    deployment_id=deployment.id,
    name="My Custom LLM",
    prompt_column_name="prompt",
    target_column_name="response",
    wait_for_completion=False
)
while validation.validation_status == "TESTING":
    import time
    time.sleep(5)
    validation = dr.genai.CustomModelLLMValidation.get(validation.id)

if validation.validation_status == "PASSED":
    print("Validation passed!")
    print(f"Access data: {validation.deployment_access_data}")
else:
    print(f"Validation failed: {validation.error_message}")
```

## Use a validated LLM in a blueprint

Once validation passes, use the validation ID in an LLM blueprint:

```
validation = dr.genai.CustomModelLLMValidation.get(validation_id)
use_case = dr.UseCase.get(use_case_id)
playground = dr.genai.Playground.create(
    name="Custom LLM Playground",
    use_case=use_case.id
)
blueprint = dr.genai.LLMBlueprint.create(
    playground=playground.id,
    name="Custom Model Blueprint",
    llm="custom-model",
    llm_settings={
        "system_prompt": "You are a helpful assistant.",
        "validation_id": validation.id
    }
)
blueprint
```

## Update validation settings

Modify validation parameters using [datarobot.CustomModelLLMValidation.update()](https://docs.datarobot.com/en/docs/api/reference/sdk/gen-llm-generation.html#datarobot.models.genai.custom_model_llm_validation.CustomModelLLMValidation.update):

```
validation = dr.genai.CustomModelLLMValidation.get(validation_id)
validation.update(
    name="Updated Custom LLM Name",
    prediction_timeout=60
)
```