Add a text generation NVIDIA NIM to an LLM blueprint¶

In a Use Case, you can add NVIDIA Inference Microservices (NIM) to the playground for prompting, comparison, and evaluation. A playground is a Use Case asset for creating and interacting with LLM blueprints. LLM blueprints represent the full context for what is needed to generate a response from an LLM, captured in the LLM blueprint settings. Within the playground you compare LLM blueprint responses to determine which blueprint to use in production for solving a business problem.

To add a deployed text generation NVIDIA NIM to the playground:

1. In Workbench, select a Use Case from the Use Case directory, and open or create a playground on the Playgrounds tab.

2. On the LLM blueprints tab within a playground, click Create LLM blueprint to add a new blueprint. Then, from the playground's blueprint Configuration panel, in the LLM drop-down, click Add deployed LLM:

   

3. In the Add deployed LLM dialog box, enter a deployed LLM Name, then select a DataRobot deployment in the Deployment name drop-down. Enter the Chat model ID to set the model parameter for requests from the playground to the deployed LLM, then click Validate and add.

   Chat model ID value

   The Chat model ID can be set to datarobot-deployed-llm, allowing the value to populate dynamically. To hard code the value, review the Chat model ID table below, locate the NVIDIA NIM you're adding to the playground, and copy the value from the Chat model ID column.

   

4. After you add a custom LLM and validation is successful, back in the blueprint's Configuration panel, in the LLM drop-down, click Deployed LLM, and then select the Validation ID of the custom model you added:

   

5. Configure the Vector database and Prompting settings, and click Save configuration to add the blueprint to the playground.