# Single LLM blueprint chat
URL: https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/rag-chatting.html

> Describes using single and comparison chats in the playground.

# Single LLM blueprint chat

Chatting is the activity of sending prompts and receiving a response from the LLM. A chat is a collection of chat prompts. Once you have set the configuration for your LLM, send it prompts—and follow-up prompts—from the entry box in the lower part of the panel to determine whether further refinements are needed before considering your LLM blueprint for deployment.

Chatting within the playground is a "conversation"—you can ask follow-up questions with subsequent prompts. Following is an example of asking the LLM to provide Python code for running DataRobot Autopilot:

The results of the follow-up questions are dependent on whether [context awareness](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/rag-chatting.html#context-aware-chatting) is enabled (see continuation of the example). Use the playground to test and tune prompts until you are satisfied with the system prompt and settings. Then, click Save configuration in the bottom of the right-hand panel.

## Context-aware chatting

When configuring an LLM blueprint, you set the history awareness in the [Prompting](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/build-llm-blueprints.html#set-system-prompt) tab.

There are two states of context. They control whether chat history is sent with the prompt to include relevant context for responses.

| State | Description |
| --- | --- |
| Context-aware | When sending input, previous chat history is included with the prompt. This state is the default. |
| No context | Sends each prompt as independent input, without history from the chat. |

> [!NOTE] Note
> Consider the context state and how it functions in conjunction with the selected [retriever method](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/build-llm-blueprints.html#retriever).

You can switch between one-time (no context) and context-aware within a chat. They each become independent sets of history context—going from context-aware, to no context, and back to aware clears the earlier history from the prompt. (This only happens once a new prompt is submitted.)

Context state is reported in two ways:

1. A badge, which displays to the right of the LLM blueprint name in both configuration and comparison views, reports the current context state:
2. In the configuration view, dividers show the state of the context setting:

Using the example of writing Python code to run Autopilot (above), you could then prompt to make a change to "that code." With context-aware enabled, the LLM responds knowing the code being referenced because it is "aware" of the previous conversation history:

See the [prompting reference](https://docs.datarobot.com/en/docs/reference/gen-ai-ref/prompting-reference.html) for information on crafting optimized prompts (including few-shot prompting).

## Single vs comparison chats

Chatting with a single LLM blueprint is a good way to tune before starting prompt comparisons with other LLM blueprints.[Comparison](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/rag-chatting.html#comparison-llm-blueprint-chat) lets you compare responses between LLM blueprints to help decide which to move to production.

> [!NOTE] Note
> You can only do comparison prompting with workflows that you created. To see the results of prompting another user’s LLM blueprint or agentic flow in a shared Use Case, copy the LLM blueprint or connect to the registered agentic flow. You can chat with the same settings applied. This is intentional behavior because prompting impacts chat history, which can impact the responses that are generated. However, you can provide response feedback on the creator's asset to assist development.

### Single LLM blueprint chat

When you first configure an [LLM blueprint](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/build-llm-blueprints.html), part of the creation process includes chatting. Set the configuration, and save, to activate chatting:

After seeing chat results, tune the configuration, if desired, and prompt again. Use the additional actions available within each chat result to retrieve more information and the prompt:

| Option | Description |
| --- | --- |
| View configuration | Shows the configuration used by that prompt in the Configuration panel on the right. If you haven't changed configurations while chatting, no change is apparent. Using this tool allows you to recall previous settings and restore the LLM blueprint to those settings. |
| Open tracing | Opens the tracing log, which shows all components and prompting activity used in generating LLM responses. |
| Delete prompt and response | Removes both the prompt and response from the chat history. If deleted, they are no longer considered as context for future responses. |

As you send prompts to the LLM, DataRobot maintains a record of those chats. You can either add to the context of an existing chat or start a new chat, which does not carry over any of the context from other chats in the history:

Starting a new chat allows you to have multiple independent conversation threads with a single blueprint. In this way, you can evaluate the LLM blueprint based on different types of topics, without bringing in the history of the previous prompt response, which could "pollute" the answers. While you could also do this by switching context off, submitting a prompt, and then switching it back on, starting a new chat is a simpler solution.

Click Start new chat to begin with a clean history; DataRobot will rename the chat from New chat to the words from your prompt once the prompt is submitted.

### Comparison LLM blueprint chat

Once you are satisfied, you can compare responses with other LLM blueprints from the LLM blueprints tab. If you determine that further tuning is needed after having started a comparison, you can still modify the configuration of individual LLM blueprints:

To compare LLM blueprint chats side-by-side, see the [LLM blueprint comparison](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/compare-llm.html) documentation.

## Response feedback

Use the response feedback "thumbs" to rate the prompt answer. Responses are recorded in the [Tracing](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/playground-eval-metrics.html#tracing), tab User feedback column. The response, as part of the exported feedback sent to the AI Catalog, can be used, for example, to train a predictive model.

## Citations

A citation is [a metric](https://docs.datarobot.com/en/docs/agentic-ai/playground-tools/playground-eval-metrics.html#add-and-configure-metrics) and is on by default (as are Latency, Prompt Tokens, and Response Tokens). Citations provide a list of the top reference document chunks, based on relevance to the prompt, retrieved from the vector database. Be aware that the embedding model used to create the vector database in the first place can affect the quality of the citations retrieved.

> [!NOTE] Note
> Citations only appear when the LLM blueprint being queried has an associated vector database. While citations are one of the available metrics, you do not need the assessment functionality enabled to have citations returned.

Use citations as a safety check to validate LLM responses. While they help to validate LLM responses, citations also allow you to validate proper and appropriate retrieval from the vector database—are you retrieving the chunks from your docs that you want to provide as context to the LLM? Additionally, if you enable the Faithfulness metric, which measures whether the LLM response matches the source, it relies on the citation output for its relevance.

### ROUGE scores

ROUGE scores, also known as confidence scores, calculates the distance between the response generated from an LLM blueprint and the documents retrieved from the vector database. They indicate how close the response is to the provided context. ROUGE scores are computed using the factual consistency metric approach, where a score is computed using the facts retrieved from the [vector database](https://docs.datarobot.com/en/docs/agentic-ai/vector-database/vector-dbs.html) and the generated text from the LLM blueprint. The similarity metric used is the [ROUGE-1](https://en.wikipedia.org/wiki/ROUGE_(metric%29) (Recall-Oriented Understudy for Gisting Evaluation) metric. DataRobot GenAI uses an improved version of ROUGE-1 based on insights from ["The limits of automatic summarization according to ROUGE"](https://aclanthology.org/E17-2007.pdf). The ROUGE scoring algorithm is not scaled; instead, DataRobot uses heuristic coefficients.

The ROUGE score is reported in the prompt response:

### Similarity score

The similarity score is based on the distance of the query embedding to the chunk embedding in the vector space; the larger the similarity score, the better. Do not use this score to compare results between vector databases, only to compare values for retrievals within a single vector databases. The score informs "this chunk is more similar than that chunk to the given query."

The similarity score is reported in the citation:

### Metadata filtering

Use metadata filtering to limit the citations returned by the prompt query. When configured, the LLM blueprint only returns chunks that include the specified metadata column-value pair. You can add a filter for each metadata column, as needed. Each metadata column can be paired with a single value.

> [!NOTE] Note
> Vector databases created before the introduction of metadata filtering do not support this feature. To use filtering with them, create a version from the original and configure the LLM blueprint to use the new vector database instead.

To create a metadata filter, click Filter metadata below the prompt entry box.

All optional metadata column names appear in the dropdown, as well as the option `source`, which is content from the `document_file_path` column. If the vector database includes no optional metadata, only `source` is available for selection. Select a column name and enter a single value. The value must be an exact string match from the vector database (partial matches are not allowed). Use Add filter to add filters for different metadata columns

> [!NOTE] Note
> If the value entered for a source is not an exact match, while a response is returned there are no citations available. This is because there was not match on the filter.

If the LLM blueprint configuration does not include a vector database, clicking Filter metadata displays the following:

To  enable filtering, add a vector database that includes a minimum of the required `document` and `document_file_path` (shown as `source` in filtering) columns.

#### Metadata filtering example

The following example, which uses the [DataRobot documentation](https://docs.datarobot.com/en/docs/get-started/how-to/genai-walk-basic.html#prerequisites) as the vector database, compares results to the same prompt ("how do I deploy a model?") with and without a metadata filter applied.

The image on the left has no filtering. The image on the right set the value of `source` to `source: datarobot_english_documentation/datarobot_docs|en|mlops|deployment|deploy-methods|deploy-model.txt`. This is a value in the `document_file_path` column of the data source. Notice the differences, particularly in prompt tokens and ROUGE score.

When you open citations for the filtered prompt, you can see the source is only the one path:

---
