# DataRobot experimentation plugin

> DataRobot experimentation plugin - Install and use the DataRobot experimentation CLI plugin to open
> a local dashboard for agent development, tracing, and batch evaluation.

This Markdown file sits beside the HTML page at the same path (with a `.md` suffix). It summarizes the topic and lists links for tools and LLM context.

Companion generated at `2026-08-30T06:11:45.597464+00:00` (UTC).

## Primary page

- [DataRobot experimentation plugin](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md): Full documentation for this topic (Markdown sidecar).

## Sections on this page

- [Access the dashboard](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#access-the-dashboard): In-page section heading.
- [Choose your path](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#choose-your-path): In-page section heading.
- [Capabilities](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#capabilities): In-page section heading.
- [Local development stack](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#local-development-stack): In-page section heading.
- [Quick start](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#quick-start): In-page section heading.
- [Access the GUI](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#access-the-gui): In-page section heading.
- [Prerequisites](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-prerequisites): In-page section heading.
- [Install the CLI and authenticate](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-install-cli): In-page section heading.
- [Install the experimentation plugin](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-install-plugin): In-page section heading.
- [Set up the App Framework project](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-setup-project): In-page section heading.
- [Configure the dashboard](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-configure): In-page section heading.
- [Start the local agent](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-start-agent): In-page section heading.
- [Start the experimentation dashboard](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-start-dashboard): In-page section heading.
- [Standalone start](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-start-standalone): In-page section heading.
- [Infra task (Agentic Starter projects)](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-start-infra-task): In-page section heading.
- [Bundled with full dev](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-start-full-dev): In-page section heading.
- [Verify the server(optional)](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-verify): In-page section heading.
- [Open the GUI in a browser](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-open-browser): In-page section heading.
- [Confirm end-to-end behavior](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-step-confirm): In-page section heading.
- [GUI access troubleshooting](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-access-troubleshooting): In-page section heading.
- [Configuration reference](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#configuration-reference): In-page section heading.
- [Command options](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#usage): In-page section heading.
- [Environment variables](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#environment-variable-fallbacks): In-page section heading.
- [Supported entity types](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#supported-entity-types): In-page section heading.
- [See also](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#see-also): In-page section heading.

## Related documentation

- [Agentic AI](https://docs.datarobot.com/en/docs/agentic-ai/index.html.md): Linked from this page.
- [CLI](https://docs.datarobot.com/en/docs/agentic-ai/cli/index.html.md): Linked from this page.
- [Batch agent evaluation](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-evaluation.html.md): Linked from this page.
- [Local tracing](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-tracing.html.md): Linked from this page.
- [dr run dev](https://docs.datarobot.com/en/docs/agentic-ai/cli/commands/run.html.md): Linked from this page.
- [getting started guide](https://docs.datarobot.com/en/docs/agentic-ai/cli/getting-started.html.md#installation): Linked from this page.
- [dr start](https://docs.datarobot.com/en/docs/agentic-ai/cli/commands/start.html.md): Linked from this page.
- [Implement tracing](https://docs.datarobot.com/en/docs/agentic-ai/agentic-develop/agentic-tracing-code.html.md): Linked from this page.

## Documentation content

The DataRobot experimentation plugin ( `dr xp`) exposes a local dashboard during agentic application development. The dashboard runs on port `8090` by default and organizes observability into tabs. The Traces tab is available as soon as the plugin is installed; additional tabs require feature flags and optional components.

The plugin uses the same OpenTelemetry standard that carries telemetry across the rest of the stack. Local traces follow the same tracing model as deployed agents on the DataRobot platform, so local inspection reflects what the agent actually did during development without deploying to the platform or switching to a separate UI tab.

## Access the dashboard

Complete these steps to open the local experimentation dashboard:

1. Install the CLI— curl https://cli.datarobot.com/install | sh ; verify with dr self version . See Install the CLI and authenticate .
2. Authenticate— dr auth login .
3. Install the plugin— dr plugin install xp ; verify with dr plugin list . See Install the experimentation plugin .
4. Create or open a project— dr start from an Agentic Starter template; confirm DATAROBOT_USE_CASE_ID in pulumi_config.json . See Set up the App Framework project .
5. Start the agent and dashboard—Run dr run dev from the project root. To launch the dashboard separately, start the agent with task agent:dev in one terminal, then run dr xp --entity-id <USE_CASE_ID> in another. See Start the experimentation dashboard .
6. Open the GUI—browse to http://localhost:8090 . See Open the GUI in a browser .
7. Confirm traces—send a test message to the agent, refresh the Traces tab. See Confirm end-to-end behavior .

For batch evaluation workflows, also install the [evaluation component](https://github.com/datarobot-community/af-component-evaluation) and set `DR_EXPERIMENT_ENABLE_EVALUATION=true` in `.env`. See [Batch agent evaluation](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-evaluation.html.md).

## Choose your path

| Goal | Start here |
| --- | --- |
| Inspect agent traces during local development | Local tracing—Traces tab (default). |
| Score agent responses on a fixed benchmark dataset | Batch agent evaluation—Evaluation tab (preview; requires --enable-evaluation). |

Both paths share the same setup: install the CLI, install the `xp` plugin, set up an App Framework project, and open the GUI.

## Capabilities

| Tab / capability | Doc | Requires |
| --- | --- | --- |
| Traces | Local tracing | Plugin installed; agent running. |
| Evaluation | Batch agent evaluation | --enable-evaluation, evaluation component installed (preview). |

Future tabs (logs, metrics, MCP tool collections) follow the same pattern: one doc page per tab, enabled by a feature flag. Run `dr xp --help` for the current flag list.

## Local development stack

Agentic Starter templates run several services during local development. The commands and ports referenced throughout this page assume this stack:

| Port | Service |
| --- | --- |
| 8090 | Experimentation dashboard (dr xp GUI). |
| 8842 | Local agent OpenAI-compatible endpoint. |
| 5173 | Frontend dev server. |
| 8080 | FastAPI backend. |
| 9000 | MCP server (configurable via MCP_SERVER_PORT). |

Ports `8090`, `8080`, and `5173` are fixed in Agentic Starter templates. The agent port ( `8842`) is set during the `dr start` wizard.

## Quick start

`<USE_CASE_ID>` below is your App Framework project's use case ID—see [Prerequisites](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#gui-prerequisites).

```
# Install the plugin
dr plugin install xp

# Show available options
dr xp --help

# Start with a use case (traces only)
dr xp --entity-id <USE_CASE_ID>

# Enable batch evaluation workflows in the GUI
dr xp --enable-evaluation --entity-id <USE_CASE_ID>

# Start with a deployment on a custom port
dr xp --entity-type deployment --entity-id <DEPLOYMENT_ID> --port 8091
```

> [!NOTE] Important
> Batch evaluation workflows in the GUI are off by default. Pass `--enable-evaluation` (or set `DR_EXPERIMENT_ENABLE_EVALUATION=true`) when starting `dr xp` to expose the Evaluation tab and evaluation API routes. Traces work without this flag. See [Enable batch evaluation workflows](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-evaluation.html.md#enable-batch-evaluation).

When [dr run dev](https://docs.datarobot.com/en/docs/agentic-ai/cli/commands/run.html.md) runs from an Agentic Starter project, the local tracing dashboard starts automatically on port `8090`. The bundled dashboard starts with traces only unless `DR_EXPERIMENT_ENABLE_EVALUATION=true` is set in `.env`.

## Access the GUI

The experimentation plugin serves a web UI from a local HTTP server. The CLI installs and launches the plugin; the browser connects to the server URL after startup completes.

### Prerequisites

Before starting the dashboard, confirm the following:

| Requirement | Notes |
| --- | --- |
| DataRobot CLI | Install with the getting started guide; verify with dr self version. |
| CLI authentication | Run dr auth login or configure credentials in ~/.config/datarobot/drconfig.yaml. |
| Experimentation plugin | Install with dr plugin install xp; verify with dr plugin list. |
| App Framework project | Required for entity context; create with dr start or clone an Agentic Starter template. |
| Use case ID | Stored in pulumi_config.json as DATAROBOT_USE_CASE_ID, or pass --entity-id explicitly. |

After confirming the prerequisites, complete the following sections in order (or jump straight to a step using the links in [Access the dashboard](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#access-the-dashboard) above).

### Install the CLI and authenticate

1. Install the DataRobot CLI:

```
curl https://cli.datarobot.com/install | sh
```

1. Verify the installation:

```
dr self version
```

1. Authenticate with DataRobot:

```
dr auth login
```

The plugin requires authentication. The CLI passes `DATAROBOT_ENDPOINT` and `DATAROBOT_API_TOKEN` to the plugin process at launch.

### Install the experimentation plugin

Install the `xp` plugin once per machine:

```
dr plugin install xp
```

Verify discovery:

```
dr plugin list
```

Confirm the list includes `xp` with a path under `~/.config/datarobot/plugins/xp/` (Linux and macOS) or the equivalent Windows config directory.

### Set up the App Framework project

1. Create or open an Agentic Starter project: drstart
2. Confirmpulumi_config.jsonat the project root contains a use case ID: "DATAROBOT_USE_CASE_ID":"<USE_CASE_ID>"
3. (Optional)To run batch evaluation workflows in the GUI, add theevaluation component: drcomponentaddhttps://github.com/datarobot-community/af-component-evaluation.
4. From the evaluation component directory (for exampleevaluations/), install dependencies: taskinstall

See [Batch agent evaluation](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-evaluation.html.md) for prerequisites and setup details.

### Configure the dashboard

Set options with flags, environment variables, or a project-level `.dr-xp.yaml` file. Flag values take precedence over environment variables, then config files, then `pulumi_config.json`.

The following table summarizes the most common settings:

| Setting | Flag | Environment variable | Default |
| --- | --- | --- | --- |
| Port | --port | DR_EXPERIMENT_PORT | 8090 |
| Entity type | --entity-type | DR_EXPERIMENT_ENTITY_TYPE | experiment_container (use case) |
| Entity ID | --entity-id | DR_EXPERIMENT_ENTITY_ID | DATAROBOT_USE_CASE_ID from pulumi_config.json |
| Evaluation workflows | --enable-evaluation | DR_EXPERIMENT_ENABLE_EVALUATION=true | Off; pass the flag to enable batch evaluation workflows. |
| Plain terminal output | --plain-output | — | Off (TUI panel shown by default). |

Example project-level config file:

```
# .dr-xp.yaml (project root)
entity_type: experiment_container
port: 8090
# enable_evaluation: true   # or set DR_EXPERIMENT_ENABLE_EVALUATION=true in .env
```

For Agentic Starter projects, add evaluation support to `.env`:

```
# .env (project root) — enables Evaluation tab with dr run dev / task infra:dev
DR_EXPERIMENT_ENABLE_EVALUATION=true
```

For judge-based evaluations, also set credentials in `.env`:

```
DATAROBOT_API_TOKEN=<API_TOKEN>
DATAROBOT_ENDPOINT=https://app.datarobot.com/api/v2
```

Judge-free benchmarks run without judge credentials. See [Batch agent evaluation](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-evaluation.html.md).

### Start the local agent

The dashboard visualizes traces from agent activity. Batch evaluations require a running agent endpoint.

Start the full development stack from the project root:

```
dr run dev
```

This typically starts the agent on `http://localhost:8842/v1` and may also launch the dashboard on port `8090`. See [Quick start](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#quick-start) for the flag/env var that enables the Evaluation tab.

To start services individually:

```
task agent:dev      # Agent on port 8842.
task infra:dev      # Dashboard on port 8090.
```

When `task dev` runs, `dr xp` may start from both `agent:dev` and `infra:dev`. The message `Already running at http://127.0.0.1:8090` is expected.

### Start the experimentation dashboard

Choose one of the following options.

#### Standalone start

From the App Framework project root:

```
dr xp --entity-id <USE_CASE_ID>
```

Add `--enable-evaluation` when the Evaluation tab is needed. See [Enable batch evaluation workflows](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/experimentation-evaluation.html.md#enable-batch-evaluation).

#### Infra task (Agentic Starter projects)

```
task infra:dev
```

This runs `dr xp --plain-output` without `--enable-evaluation`. To enable batch evaluation workflows, set `DR_EXPERIMENT_ENABLE_EVALUATION=true` in the project `.env`.

#### Bundled with full dev

```
dr run dev
```

Expected startup output:

```
Starting server...
Local Experimentation running on http://127.0.0.1:8090 for experiment_container:<USE_CASE_ID>
```

If a server is already listening on the port, the CLI prints:

```
Already running at http://127.0.0.1:8090 — no new server started.
```

The server runs in the foreground. Stop it with Ctrl + C.

### Verify the server (optional)

Confirm the dashboard responds:

```
curl -s http://127.0.0.1:8090/api/info
```

Example response with evaluation enabled:

```
{"entity_type":"experiment_container","entity_id":"<USE_CASE_ID>","features":["traces","evaluation"]}
```

When evaluation is disabled, `features` contains `"traces"` only and evaluation API routes return `404`.

### Open the GUI in a browser

Open one of the following URLs:

| Environment | URL |
| --- | --- |
| Local machine | http://localhost:8090 or http://127.0.0.1:8090. |
| DataRobot codespace or notebook | Exposed-port URL for port 8090 (shown when dr run dev or task dev completes). |

The page title is Experimentation UI. Available tabs depend on enabled features:

| Tab | Doc | Requires |
| --- | --- | --- |
| Traces | Local tracing | Plugin installed and agent running. |
| Evaluation | Batch agent evaluation | --enable-evaluation (or DR_EXPERIMENT_ENABLE_EVALUATION=true) and the evaluation component installed. |

### Confirm end-to-end behavior

1. Send a test message to the agent (for example at http://localhost:5173 ).
2. Refresh the dashboard. A new trace appears in the Traces tab. See Local tracing .
3. (Optional) When evaluation is enabled, open the Evaluation tab, select a pipeline, dataset, and agent endpoint ( http://localhost:8842/v1 ), then start a run. See Batch agent evaluation .

## GUI access troubleshooting

| Symptom | Fix |
| --- | --- |
| dr: command not found | Reinstall the CLI or add the binary directory to PATH; see Getting started. |
| Plugin not found | Run dr plugin install xp. |
| Missing entity ID | Run dr start to create pulumi_config.json, or pass --entity-id <USE_CASE_ID>. |
| Port 8090 already in use | List the process with lsof -i :8090 (macOS/Linux), note the PID, and run kill <PID>. On Windows, run netstat -ano \| findstr :8090, then taskkill /PID <pid> /F. Alternatively, start on another port with --port 8091. |
| No Evaluation tab | See Evaluation troubleshooting. |
| No traces in the dashboard | See Tracing troubleshooting. |
| Evaluation APIs return 404 | The running instance was started without --enable-evaluation; restart with the flag enabled. |

## Configuration reference

Use these options when non-default entity types, ports, or environment-based configuration is required.

### Command options

```
dr xp --entity-type [TYPE] --entity-id [ID] --port [PORT] [--enable-evaluation] [--plain-output]
```

The server runs in the foreground and logs to the terminal. Stop it with Ctrl + C. For a step-by-step walkthrough, see [Access the GUI](https://docs.datarobot.com/en/docs/agentic-ai/cli/local-experimentation/index.html.md#access-the-gui).

- --entity-id is required (falls back to DATAROBOT_USE_CASE_ID in pulumi_config.json if present).
- --entity-type defaults to experiment_container (use case). Also accepts the alias use_case .
- --enable-evaluation enables batch evaluation workflows in the GUI when the evaluation component is present in the project. Without this flag, the dashboard exposes traces only.
- --plain-output streams logs directly without the persistent TUI panel (used by task infra:dev ).
- Value precedence: flag > environment variable > config file ( .dr-xp.yaml ) > pulumi_config.json > default.

Run `dr xp --help` for the full flag list, including optional feature gates ( `--enable-logs`, `--enable-metrics`, `--enable-mcp-inspector`, and others).

### Environment variables

Add these variables to the `.env` file as needed.

- DR_EXPERIMENT_ENTITY_TYPE (default: experiment_container )
- DR_EXPERIMENT_ENTITY_ID
- DR_EXPERIMENT_PORT (default: 8090 )
- DR_EXPERIMENT_ENABLE_EVALUATION (set to true for evaluation workflows)

### Supported entity types

| Entity type | Description |
| --- | --- |
| experiment_container (alias: use_case) | Use case experiment container (default). |
| deployment | Model deployment. |
| custom_application | Custom application. |
| workload | Workload. |

## See also

- Local tracing —inspect OpenTelemetry spans during development.
- Batch agent evaluation —score agent responses on a fixed dataset.
- Implement tracing —add custom spans and attributes.
- DataRobot CLI getting started —install and configure the CLI.
- af-component-evaluation repository —evaluation component source and reference docs.