DataRobotエクスペリメントプラグイン¶

The DataRobot experimentation plugin (dr xp) exposes a local dashboard during agentic application development. The dashboard runs on port 8090 by default and organizes observability into tabs. The Traces tab is available as soon as the plugin is installed; additional tabs require feature flags and optional components.

The plugin uses the same OpenTelemetry standard that carries telemetry across the rest of the stack. Local traces follow the same tracing model as deployed agents on the DataRobot platform, so local inspection reflects what the agent actually did during development without deploying to the platform or switching to a separate UI tab.

Access the dashboard¶

Complete these steps to open the local experimentation dashboard:

1. Install the CLI—curl https://cli.datarobot.com/install | sh; verify with dr self version. See Install the CLI and authenticate.
2. Authenticate—dr auth login.
3. Install the plugin—dr plugin install xp; verify with dr plugin list. See Install the experimentation plugin.
4. Create or open a project—dr start from an Agentic Starter template; confirm DATAROBOT_USE_CASE_ID in pulumi_config.json. See Set up the App Framework project.
5. Start the agent and dashboard—Run dr run dev from the project root. To launch the dashboard separately, start the agent with task agent:dev in one terminal, then run dr xp --entity-id <USE_CASE_ID> in another. See Start the experimentation dashboard.
6. Open the GUI—browse to http://localhost:8090. See Open the GUI in a browser.
7. Confirm traces—send a test message to the agent, refresh the Traces tab. See Confirm end-to-end behavior.

For batch evaluation workflows, also install the evaluation component and set DR_EXPERIMENT_ENABLE_EVALUATION=true in .env. See Batch agent evaluation.

Choose your path¶

Both paths share the same setup: install the CLI, install the xp plugin, set up an App Framework project, and open the GUI.

機能¶

Future tabs (logs, metrics, MCP tool collections) follow the same pattern: one doc page per tab, enabled by a feature flag. Run dr xp --help for the current flag list.

Local development stack¶

Agentic Starter templates run several services during local development. The commands and ports referenced throughout this page assume this stack:

Ports 8090, 8080, and 5173 are fixed in Agentic Starter templates. The agent port (8842) is set during the dr start wizard.

クイックスタート¶

<USE_CASE_ID> below is your App Framework project's use case ID—see Prerequisites.

# Install the plugin
dr plugin install xp

# Show available options
dr xp --help

# Start with a use case (traces only)
dr xp --entity-id <USE_CASE_ID>

# Enable batch evaluation workflows in the GUI
dr xp --enable-evaluation --entity-id <USE_CASE_ID>

# Start with a deployment on a custom port
dr xp --entity-type deployment --entity-id <DEPLOYMENT_ID> --port 8091 

When dr run dev runs from an Agentic Starter project, the local tracing dashboard starts automatically on port 8090. The bundled dashboard starts with traces only unless DR_EXPERIMENT_ENABLE_EVALUATION=true is set in .env.

Access the GUI¶

The experimentation plugin serves a web UI from a local HTTP server. The CLI installs and launches the plugin; the browser connects to the server URL after startup completes.

前提条件¶

Before starting the dashboard, confirm the following:

After confirming the prerequisites, complete the following sections in order (or jump straight to a step using the links in Access the dashboard above).

Install the CLI and authenticate¶

1. Install the DataRobot CLI:

curl https://cli.datarobot.com/install | sh 

1. Verify the installation:

dr self version 

1. Authenticate with DataRobot:

dr auth login 

The plugin requires authentication. The CLI passes DATAROBOT_ENDPOINT and DATAROBOT_API_TOKEN to the plugin process at launch.

Install the experimentation plugin¶

Install the xp plugin once per machine:

dr plugin install xp 

Verify discovery:

dr plugin list 

Confirm the list includes xp with a path under ~/.config/datarobot/plugins/xp/ (Linux and macOS) or the equivalent Windows config directory.

Set up the App Framework project¶

1. Create or open an Agentic Starter project:

   dr start 
   

2. Confirm pulumi_config.json at the project root contains a use case ID:

   "DATAROBOT_USE_CASE_ID": "<USE_CASE_ID>" 
   

3. (Optional)

   dr component add https://github.com/datarobot-community/af-component-evaluation . 
   

4. From the evaluation component directory (for example evaluations/), install dependencies:

   task install 
   

See Batch agent evaluation for prerequisites and setup details.

Configure the dashboard¶

Set options with flags, environment variables, or a project-level .dr-xp.yaml file. Flag values take precedence over environment variables, then config files, then pulumi_config.json.

The following table summarizes the most common settings:

Example project-level config file:

# .dr-xp.yaml (project root)
entity_type: experiment_container
port: 8090
# enable_evaluation: true   # or set DR_EXPERIMENT_ENABLE_EVALUATION=true in .env 

For Agentic Starter projects, add evaluation support to .env:

# .env (project root) — enables Evaluation tab with dr run dev / task infra:dev
DR_EXPERIMENT_ENABLE_EVALUATION=true 

For judge-based evaluations, also set credentials in .env:

DATAROBOT_API_TOKEN=<API_TOKEN>
DATAROBOT_ENDPOINT=https://app.datarobot.com/api/v2 

Judge-free benchmarks run without judge credentials. See Batch agent evaluation.

Start the local agent¶

The dashboard visualizes traces from agent activity. Batch evaluations require a running agent endpoint.

Start the full development stack from the project root:

dr run dev 

This typically starts the agent on http://localhost:8842/v1 and may also launch the dashboard on port 8090. See Quick start for the flag/env var that enables the Evaluation tab.

To start services individually:

task agent:dev      # Agent on port 8842.
task infra:dev      # Dashboard on port 8090. 

When task dev runs, dr xp may start from both agent:dev and infra:dev. The message Already running at http://127.0.0.1:8090 is expected.

Start the experimentation dashboard¶

Choose one of the following options.

Standalone start¶

From the App Framework project root:

dr xp --entity-id <USE_CASE_ID> 

Add --enable-evaluation when the Evaluation tab is needed. See Enable batch evaluation workflows.

Infra task (Agentic Starter projects)¶

task infra:dev 

This runs dr xp --plain-output without --enable-evaluation. To enable batch evaluation workflows, set DR_EXPERIMENT_ENABLE_EVALUATION=true in the project .env.

Bundled with full dev¶

dr run dev 

Expected startup output:

Starting server...
Local Experimentation running on http://127.0.0.1:8090 for experiment_container:<USE_CASE_ID> 

If a server is already listening on the port, the CLI prints:

Already running at http://127.0.0.1:8090 — no new server started. 

The server runs in the foreground. Stop it with Ctrl+C.

Verify the server (optional)¶

Confirm the dashboard responds:

curl -s http://127.0.0.1:8090/api/info 

Example response with evaluation enabled:

{"entity_type":"experiment_container","entity_id":"<USE_CASE_ID>","features":["traces","evaluation"]} 

When evaluation is disabled, features contains "traces" only and evaluation API routes return 404.

Open the GUI in a browser¶

Open one of the following URLs:

The page title is Experimentation UI. Available tabs depend on enabled features:

Confirm end-to-end behavior¶

1. Send a test message to the agent (for example at http://localhost:5173).
2. Refresh the dashboard. A new trace appears in the Traces tab. See Local tracing.
3. (Optional) See Batch agent evaluation.

GUI access troubleshooting¶

Configuration reference¶

Use these options when non-default entity types, ports, or environment-based configuration is required.

Command options¶

dr xp --entity-type [TYPE] --entity-id [ID] --port [PORT] [--enable-evaluation] [--plain-output] 

The server runs in the foreground and logs to the terminal. Stop it with Ctrl+C. For a step-by-step walkthrough, see Access the GUI.

• --entity-idは必須です（存在する場合、pulumi_config.json内のDATAROBOT_USE_CASE_IDにフォールバックします）。
• --entity-type defaults to experiment_container (use case). Also accepts the alias use_case.
• --enable-evaluation enables batch evaluation workflows in the GUI when the evaluation component is present in the project. Without this flag, the dashboard exposes traces only.
• --plain-output streams logs directly without the persistent TUI panel (used by task infra:dev).
• 値の優先順位：フラグ > 環境変数 > 設定ファイル（.dr-xp.yaml） > pulumi_config.json > デフォルト。

Run dr xp --help for the full flag list, including optional feature gates (--enable-logs, --enable-metrics, --enable-mcp-inspector, and others).

環境変数¶

必要に応じて、これらの変数を.envファイルに追加します。

• DR_EXPERIMENT_ENTITY_TYPE (default: experiment_container)
• DR_EXPERIMENT_ENTITY_ID
• DR_EXPERIMENT_PORT（デフォルト：8090）
• DR_EXPERIMENT_ENABLE_EVALUATION (set to true for evaluation workflows)

サポートされているエンティティタイプ¶

See also¶

• Local tracing—inspect OpenTelemetry spans during development.
• Batch agent evaluation—score agent responses on a fixed dataset.
• Implement tracing—add custom spans and attributes.
• DataRobot CLI getting started—install and configure the CLI.
• af-component-evaluation repository—evaluation component source and reference docs.