DataRobot experimentation plugin¶

The DataRobot experimentation plugin exposes a local dashboard for OpenTelemetry spans while you develop agentic applications. The plugin integrates with the DataRobot CLI as dr xp and uses the same open standard that carries telemetry across the rest of your stack. Local traces follow the same tracing model as deployed agents on the DataRobot platform, so what you inspect locally reflects what your agent actually did during development without deploying to the platform or switching to a separate UI tab.

The plugin bridges local development and the DataRobot platform by bringing tracing and observability into your workspace. Guardrail results, metrics, and spans appear while you are still editing the code that produced them, which shortens the iteration cycle when building agentic applications.

Quick start¶

# Install the plugin
dr plugin install xp

# Show available options
dr xp --help

# Start with a Use Case (the default entity type)
dr xp --entity-id <use-case-id>

# Start with a deployment on a custom port
dr xp --entity-type deployment --entity-id <deployment-id> --port 8091

Local tracing dashboard¶

After you install the plugin and start your agent locally (for example, with dr start or task agent:dev), run dr xp to launch the local tracing dashboard. By default, the dashboard is available at http://localhost:8090. Configure the port with --port or DR_EXPERIMENT_PORT.

The dashboard lists every request your agent handled during the session. Each row corresponds to a trace. Click a trace to open a detailed breakdown of its spans, including:

• Tool invocations and their execution order.
• Duration and latency for each span.
• Token counts, when reported by instrumentation.
• Associated error logs when a span or trace fails.

This is the same class of observability you would normally access only after deployment. With the experimentation plugin, it is available on your local machine from the start of development.

Filter and search traces¶

As request volume grows, use the dashboard controls to narrow results:

Usage¶

dr xp --entity-type [TYPE] --entity-id [ID] --port [PORT]

The server runs in the foreground and logs to your terminal. Stop it with Ctrl+C.

• --entity-id is required (falls back to DATAROBOT_USE_CASE_ID in pulumi_config.json if present).
• --entity-type defaults to Use Case (use_case). Also accepts the internal name experiment_container.
• Value precedence: flag > environment variable > config file (.dr-xp.yaml) > pulumi_config.json > default.

Environment variable fallbacks¶

Add these variables to the .env file as needed.

• DR_EXPERIMENT_ENTITY_TYPE (default: use_case)
• DR_EXPERIMENT_ENTITY_ID
• DR_EXPERIMENT_PORT (default: 8090)

Supported entity types¶

Why use local tracing¶

Local tracing supports agent development in several ways:

• Faster iteration: Review traces, logs, and spans without leaving your local environment or breaking your development flow.
• Behavior verification: Validate agent behavior across multiple real requests before deployment, not only on a single demo input.
• Span-level debugging: Inspect individual spans to explain agent decisions instead of inferring them from the final response.
• Earlier issue detection: Catch errors, guardrail violations, and performance problems while changes are still local, before they reach production.

For guidance on adding custom spans and attributes that appear in local and deployed traces, see Implement tracing. After deployment, review traces in the DataRobot UI; see Tracing for custom applications.