Data exploration¶

On a deployment's Monitoring > Data exploration tab, you can interact with a deployment's stored data to gain insight into model or agent performance. You can also download deployment data to use in custom metric calculations. The Data exploration summary includes the following functionality, depending on the deployment type:

Configure data exploration range¶

In the deployment from which you want to export stored training data, prediction data, or actuals, click the Monitoring > Data exploration tab and configure the following settings to specify the stored training data, prediction data, or actuals you want to export:

Export deployment data¶

On the Data exploration summary page (or the Data export tab of the Data exploration summary), you can download a deployment's stored data. This can include training data, prediction data, actuals, and custom metric data. Use the exported data to compute and monitor custom business or performance metrics on the Custom metrics tab or outside of DataRobot. To export deployment data for custom metrics, verify that the deployment stores prediction data, generate data for a specified time range, and then view or download that data.

Export a deployment's production data¶

To access deployment data export for prediction data, actuals, or custom metric data, on the Data exploration summary page, locate the Production data panel. On the Production data panel, in the Generate button, click the down arrow and select one of the data generation options. The availability of the following options depends on the data stored in the deployment for the model and time range selected:

Production data appears in the table below the panels. You can identify the data type in the Exported data column.

Export a deployment's training data¶

To access deployment data export for training data, on the Data exploration summary page, locate the Training data panel and click Generate training data to generate data for the specified model and time range:

Options for interacting with the training data appear in the Training data panel. Click the down arrow to choose between Open training data and Download training data :

Review and download data¶

After the production or training data are generated, you can view or download the data. Production data appears in the table below the panels, where you can identify the data type in the Exported data column. Training data appears in the Training data panel.

Use exported deployment data for custom metrics¶

To use the exported deployment data to create your own custom metrics, you can implement a script to read from the CSV file containing the exported data and then calculate metrics using the resulting values, including columns automatically generated during the export process.

This example uses the exported prediction data to calculate and plot the change in the time_in_hospital feature over a 30-day period using the DataRobot prediction timestamp (DR_RESERVED_PREDICTION_TIMESTAMP) as the DateFrame index (or row labels). It also uses the exported training data as the plot's baseline:

import pandas as pd
feature_name = "<numeric_feature_name>"
training_df = pd.read_csv("<path_to_training_data_csv>")
baseline = training_df[feature_name].mean()
prediction_df = pd.read_csv("<path_to_prediction_data_csv>")
prediction_df["DR_RESERVED_PREDICTION_TIMESTAMP"] = pd.to_datetime(
    prediction_df["DR_RESERVED_PREDICTION_TIMESTAMP"]
)
predictions = prediction_df.set_index("DR_RESERVED_PREDICTION_TIMESTAMP")["time_in_hospital"]
ax = predictions.rolling('30D').mean().plot()
ax.axhline(y=baseline - 2, color="C1", label="training data baseline")
ax.legend()
ax.figure.savefig("feature_over_time.png")

DataRobot column reference¶

DataRobot automatically adds the following columns to the prediction data generated for export:

Explore deployment data tracing¶

On the Data exploration tab of a custom or external model deployment, click Tracing to explore traces from the model or agentic workflow. Each trace—identified by a timestamp and a trace ID—contains a visual timeline that represents actions carried out by the model or agent, and revealing the order and duration of these actions.

Traces represent the path taken by a request to a model or agentic workflow. DataRobot uses the OpenTelemetry framework for tracing. A trace follows the entire end-to-end path of a request, from origin to resolution. Each trace contains one or more spans, starting with the root span. The root span represents the entire path of the request and contains a child span for each individual step in the process. The root (or parent) span and each child span share the same Trace ID.

In the Tracing table, you can review the following fields related to each trace:

Click Filter to filter by Min span duration, Max span duration, Min trace cost, and Max trace cost. The unit for span filters is nanoseconds (ns), the chart displays spans in milliseconds (ms).

To review the spans contained in a trace, along with trace details, click a trace row in the Tracing table. The span colors correspond to a Span service, usually a deployment. Restricted span appears when you don’t have access to the deployment or service associated with the span. You can view spans in Chart format or List format.

For either view, click the Span service name to access the deployment or resource (if you have access). Additional information, dependent on the configuration of the generative AI model or agentic workflow, is available on the Info, Resources, Events, and Errors tabs. The Errors tab only appears when an error occurs in a trace.

Explore deployment data quality¶

On the Data exploration tab of a generative AI deployment, click Data quality to explore prompts and responses alongside user ratings and custom metrics, if implemented, providing insight into the quality of the generative AI model. Prompts, responses, and any available metrics are matched by association ID:

To configure the rows displayed in the data quality table, click Settings to open the Column management panel, where columns can be selected, hidden, or rearranged.

Locate specific rows in the Data quality table by searching. Click Search by and select Prompt values, Response, or Actual values. Then, click Search:

In addition, you can filter the Data quality table on a single custom metric value from one of the custom metrics created for the current deployment. To filter the table, click Filter, select a Metric, enter a Metric value, and then click Apply filters:

Click the open icon to expand the details panel. The display shows a row's full Prompt and the Response matched with the prompt by association ID. It also shows custom metric values and citations (if configured):

To export columns for external use, click Export all in selected range to export every row in the time range defined at the top of the Custom metrics, or click Export selected rows if you've selected one or more rows in the table: