Usage¶

After deploying a model and making predictions in production, monitoring model quality and performance over time is critical to ensure the model remains effective. This monitoring occurs on the Data drift and Accuracy tabs and requires processing large amounts of prediction data. Prediction data processing can be subject to delays or rate limiting. Track prediction processing progress on the Usage tab.

Two different versions of the Usage tab are available, depending on deployment type:

• For non-agentic and non-NIM (NVIDIA Inference Microservices) deployments, this page provides prediction tracking details.
• For agentic workflow and NIM deployments, this page provides quota usage monitoring details.

Prediction tracking chart¶

On the left side of the Monitoring > Usage tab is the Prediction tracking chart, a bar chart of the prediction processing status over the last 24 hours or 7 days, tracking the number of processed, missing association ID, and rate-limited prediction rows. Depending on the selected view (24-hour or 7-day), the histogram's bins are hour-by-hour or day-by-day.

To view additional information on the Prediction Tracking chart, hover over a column to see the time range during which the prediction data was received and the number of rows that were Processed, Rate Limited, or Missing Association ID:

Prediction and actuals processing delay¶

On the right side of the Usage tab are the processing delays for Predictions Processing (Champion) and Actuals Processing (the delay in actuals processing is for all models in the deployment):

The Usage tab recalculates the processing delays without reloading the page. You can check the Updated value to determine when the delays were last updated.

Predictions and actuals upload limits¶

From the Usage tab, you can monitor the hourly, daily, and weekly upload limits configured for your organization's deployments. View charts that visualize the number of predictions and actuals processed and tiles that display the table size limits for returned prediction results.

The Totals tile shows how many predictions and actuals have been processed relative to the configured interval limit (displayed at the bottom of the tile). Additionally, you can view the table size limit for returned prediction results. The table size limits the number of prediction rows stored in DataRobot's database for a deployment. DataRobot stores one row per prediction (or two, for binary classification deployments). For multiclass deployments, information for all classes is stored in one row. Note that the table limit does not change when you change the time interval limit (hourly, daily, weekly). Any request that exceeds the table limit will be rejected, regardless of the time.

If you reach the exact processing limit value (for example, uploading 50,000 actuals in an hour with 50,000 as the limit), and you make an additional request (uploading 10,000 more actuals), then DataRobot processes the additional request and none of the actuals are rate limited. However, DataRobot treats predictions differently because they are processed in smaller chunks. A small chunk is processed, while the remaining predictions are rate limited. For example, if you reached a 50,000 prediction limit and uploaded 50,000 more, a chunk of 1,000 predictions may be processed as part of the small chunk.

You can view the prediction limits configured for a deployment by navigating to the Settings > Service Health tab to know when you can make predictions next if you have already reached the processing limit.

Quota usage monitoring¶

On the Monitoring > Usage tab for agentic workflow and NIM (NVIDIA Inference Microservices) deployments, the Quota monitoring dashboard visualizes the historical usage of an agentic workflow or NIM deployment segmented by user or agent.

The Quota monitoring dashboard displays three key metric tiles at the top of the page:

Each metric displays the value for the selected time frame and the average per minute in green. Click the metric tile to review the corresponding chart below:

• Total requests
• Total rate limited requests
• Total token count
• Average concurrent requests

You can configure the Quota monitoring dashboard to focus the visualized statistics on specific entities and time frames. The following controls are available:

Quota monitoring charts¶

The Quota monitoring charts display an area chart showing the distribution of requests over time, rate limited requests over time, or token count over time. This chart is a stacked chart (or stacked graph), a chart stacking multiple data series on top of each other to visualize how each entity contributes to the total over time and across categories. Each chart is segmented by entity (user or agent). Each entity is represented by a different color in the chart legend.

Hover over the chart to view detailed information about the number of requests for each entity at specific time points.

Request tracing table¶

On any Quota monitoring chart, click Show tracing to view tracing data for the deployment. This tracing chart functions similarly to the tracing chart on the Data Exploration tab.

Traces represent the path taken by a request to a model or agentic workflow. DataRobot uses the OpenTelemetry framework for tracing. A trace follows the entire end-to-end path of a request, from origin to resolution. Each trace contains one or more spans, starting with the root span. The root span represents the entire path of the request and contains a child span for each individual step in the process. The root (or parent) span and each child span share the same Trace ID.

In the Tracing table, you can review the following fields related to each trace:

Click Filter to filter by Min span duration, Max span duration, Min trace cost, and Max trace cost. The unit for span filters is nanoseconds (ns), the chart displays spans in milliseconds (ms).

To review the spans contained in a trace, along with trace details, click a trace row in the Tracing table. The span colors correspond to a Span service, usually a deployment. Restricted span appears when you don’t have access to the deployment or service associated with the span. You can view spans in Chart format or List format.

For either view, click the Span service name to access the deployment or resource (if you have access). Additional information, dependent on the configuration of the generative AI model or agentic workflow, is available on the Info, Resources, Events, and Errors tabs. The Errors tab only appears when an error occurs in a trace.

Rate limited requests table¶

The Rate limited requests table provides a detailed breakdown of rate limiting by entity:

The table helps identify which entities are experiencing rate limiting and to what extent, allowing you to adjust quotas or usage patterns accordingly.