Monitor applications¶

Consistently monitoring applications allows you to proactively detect issues, troubleshoot performance bottlenecks, and quickly respond to service disruptions, minimizing downtime and improving the overall user experience. To access monitoring information, go to the Applications page and click on the app you want to view.

From here, you can access the following monitoring options:

• Service health: Provides a dashboard that displays memory, CPU, and network usage.
• Activity log: Displays access, runtime, build, and version history logs.
• Tracing: Displays traces from the model or agentic workflow. Search and filter traces, review duration and token usage, and expand spans to view attributes, logs, input, and output.

Resource usage¶

The Service health tab allows you to monitor usage, service health, and resource consumption for individual applications. Monitoring resource consumption is essential for cost management to ensure that resources are used efficiently, helping optimize cost.

To access application monitoring capabilities, on the Applications page, click the app you want to view and select Service health. From here, you can:

Activity logs¶

DataRobot records the following activity logs for custom applications:

To access these activity logs, from the Applications page, click the app you want to view, and select Activity log.

Build and runtime logs¶

From the Build logs and Runtime logs tabs, you can browse logs that detail the history of compiling, building, and executing the custom application. This includes dependency checks, packaging, and any warnings or errors thrown.

Access logs¶

From the Access logs tab, you can monitor the history of users who have opened or operated a custom application.

You can also view access logs directly from an application source. Navigate to Registry > Application sources, locate the application source for your custom application, and expand the dropdown to view the applications built from the source. Then, click the custom application you want to view the access logs from to access a detailed view.

On the Overview tab, scroll down to the Access logs section.

The access logs detail users' visits to the application, including their email, user ID, time of visit, and their role for the application.

Tracing¶

The Tracing tab of an application displays traces from a model or agentic workflow. Each trace includes a combined span list and timeline that shows the order and duration of actions carried out by the model or agent. Use the search row to filter traces by last timestamp, status (Success or Alert), and type, including generation type (operational spans or generation-specific spans). Application tracing uses the same interface as deployment tracing.

While developing an agent locally, you can add custom OpenTelemetry spans and attributes (for example, runtime configuration values) so they appear in traces after deployment. For setup steps and code examples, see Add custom tracing to agent.

Traces represent the path taken by a request to a model or agentic workflow. DataRobot uses the OpenTelemetry framework for tracing. A trace follows the entire end-to-end path of a request, from origin to resolution. Each trace contains one or more spans, starting with the root span. The root span represents the entire path of the request and contains a child span for each individual step in the process. The root (or parent) span and each child span share the same Trace ID.

To view or change your organization's retention settings, see Configure OTel retention.

Traces appear in a list by Root span name. Each entry shows the status of the trace and the timestamp. Select a trace to open the detail pane, which shows high-level metadata for the request alongside a combined span list and timeline.

Depending on the request, the list and detail pane can include the following fields:

Search and filter traces¶

Use the search row above the trace list to locate traces.

Review spans¶

Select a trace to review its spans. The span list and timeline appear together: each row shows the span name, duration, and token usage, and a timeline chart shows when the span ran relative to the rest of the trace. Expand a parent span to show its child spans. Restricted span appears when you do not have access to the deployment or service associated with the span.

In the span view, filter spans by duration, tokens, name, or status code, or isolate identical spans.

When you select a span, high-level metadata, logs, input, output, and remaining metadata are available on the same page. Formatted text in a span appears alongside attributes. Click the span service or model name to access the deployment or resource (if you have access).

When you expand a span, logs appear with the span attributes, input, and output. The results shown are a subset of the full deployment logs.

Analyze attribute distributions¶

For a span attribute, you can view how its values are distributed across all spans with the same span name and see an anomaly score for the selected span's value. Use the distribution to check whether a value, such as a latency, token count, or routing decision, is typical for that step of your agent.

To view the distribution, select a span, and on the Attributes tab, hover over an attribute and click the histogram icon. The icon is available for numeric and text attributes, but not for nested objects.

The attribute distribution opens in a modal named for the attribute:

The scoring method depends on the attribute type:

Boolean attributes (true or false) don't have a distribution or an anomaly score. If no matching spans have the attribute, No data appears; widen the time filter in the trace list and try again.

How anomaly scores are calculated¶

Scores are calculated from the same histogram shown in the chart. The population is every span that has the attribute and the same span name, in the same deployment, application, or workload, within the time interval and the retention period. The selected span is included. Because medians are estimated from histogram bins, scores are approximate.

DMAD (numeric attributes)¶

DMAD measures how far a value is from the typical value, in units of the typical spread on that side of the distribution. It is calculated as follows:

1. Find the median, \(m\), of all values in the histogram.
2. Choose the side of the distribution the value falls on. If the value \(x\) is greater than or equal to \(m\), use the values at or above \(m\); otherwise, use the values at or below \(m\).
3. Find the median of that side, \(m_{side}\). The spread is the distance between the two medians: \(d = |m_{side} - m|\).
4. Find the distance from the median in spreads, \(k = \frac{|x - m|}{d}\), and convert it into a score:

Because each side has its own spread, a long tail on one side (for example, occasional slow requests) doesn't hide unusual values on the other side. If the spread on the value's side is 0, as with near-constant attributes, no score is calculated and N/A appears.

The score depends only on how many spreads the value is from the median:

For example, if the median latency of a span is 2 s and the median of latencies above 2 s is 3 s, the spread is 1 s. A latency of 5 s is 3 spreads from the median and scores 0.75.

Relative Frequency (text attributes)¶

Relative Frequency is the number of spans with the same value divided by the total number of spans in the histogram:

Unlike DMAD, a lower Relative Frequency score indicates a more unusual value. A value that no other span has scores \(1/N\), where \(N\) is the number of spans with the attribute.

Tracing table OTel attributes¶

For Cost, Prompt, Completion, and Tools, DataRobot reads specific span attributes across all spans that belong to the trace. Other fields (such as Timestamp, Latency, and Tokens) come from trace and span metadata rather than these attributes.

Attribute keys must match exactly (including the underscore in gen_ai). Names such as genai.prompt or GenAI.prompt are not read for the Prompt and Completion columns.

Automatic instrumentation (including DataRobot agent templates) often sets gen_ai.prompt, gen_ai.completion, and sometimes tool_name. For custom or external models, frameworks differ: tool execution may not emit tool_name even when tools run (for example, some LangGraph callback flows). In that case Prompt and Completion can populate while Tools remains empty until tool_name is configured on a span that runs inside the tool—see Implement tracing.