Agent event log¶
On a deployment's Service Health tab, under Recent Activity, you can view Management events (e.g., deployment actions) and Monitoring events (e.g., spooler channel and rate limit events).
The Monitoring events can help you quickly diagnose MLOps agent issues. The spooler channel error events can help you diagnose and fix spooler configuration issues. The rate limit enforcement events can help you identify if service health stats, data drift values, or accuracy values aren't updating because you exceeded the API request rate limit.
Enable agent event log¶
To view Monitoring events, you must provide a predictionEnvironmentID
in the agent configuration file (conf\mlops.agent.conf.yaml
) as shown below. If you haven't already installed and configured the MLOps agent, see the Installation and configuration guide.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
|
View agent activity¶
To view the agent event log, on the Service Health tab, navigate to the Recent Activity section. The most recent events appear at the top of the list.
Event information¶
Each event shows the time it occurred, a description, and an icon indicating its status:
Status icon | Description |
---|---|
Green / Passing | No action needed. |
Red / Failing | Immediate action needed. |
Gray / Informational | Details a deployment action (e.g., deployment launch has started). |
Recent activity log¶
In the Recent Activity log, you can filter the activity list and access additional information:
Element | Description | |
---|---|---|
1 | Filters | Set the Event Type filter to limit the list to Management events (e.g., deployment actions) or Monitoring events (e.g., spooler channel and rate limit events). |
2 | Events | Click an event in the log to view additional Event Details for that event. The Event Details include the Event name, a Timestamp, a Channel Name, the event Type, the associated Prediction Environment, and an event Message. |
3 | Event Details | Click the Prediction Environment name to open the Prediction Environments tab, where you can create, manage, and share prediction environments. |
Monitoring events¶
Monitoring events can help you diagnose and fix MLOps agent issues. Currently, the following events can appear in the Recent Activity log:
Event | Description |
---|---|
Monitoring Spooler Channel | Identify spooler configuration issues so you can resolve them. |
Rate limit was enforced | Identify when an operation exceeds API request rate limits, resulting in updates to service health stats, data drift calculations, or accuracy calculations stalling. This event reports how long the affected operation is suspended. Rate limits are applied per deployment, per operation. |
What are the rate limits for the deployments API?
Operation | Endpoint (POST) | Limit |
---|---|---|
Submit Metrics (Service Health) | api/v2/deployments/<id>/predictionRequests/fromJSON/ |
1M requests / hour |
Submit Prediction Results (Data Drift) | api/v2/deployments/<id>/predictionInputs/fromJSON/ |
1M requests / hour |
Submit Actuals (Accuracy) | api/v2/deployments/<id>/actuals/fromJSON/ |
40 requests / second |