Resource Monitor¶
Availability information
This feature is only available on the Self-Managed AI Platform.
Required permission: Enable Resource Monitor
The Resource Monitor provides visibility into DataRobot's active modeling and EDA workers across the installation, providing general information about the current state of the application and specific information about the status of components. With this service in place, you can easily track user activity on each project and know when DataRobot has available resources.
Specifically, the Resource Monitor provides the number of currently running jobs, number of allowed concurrent jobs, and number of jobs waiting for a worker. Additionally, the tool provides information on which specific users are employing system resources.
Additionally, monitoring resources over time helps to determine whether your organization has the correct number of workers to meet usage needs.
Worker terminology¶
The Resource Monitor reports on the system’s queue and workers, both overall and for individual users. See the overview for a discussion of workers; the following table describes the terminology used to describe DataRobot activity:
Term | Description |
---|---|
Jobs | The tasks DataRobot completes with workers, such as model building and certain calculations. Statistics are based on jobs. |
Modeling worker | Jobs displayed in the Worker Queue, such as model building or Feature Impact calculations. |
EDA worker | Jobs displayed in the Worker Queue, such as calculating EDA1 and other related tasks. |
In Progress (or running) | A job that has received a worker and is currently executing on the worker. |
Waiting for resources (or waiting for worker) | A job that is ready to execute, but has not yet received a worker. These jobs appear as “Waiting for worker” in the Processing section of the Worker Queue. |
Queued | A job that is in the queue but is not ready to execute. These jobs appear in the Queue section of the Worker Queue. |
Active User | A user that is the owner of at least one in-progress or waiting job. |
Modeling resources¶
The Modeling tab of the Resource Monitor reports the number of configured workers being used for modeling and related tasks.
The following table describes the fields displayed on this tab. Use the refresh option to redisplay results.
Field | Description |
---|---|
Total | Total number of workers allocated to the installation. |
In use | Number of workers currently in use across the system. |
Not in use | Number of workers not currently in use and therefore available. |
Jobs waiting | Number of queued jobs. |
Users waiting | Number of individual users with at least one job waiting. |
Demand | Number of jobs trying to run vs. the number allowed by the organization's license (capacity). To calculate, add all In progress and Waiting jobs for all active users and divide that value by the total number of available workers. |
Active now | Number of users with a job running or waiting that requires a modeling worker. This value matches the total of the In use and Jobs waiting fields. |
Worker usage by user | Graphic and breakdown of per-active-user usage. (See Interpreting "Worker" usage by user.) |
EDA resources¶
The EDA tab of the Resource Monitor reports the number of configured workers being used for EDA1 and related tasks.
The following table describes the fields displayed on this tab. Use the refresh option to redisplay results.
Field | Description |
---|---|
Jobs initialized | Number of queued jobs but are not yet scheduled to run. |
Jobs waiting | Number of jobs approved for execution and waiting for resources. |
Jobs in progress | Number of running jobs. |
Available workers | Number of workers not currently in use and therefore available. |
Total workers | Total number of workers allocated to the installation. |
Demand | Number of jobs trying to run vs. the number allowed by the organization's license (capacity). |
Users with initialized jobs | Number of individual users with at least one job initialized. |
Users with waiting jobs | Number of individual users with at least one job waiting. |
Users with running jobs | Number of individual users with at least one job running. |
Active now | Number of users with a job running or waiting that requires a modeling worker. This value matches the total of the In use and Jobs waiting fields. |
Worker usage by user | Graphic and breakdown of per-active-user usage. (See Interpreting "Worker" usage by user.) |
Interpret "Worker usage by user"¶
The Users and current activity section reports on users that are actively using the system. An active user is one that has at least one running or waiting job. The bar graph is a quick visual indicator of usage, with active users listed below it. For each user name, DataRobot displays:
- In progress: the number of in-progress jobs. The number of jobs a user can have in progress is determined by both system availability and the individual allowance.
- Waiting: the number of jobs awaiting an available worker.
- Max workers: an individual's worker allowance. This value corresponds to the maximum setting of the Workers value at the top of the Worker Queue.
Refresh the Resource Monitor¶
DataRobot refreshes the Resource Monitor display at the interval selected from the dropdown. Expand the dropdown to change the interval or click the Refresh Now button to immediately update the page.