Capacity & resources¶

Capacity and scaling¶

The Workload API control plane (workload-api, covalent-cloud-server, and the LRS operator) is separate from the user workloads it schedules. The defaults below suit small-to-medium installations; increase them for higher concurrent usage.

Control-plane sizing¶

workload-api and covalent-cloud-server each run two replicas by default. workload-api also ships a HorizontalPodAutoscaler (enabled by default: minimum 2, maximum 10 replicas, scaling at 100% CPU). Under sustained load—for example, roughly 100 concurrent users—the control plane may need to scale well beyond these defaults (workload-api toward ~20 replicas at 70–80% CPU utilization). Size workload-api's replica ceiling and resources for your expected concurrency; covalent-cloud-server ships its own resource defaults, so scale its replica count only if scheduling becomes a bottleneck.

workload-api:
  component:
    app:
      replicaCount: 2
      autoscaling:
        enabled: true
        minReplicas: 2
        maxReplicas: 10
        targetCPUUtilizationPercentage: 100
      resources:
        limits:
          cpu: 2
          memory: 12Gi
        requests:
          cpu: 500m
          memory: 228Mi

covalent-cloud-server:
  replicaCount: 2

Distributed task workers¶

workload-api offloads long-running work—container builds, container replacement, and the network-path readiness gate—to distributed task workers. They are enabled by default, and the readiness gate and container replacement depend on them. Scale taskWorkers.replicaCount alongside control-plane load.

workload-api:
  component:
    taskWorkers:
      enabled: true
      replicaCount: 2

LRS operator throughput¶

The LRS operator reconciles each workload's Kubernetes resources. On large clusters, raise its reconcile concurrency and Kubernetes client rate limits so reconciliation keeps pace as the number of workloads grows.

lrs-operator:
  operator:
    replicaCount: 2
    maxConcurrentReconciles: 10
    kubernetesClientConfigQPS: 150
    kubernetesClientConfigBurst: 250

Workload quotas and limits¶

Two independent tiers of limits apply to workloads.

Per-container limits (chart values) cap the size of a single workload's container group:

workload-api:
  workloads:
    maxContainersPerGroup: 10
    maxCredentialsPerContainer: 10

• maxContainersPerGroup—maximum number of containers allowed in one container group.
• maxCredentialsPerContainer—maximum number of unique DataRobot credential IDs a single container may reference.

Per-organization limits (entitlements) are resolved per request from the organization's DataRobot entitlements (via the account information Public API), not from Helm values:

• max_concurrent_workloads—the ceiling on simultaneously running workloads, counted org-wide across all users in the organization (not per user). Creating or starting a workload beyond this limit is rejected.
• max_workload_replicas—the maximum replicas any single workload may run. A create request above the limit is rejected; a replacement request is silently capped to the limit.

An unset value means "no limit" (the quota is not enforced). Configure these limits through the organization's DataRobot entitlements.

Resource bundle configuration¶

Resource bundles define the CPU and GPU sizes available to users when creating workloads. All bundles defined in LRS_CPU_CONTAINER_SIZES and LRS_GPU_CONTAINER_SIZES are exposed to Workload API users.

Default CPU bundles ship with the platform—no configuration is required unless you want to override or extend them. GPU bundles have no defaults and must be configured explicitly to make GPU workloads available.

CPU bundles¶

Only configure LRS_CPU_CONTAINER_SIZES if the default bundles do not meet your requirements. For details on default bundles and how use_cases controls bundle availability, see Adding custom resource bundles.

core:
  config_env_vars:
    LRS_CPU_CONTAINER_SIZES: |-
      [
        {
          "id": "cpu.small",
          "name": "CPU Small",
          "description": "1 CPU | 4GB RAM",
          "cpu_count": 1,
          "memory_mb": 4096,
          "use_cases": ["customModel", "workload"]
        },
        {
          "id": "cpu.medium",
          "name": "CPU Medium",
          "description": "2 CPU | 8GB RAM",
          "cpu_count": 2,
          "memory_mb": 8192,
          "use_cases": ["customModel", "workload"]
        }
      ]

GPU bundles¶

GPU bundles have no defaults and are configured exactly as for Custom Models: define them in LRS_GPU_CONTAINER_SIZES, and map each bundle's gpu_type_label to a cluster node group through the LRS operator's nodeGroups (a JSON string of labelToNodeGroupMap + nodeGroupsByName). Rather than duplicate it here, see Custom Models Configuration for the full bundle format and node-group schema, and the GPU acceleration guide for preparing GPU nodes.

The only Workload API–specific requirement is that each GPU bundle include "workload" in its use_cases (see Adding custom resource bundles); otherwise the bundle is exposed only to Custom Models.

Bundle fields¶

The following table describes each bundle field.

Adding custom resource bundles¶

The default CPU and GPU bundles that ship with DataRobot (for example, cpu.small, cpu.medium, and the default GPU bundles) already include "workload" in their use_cases—no configuration is required to use them with the Workload API out of the box.

If you configure custom resource bundles for your installation—for example, following the GPU bundle configuration described in Custom Models Configuration—those bundles are not eligible for Workloads unless you explicitly add "workload" to each bundle's use_cases. This applies to both LRS_CPU_CONTAINER_SIZES and LRS_GPU_CONTAINER_SIZES: a bundle configured only with "use_cases": ["customModel"] does not appear in Workload API scheduling until "workload" is added to that list.

Verify bundle availability¶

After updating a bundle's configuration, confirm it's visible to the Workload API by querying the compute bundles endpoint with the workload use case:

curl -s "${DATAROBOT_ENDPOINT}/mlops/compute/bundles/?useCases=workload" \
  -H "Authorization: Bearer ${DATAROBOT_API_TOKEN}"

A bundle you've added or updated should appear in the response before you reference its id in runtime.containerGroups[].resourceBundles.