Workload API configuration¶

The Workload API lets users run containerized workloads on DataRobot infrastructure. Users supply pre-built container images; the platform handles scheduling, lifecycle management, inference routing, and observability.

Prerequisites¶

• LRS operator installed and configured in your cluster.
• datarobot-otel-collector chart enabled (see Helm chart deployments section below).

Deployed Kubernetes components¶

The following Kubernetes deployments are required for the Workload API to function:

Enable the Workload API¶

Enable Helm chart deployments¶

global.workload-api.enabled: true deploys workload-api, covalent-cloud-server, and workload-monitor. Enable the OTEL collector separately, as it is not included in the global flag:

datarobot-otel-collector:
  enabled: true

Enable feature flags¶

Set global.workload-api.enabled: true to activate all required Workload API feature flags in a single step:

Full example:

global:
  workload-api:
    enabled: true

datarobot-otel-collector:
  enabled: true

Post-deployment admin configuration¶

After a successful deployment, Workload API functionality is hidden from end users until a System Administrator explicitly grants access via the feature management console.

To grant access:

1. Log in as a System Administrator.
2. Navigate to Administrator settings > Orgs/Groups/Users > Permissions > Generative AI.
3. Locate and enable Enable Access to Workload API for Image Containers and Enable OpenTelemetry-Compliant Metrics and Logs for Deployments.
4. Set each flag to Enabled for the desired organization(s).

Once enabled, users within those organizations can access the Workload API and Deployment OTEL API via their API keys.

Additional configuration¶

Image pull secrets for private registries¶

To allow workloads to pull images from private registries, supply extra image pull secrets to be mounted on each LRS instance:

covalent-cloud-server:
  imagePullSecrets:
    - secret-name-1
    - secret-name-2

Resource bundle configuration¶

Resource bundles define the CPU and GPU sizes available to users when creating workloads. All bundles defined in LRS_CPU_CONTAINER_SIZES and LRS_GPU_CONTAINER_SIZES are exposed to Workload API users.

Default CPU bundles ship with the platform—no configuration is required unless you want to override or extend them. GPU bundles have no defaults and must be configured explicitly to make GPU workloads available.

CPU bundles¶

Only configure LRS_CPU_CONTAINER_SIZES if the default bundles do not meet your requirements. For details on default bundles and how use_cases controls bundle availability, see Workload API resource bundles.

core:
  config_env_vars:
    LRS_CPU_CONTAINER_SIZES: |-
      [
        {
          "id": "cpu.small",
          "name": "CPU Small",
          "description": "1 CPU | 4GB RAM",
          "cpu_count": 1,
          "memory_mb": 4096,
          "use_cases": ["customModel", "workload"]
        },
        {
          "id": "cpu.medium",
          "name": "CPU Medium",
          "description": "2 CPU | 8GB RAM",
          "cpu_count": 2,
          "memory_mb": 8192,
          "use_cases": ["customModel", "workload"]
        }
      ]

GPU bundles¶

GPU bundles require the node group configuration in the LRS operator to map GPU types to the cluster's node groups. See the GPU acceleration guide for node group setup.

core:
  config_env_vars:
    LRS_GPU_CONTAINER_SIZES: |-
      [
        {
          "id": "gpu.medium",
          "name": "GPU Medium",
          "description": "1 x NVIDIA A10 | 24GB VRAM | 8 CPU | 32GB RAM",
          "gpu_maker": "nvidia",
          "gpu_type_label": "a10",
          "gpu_count": 1,
          "gpu_memory_mb": 24576,
          "cpu_count": 8,
          "memory_mb": 32768,
          "use_cases": ["customModel", "workload"]
        }
      ]

lrs-operator:
  operator:
    config:
      nodeGroups:
        - gpuType: "a10"
          nodeGroupLabel: "<your-node-group-label>"

Bundle fields¶

Networking¶

The LRS operator applies a default NetworkPolicy to all workload pods in the LRS namespace. No additional configuration is required for most deployments. The defaults are:

• Deny-all ingress and egress (namespace-wide base policy).
• Egress to 0.0.0.0/0 on TCP and UDP (allows workload pods to reach external services).
• Ingress on the service port (TCP) defined in the artifact spec, from all namespaces and pods (allows traffic to reach the workload endpoint).

To add extra egress NetworkPolicies—applied to all LRS instances, including Custom Models and Custom Apps—use lrsEgressNetworkPolicies. Each entry selects pods by a label key/value within the LRS namespace and opens the specified ports for egress:

lrs-operator:
  operator:
    lrsEgressNetworkPolicies:
      - name: "allow-some-label-name-egress"
        podSelectorLabel: "some-label-name"
        podSelectorValue: "some-label-value"
        namespaceSelector: "some-namespace"
        ports:
          - protocol: "TCP"
            port: 9999

User-facing OpenTelemetry endpoint¶

The Workload API injects an OTLP endpoint into each workload pod as OTEL_EXPORTER_OTLP_ENDPOINT, allowing user workloads to emit traces, metrics, and logs to the DataRobot OTEL collector.

For instrumentation guidance, see Instrument a Workload with OpenTelemetry.

KEDA autoscaling¶

KEDA autoscaling is optional and only required to support the httpRequestConcurrency scaling metric. The Workload API and Covalent are ready to work with KEDA when it is installed. Pods deployed through the Workload API require KEDA HTTP Add-on v0.12.0 or later for header-based routing and scale-to-zero support.

Refer to the KEDA autoscaling configuration guide for installation and configuration details.

Graceful workload shutdown¶

Each workload container is configured with a graceful shutdown period (terminationGracePeriodSeconds) and a preStop hook. When a pod is terminated, the preStop hook runs first, followed by SIGTERM, giving the container time to drain connections before it is forcefully stopped.