Runtime settings¶

A running Workload's configuration is mutable, but the mutations split across two endpoints with different semantics. Identity fields—name, description, importance—update in place via PATCH /workloads/{id}. Runtime and resource changes—replica count, autoscaling, CPU, memory, GPU, and bundle selection—go through PATCH /workloads/{id}/settings and queue a rolling replacement so the Workload stays available during the swap. The sections that follow cover the fields exposed at each layer and the PATCH body shapes.

Importance and metadata¶

Use PATCH /workloads/{id} to update name, description, and importance. The importance field is mutable on a running Workload:

curl -X PATCH "${DATAROBOT_ENDPOINT}/workloads/${WORKLOAD_ID}" \
  -H "Authorization: Bearer ${DATAROBOT_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"importance": "critical"}'

Runtime, autoscaling, and replicas¶

Update a Workload's runtime configuration (replica count, autoscaling policies, per-container resource allocation, resource bundles) with PATCH /workloads/{workload_id}/settings. The PATCH returns a Replacement (HTTP 202) and queues a rolling replacement onto the new runtime; see Replace and roll out.

All runtime fields are nested under runtime.containerGroups[], with each entry matched to the artifact's container group by name:

# Fixed scaling
curl -X PATCH "${DATAROBOT_ENDPOINT}/workloads/${WORKLOAD_ID}/settings" \
  -H "Authorization: Bearer ${DATAROBOT_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "runtime": {
      "containerGroups": [{
        "name": "default",
        "replicaCount": 5
      }]
    }
  }'

# Dynamic scaling
curl -X PATCH "${DATAROBOT_ENDPOINT}/workloads/${WORKLOAD_ID}/settings" \
  -H "Authorization: Bearer ${DATAROBOT_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "runtime": {
      "containerGroups": [{
        "name": "default",
        "autoscaling": {
          "enabled": true,
          "minReplicaCount": 2,
          "maxReplicaCount": 10,
          "policies": [{
            "scalingMetric": "httpRequestsConcurrency",
            "target": 20
          }]
        }
      }]
    }
  }'

You can also set autoscaling policies at Workload creation time inside the initial POST /workloads/ payload—see Tutorial: Deploy a production-ready container.

Scaling metrics¶

The autoscaling object accepts minReplicaCount and maxReplicaCount at the same level as policies—these set the replica bounds for the Workload regardless of how many policies are defined. Each AutoscalingPolicy then sets scalingMetric and target only. maxReplicaCount must be at least 1; minReplicaCount must be less than or equal to maxReplicaCount. Custom metric names (NIM 2.0 only) follow Prometheus/OpenMetrics naming conventions: up to 63 characters, matching the pattern [a-zA-Z_:][a-zA-Z0-9_:]*.

For scale-to-zero examples, see Configure autoscaling in the same call. The Console maps the same values in Configure autoscaling.

Resource allocation and bundles¶

All resource configuration is runtime-side—the artifact describes container topology only and carries no CPU, memory, or GPU fields. The runtime declares resources at two layers:

To list the resource bundle IDs available for Workloads, query GET /mlops/compute/bundles/ with useCases=workload:

curl -s "${DATAROBOT_ENDPOINT}/mlops/compute/bundles/?useCases=workload" \
  -H "Authorization: Bearer ${DATAROBOT_API_TOKEN}"

Only bundles returned by this query are valid values for runtime.containerGroups[].resourceBundles.

memory accepts either a human-readable string with one of B, KB, MB, GB (1000-based—for example, "4GB", "512MB") or a raw byte integer. Kubernetes-style binary suffixes (Mi, Gi) are not supported and return a validation error. cpu has a minimum of 0.1. The gpu field controls GPU count only; GPU model and VRAM are determined entirely by the selected resource bundle—there is no per-container GPU type or VRAM field. Each override's name follows DNS-label syntax (lowercase letters, digits, and hyphens; must start with a letter and end with a letter or digit; up to 63 characters) and must match a container declared in the artifact group.

Set both layers on the Workload's runtime. Changing them on a running Workload via PATCH /workloads/{workload_id}/settings queues a rolling replacement using the strategies in Replace and roll out.

curl -X PATCH "${DATAROBOT_ENDPOINT}/workloads/${WORKLOAD_ID}/settings" \
  -H "Authorization: Bearer ${DATAROBOT_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "runtime": {
      "containerGroups": [{
        "name": "default",
        "resourceBundles": ["cpu.medium"],
        "containers": [{
          "name": "agent",
          "resourceAllocation": {"cpu": 2, "memory": "4GB"}
        }]
      }]
    }
  }'