Capacity & resources¶
This section describes how to size the control plane, configure the resource bundles that shape task Jobs, and set per-dispatch limits.
Control-plane sizing¶
The pipelines-api-server Deployment runs the API and the background dispatch loops (the task executor and the reconciler). It ships with sensible defaults; tune pipelines-api.replicaCount and pipelines-api.resources to fit your cluster.
The control plane submits and watches Kubernetes Jobs but does not run user code itself—task execution happens in separate per-task Job pods. For higher dispatch concurrency, increase replicaCount; each replica runs its own executor and reconciler loops, and the database prevents two replicas from picking up the same work, so extra replicas add throughput safely.
Resource bundles¶
Each task and pipeline declares a resource_bundle (for example, cpu.small) that maps to CPU/memory requests on the task's Kubernetes Job. The catalog of available bundles is rendered by the chart into a pipelines-api-resource-bundles ConfigMap, which the service loads once at startup.
Note
A dispatch that references an unknown bundle fails validation. Confirm the bundle catalog is present before enabling access—see Troubleshooting.
Confirm the catalog the service loaded:
kubectl get configmap pipelines-api-resource-bundles -n <namespace> -o jsonpath='{.data}'
Dispatch limits¶
The following limits bound what a dispatch can submit and how long draft state is retained:
| Setting | Chart value | Default | Description |
|---|---|---|---|
PIPELINES_API_MAX_FILE_SIZE |
config.pipelines.maxFileSize |
5242880 (5 MB) |
Maximum size of an uploaded pipeline source or input payload. |
PIPELINES_API_DRAFT_DISPATCH_TTL_HOURS |
config.pipelines.draftDispatchTtlHours |
168 (7 days) |
Retention for draft (unlocked) dispatch records; locked dispatches are retained indefinitely. |
Finished task Jobs and their per-Job callback Secrets are cleaned up by Kubernetes (ttlSecondsAfterFinished).