NIM Containers – Resource bundles¶
Below is a quick-look bundle-sizing guide for the three managed-Kubernetes platforms. Each table shows, per cloud, how our standard bundle labels (M, L, XL, 2 XL, 3 XL, 4 XL) map to GPU nodes and the aggregate FP16 throughput and memory you can expect.
See also NIM validation on GPU bundles
EKS (AWS):
| Bundle | GPU Type | GPU Count | FP16 TFLOPS | VRAM (GB) | Total FP16 TFLOPS | Total VRAM (GB) | RAM (GB) |
|---|---|---|---|---|---|---|---|
| S | T4 | 1 | 65 | 16 | 65 | 16 | 10 |
| M | A10 | 1 | 125 | 24 | 125 | 24 | 32 |
| L | L40S | 1 | 181 | 48 | 181 | 48 | 32 |
| XL | A10 | 4 | 125 | 24 | 500 | 96 | 192 |
| XXL | L40S | 4 | 181 | 48 | 724 | 192 | 384 |
| 3XL | L40S | 8 | 181 | 48 | 1448 | 384 | 1536 |
| 4XL | A100 SXM | 8 | 312 | 80 | 2496 | 640 | 1152 |
AKS (Azure):
| Bundle | GPU Type | GPU Count | FP16 TFLOPS | VRAM (GB) | Total FP16 TFLOPS | Total VRAM (GB) | RAM (GB) |
|---|---|---|---|---|---|---|---|
| M | A10 | 1 | 125 | 24 | 125 | 24 | 440 |
| L | A10 | 2 | 125 | 24 | 250 | 48 | 880 |
| XL | A100 PCIe | 1 | 312 | 80 | 312 | 80 | 220 |
| 2XL | A100 PCIe | 2 | 312 | 80 | 624 | 160 | 440 |
| 3XL | A100 PCIe | 4 | 312 | 80 | 1248 | 320 | 900 |
| 4XL | A100 SXM | 8 | 312 | 80 | 2496 | 640 | 1900 |
OCP (Oracle):
| Bundle | GPU Type | GPU Count | FP16 TFLOPS | VRAM (GB) | Total FP16 TFLOPS | Total VRAM (GB) | RAM (GB) |
|---|---|---|---|---|---|---|---|
| XL | A100 SXM | 1 | 312 | 80 | 312 | 80 | 80 |
| 2XL | A100 SXM | 2 | 312 | 80 | 624 | 160 | 160 |
| 3XL | A100 SXM | 4 | 312 | 80 | 1248 | 320 | 320 |
| 4XL | A100 SXM | 8 | 312 | 80 | 2496 | 640 | 640 |
OKE H100 (Oracle):
| Bundle | GPU Type | GPU Count | FP16 TFLOPS | VRAM (GB) | Total FP16 TFLOPS | Total VRAM (GB) | RAM (GB) |
|---|---|---|---|---|---|---|---|
| XL | H100 SXM | 1 | 1000 | 80 | 1000 | 80 | 80 |
| 2XL | H100 SXM | 2 | 1000 | 80 | 2000 | 160 | 160 |
| 3XL | H100 SXM | 4 | 1000 | 80 | 4000 | 320 | 320 |
| 4XL | H100 SXM | 8 | 1000 | 80 | 8000 | 640 | 640 |