NIM Containers – GPU Support Matrix
NIM Containers – GPU Support Matrix¶
See the Generic configuration guide.
This table maps each NVIDIA NIM container image to the set of GPU hardware profiles that have been validated by NVIDIA for that exact model version.
- The smallest-count, lowest-tier device in each row (e.g., 1xT4 for PaddleOCR) is the least hardware you can run the container on without exceeding memory limits.
- The "2x", "4x", "8x" prefixes indicate how many identical GPUs must be present in the node.
- Any GPU shown in a row can host that model, letting you match performance, memory, or cost targets with the hardware you already own.
Consult the first (smallest) configuration to see the minimum requirement, and pick any other listed configuration when you need higher batch sizes, faster latency, or have newer GPUs available.
| Name | GPU Requirements |
|---|---|
| nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 |
| nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 1xH100, 2xH100 |
| nvcr.io/nim/black-forest-labs/flux.1-dev:1.0.1 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-llama-8b:1.5.2 | 1xA100, 2xA10G, 1xH100, 4xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH20, 1xH200, 1xL20, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH100, 1xH20, 1xH200, 2xH200, 1xL20, 2xL20 2xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-7b:1.1.0 | 1xA10G, 1xL4OS |
| nvcr.io/nim/defog/llama-3-sqlcoder-8b:1.2.3 | 1xA10G, 2xA10G, 4xA10G, 1xH100, 2xH100, 1xL40S 2xL40S |
| nvcr.io/nim/google/gemma-2-2b-instruct:1.4.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/google/gemma-2-9b-it:1.4.0 | 1xA100, 1xA10G, 1xH100 |
| nvcr.io/nim/ipd/proteinmpnn:1 | 1xA10G, 1xL40S |
| nvcr.io/nim/ipd/rfdiffusion:2 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/meta/codellama-13b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100 |
| nvcr.io/nim/meta/codellama-34b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100 4xL40S |
| nvcr.io/nim/meta/codellama-70b-instruct:1.2.2 | 4xA100, 8xA100, 8xA10G, 4xH100, 8xH100 |
| nvcr.io/nim/meta/llama-2-13b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S |
| nvcr.io/nim/meta/llama-2-70b-chat:1.0.3 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 4xL40S |
| nvcr.io/nim/meta/llama-2-7b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S |
| nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200 2xH200, 4xH200, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-70b-instruct:1.8.4 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200 2xH200, 4xH200, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-base:1.1.2 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100 2xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 2xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct:1.8.4 | 2xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/meta/llama-3.2-11b-vision-instruct:1.1.1 | 1xA100, 2xA100, 4xA10G, 8xA10G, 1xH100, 2xH100 1xH200, 2xH200, 2xL40S, 4xL40S |
| nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xA10G, 1xL4OS |
| nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 1xA100, 2xA10G, 1xH100, 2xL40S |
| nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200 2xH200, 4xH200, 8xL40S |
| nvcr.io/nim/meta/llama-3.3-70b-instruct:1.8.5 | 8xA100, 8xH100, 4xH200, 8xL40S |
| nvcr.io/nim/meta/llama3-70b-instruct:1.0.3 | 4xA100, 4xH100, 8xH100 |
| nvcr.io/nim/meta/llama3-8b-instruct:1.0.3 | 1xA100, 2xA100, 1xA10G, 2xA10G, 1xH100, 2xH100 1xL40S, 2xL40S |
| nvcr.io/nim/microsoft/phi-3-mini-4k-instruct:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/mistralai/mistral-7b-instruct-v0.3:1.3.0 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100 1xL40S, 2xL40S |
| nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xA100, 8xH100 |
| nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:1.3 | 2xA100, 4xA100, 8xA10G, 2xH100, 4xH100, 4xL40S |
| nvcr.io/nim/nv-mistralai/mistral-nemo-12b-instruct:1.2.2 | 1xA100, 2xA100, 8xA10G, 1xH100, 2xH100 |
| nvcr.io/nim/nv-mistralai/mistral-nemo-minitron-8b-8k-instruct:1.2.3 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100 1xL40S, 2xL40S |
| nvcr.io/nim/nvidia/cosmos-predict1-7b-text2world:1.0.0 | 1xH100 |
| nvcr.io/nim/nvidia/cosmos-predict1-7b-video2world:1.0.0 | 1xH100 |
| nvcr.io/nim/nvidia/domino-automotive-aero:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/genmol:1.0.0 | 1xA100, 1xA10G, 1xA6000, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-content-safety:1.0.0 | 1xL40S, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-topic-control:1.0.0 | 1xL40S, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-70b-instruct:1.2 | 4xA100, 8xA100, 4xH100, 8xH100 |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xA100, 2xA100, 2xA10G, 1xH100, 2xH100, 1xH200 2xH200, 1xL40S, 2xL40S, 4xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 4xB100, 4xH100, 8xH100 |
| nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:1.5.0 | 1xA100, 1xA10G, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.4 | 4xA100, 8xA100, 8xA10G, 1xH100, 2xH100, 4xH100 8xH100, 1xH200, 2xH200, 4xH200, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/molmim:1.0.0 | 1xA10G, 1xL40S |
| nvcr.io/nim/nvidia/nemoguard-jailbreak-detect:1.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-e5-v5-pb24h2:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-e5-v5:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-mistral-7b-v2:1.0.1 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-rerankqa-mistral-4b-v3:1.0.2 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-yolox-page-elements-v1:1.1.0-rtx | 1xA100, 1xA10G, 1xL40S |
| nvcr.io/nim/nvidia/nvclip:2.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 |
| nvcr.io/nim/qwen/qwen-2.5-7b-instruct:1.0.0 | 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/snowflake/arctic-embed-l:1.0.1 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/tokyotech-llm/llama-3-swallow-70b-instruct-v0.1:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S |
| nvcr.io/nim/tokyotech-llm/llama-3.1-swallow-70b-instruct-v0.1:1.3.2 | 4xA100, 4xH100, 2xH200, 8xL40S |
| nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH200, 1xH20, 1xL20, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH20, 2xH200, 1xH200, 2xL40S, 1xH100, 2xL20, 1xL20 |
| nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 1xH200, 2xH200, 4xH200, 2xH100, 4xH100, 8xH100, 2xH100, 4xH100, 8xH100, 4xA100, 8xA100, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 1xH100, 1xL40S, 2xA10G |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xH100, 2xH100, 1xH200, 2xH200, 2xA100, 1xA100, 1xL40S, 2xL40S, 4xL40S, 2xA10G |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 8xH100, 4xH100, 4xB100 |
| nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xL4OS, 1xA10G |
| nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 2xA10G, 2xL40S, 1xA100, 1xH100 |
| nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xH200, 8xH100, 4xH100, 8xA100, 4xA100, 8xH100, 4xH100, 8xA100, 4xA100, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.6 | 4xL40S, 8xL40S, 8xA10G |
| nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xH100, 8xA100 |
| nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA10G, 1xL40S, 1xA100, 1xH100 |
| nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 |
| nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 |
| nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 2xH100, 1xH100 |