NIM GPU support matrix¶
This page maps each NIM container image to NVIDIA-validated GPU hardware for that model version, with deployment context in the Generic configuration guide.
- The smallest-count, lowest-tier device in each row (e.g., 1xT4 for PaddleOCR) is the least hardware you can run the container on without exceeding memory limits.
- The "2x", "4x", "8x" prefixes indicate how many identical GPUs must be present in the node.
- Any GPU shown in a row can host that model, letting you match performance, memory, or cost targets with the hardware you already own.
Consult the first (smallest) configuration to see the minimum requirement, and pick any other listed configuration when you need higher batch sizes, faster latency, or have newer GPUs available.
| Name | GPU Requirements |
|---|---|
| nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 |
| nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 1xH100, 2xH100 |
| nvcr.io/nim/black-forest-labs/flux.1-dev:1.0.1 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-llama-8b:1.5.2 | 1xA100, 2xA10G, 1xH100, 4xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH20, 1xH200, 1xL20, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH100, 1xH20, 1xH200, 2xH200, 1xL20, 2xL20 2xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-7b:1.1.0 | 1xA10G, 1xL4OS |
| nvcr.io/nim/defog/llama-3-sqlcoder-8b:1.2.3 | 1xA10G, 2xA10G, 4xA10G, 1xH100, 2xH100, 1xL40S 2xL40S |
| nvcr.io/nim/google/gemma-2-2b-instruct:1.4.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/google/gemma-2-9b-it:1.4.0 | 1xA100, 1xA10G, 1xH100 |
| nvcr.io/nim/ipd/proteinmpnn:1 | 1xA10G, 1xL40S |
| nvcr.io/nim/ipd/rfdiffusion:2 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/meta/codellama-13b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100 |
| nvcr.io/nim/meta/codellama-34b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100 4xL40S |
| nvcr.io/nim/meta/codellama-70b-instruct:1.2.2 | 4xA100, 8xA100, 8xA10G, 4xH100, 8xH100 |
| nvcr.io/nim/meta/llama-2-13b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S |
| nvcr.io/nim/meta/llama-2-70b-chat:1.0.3 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 4xL40S |
| nvcr.io/nim/meta/llama-2-7b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S |
| nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200 2xH200, 4xH200, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-70b-instruct:1.8.4 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200 2xH200, 4xH200, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-base:1.1.2 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100 2xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 2xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct:1.8.4 | 2xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/meta/llama-3.2-11b-vision-instruct:1.1.1 | 1xA100, 2xA100, 4xA10G, 8xA10G, 1xH100, 2xH100 1xH200, 2xH200, 2xL40S, 4xL40S |
| nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xA10G, 1xL4OS |
| nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 1xA100, 2xA10G, 1xH100, 2xL40S |
| nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200 2xH200, 4xH200, 8xL40S |
| nvcr.io/nim/meta/llama-3.3-70b-instruct:1.8.5 | 8xA100, 8xH100, 4xH200, 8xL40S |
| nvcr.io/nim/meta/llama3-70b-instruct:1.0.3 | 4xA100, 4xH100, 8xH100 |
| nvcr.io/nim/meta/llama3-8b-instruct:1.0.3 | 1xA100, 2xA100, 1xA10G, 2xA10G, 1xH100, 2xH100 1xL40S, 2xL40S |
| nvcr.io/nim/microsoft/phi-3-mini-4k-instruct:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/mistralai/mistral-7b-instruct-v0.3:1.3.0 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100 1xL40S, 2xL40S |
| nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xA100, 8xH100 |
| nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:1.3 | 2xA100, 4xA100, 8xA10G, 2xH100, 4xH100, 4xL40S |
| nvcr.io/nim/nv-mistralai/mistral-nemo-12b-instruct:1.2.2 | 1xA100, 2xA100, 8xA10G, 1xH100, 2xH100 |
| nvcr.io/nim/nv-mistralai/mistral-nemo-minitron-8b-8k-instruct:1.2.3 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100 1xL40S, 2xL40S |
| nvcr.io/nim/nvidia/cosmos-predict1-7b-text2world:1.0.0 | 1xH100 |
| nvcr.io/nim/nvidia/cosmos-predict1-7b-video2world:1.0.0 | 1xH100 |
| nvcr.io/nim/nvidia/domino-automotive-aero:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/genmol:1.0.0 | 1xA100, 1xA10G, 1xA6000, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-content-safety:1.0.0 | 1xL40S, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-topic-control:1.0.0 | 1xL40S, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-70b-instruct:1.2 | 4xA100, 8xA100, 4xH100, 8xH100 |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xA100, 2xA100, 2xA10G, 1xH100, 2xH100, 1xH200 2xH200, 1xL40S, 2xL40S, 4xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 4xB100, 4xH100, 8xH100 |
| nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:1.5.0 | 1xA100, 1xA10G, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.4 | 4xA100, 8xA100, 8xA10G, 1xH100, 2xH100, 4xH100 8xH100, 1xH200, 2xH200, 4xH200, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/molmim:1.0.0 | 1xA10G, 1xL40S |
| nvcr.io/nim/nvidia/nemoguard-jailbreak-detect:1.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-e5-v5-pb24h2:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-e5-v5:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-mistral-7b-v2:1.0.1 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-rerankqa-mistral-4b-v3:1.0.2 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-yolox-page-elements-v1:1.1.0-rtx | 1xA100, 1xA10G, 1xL40S |
| nvcr.io/nim/nvidia/nvclip:2.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 |
| nvcr.io/nim/qwen/qwen-2.5-7b-instruct:1.0.0 | 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/snowflake/arctic-embed-l:1.0.1 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/tokyotech-llm/llama-3-swallow-70b-instruct-v0.1:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S |
| nvcr.io/nim/tokyotech-llm/llama-3.1-swallow-70b-instruct-v0.1:1.3.2 | 4xA100, 4xH100, 2xH200, 8xL40S |
| nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH200, 1xH20, 1xL20, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH20, 2xH200, 1xH200, 2xL40S, 1xH100, 2xL20, 1xL20 |
| nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 1xH200, 2xH200, 4xH200, 2xH100, 4xH100, 8xH100, 2xH100, 4xH100, 8xH100, 4xA100, 8xA100, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 1xH100, 1xL40S, 2xA10G |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xH100, 2xH100, 1xH200, 2xH200, 2xA100, 1xA100, 1xL40S, 2xL40S, 4xL40S, 2xA10G |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 8xH100, 4xH100, 4xB100 |
| nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xL4OS, 1xA10G |
| nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 2xA10G, 2xL40S, 1xA100, 1xH100 |
| nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xH200, 8xH100, 4xH100, 8xA100, 4xA100, 8xH100, 4xH100, 8xA100, 4xA100, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.6 | 4xL40S, 8xL40S, 8xA10G |
| nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xH100, 8xA100 |
| nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA10G, 1xL40S, 1xA100, 1xH100 |
| nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 |
| nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 |
| nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 2xH100, 1xH100 |