NIM GPU support matrix¶
This page maps each NIM container image to NVIDIA-validated GPU hardware for that model version, with deployment context in the Generic configuration guide.
- The smallest-count, lowest-tier device in each row (e.g., 1xT4 for PaddleOCR) is the least hardware you can run the container on without exceeding memory limits.
- The "2x", "4x", "8x" prefixes indicate how many identical GPUs must be present in the node.
- Any GPU shown in a row can host that model, letting you match performance, memory, or cost targets with the hardware you already own.
Consult the first (smallest) configuration to see the minimum requirement, and pick any other listed configuration when you need higher batch sizes, faster latency, or have newer GPUs available.
| 名前 | GPU Requirements |
|------|--------|
| nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 |
| nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 1xH100, 2xH100 |
| nvcr.io/nim/black-forest-labs/flux.1-dev:1.0.1 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-llama-8b:1.5.2 | 1xA100, 2xA10G, 1xH100, 4xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH20, 1xH200, 1xL20, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH100, 1xH20, 1xH200, 2xH200, 1xL20, 2xL20
2xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-7b:1.1.0 | 1xA10G, 1xL4OS |
| nvcr.io/nim/defog/llama-3-sqlcoder-8b:1.2.3 | 1xA10G, 2xA10G, 4xA10G, 1xH100, 2xH100, 1xL40S
2xL40S |
| nvcr.io/nim/google/gemma-2-2b-instruct:1.4.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/google/gemma-2-9b-it:1.4.0 | 1xA100, 1xA10G, 1xH100 |
| nvcr.io/nim/ipd/proteinmpnn:1 | 1xA10G, 1xL40S |
| nvcr.io/nim/ipd/rfdiffusion:2 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/meta/codellama-13b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100 |
| nvcr.io/nim/meta/codellama-34b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100
4xL40S |
| nvcr.io/nim/meta/codellama-70b-instruct:1.2.2 | 4xA100, 8xA100, 8xA10G, 4xH100, 8xH100 |
| nvcr.io/nim/meta/llama-2-13b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S |
| nvcr.io/nim/meta/llama-2-70b-chat:1.0.3 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 4xL40S |
| nvcr.io/nim/meta/llama-2-7b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S |
| nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200
2xH200, 4xH200, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-70b-instruct:1.8.4 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200
2xH200, 4xH200, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-base:1.1.2 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100
2xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 2xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct:1.8.4 | 2xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/meta/llama-3.2-11b-vision-instruct:1.1.1 | 1xA100, 2xA100, 4xA10G, 8xA10G, 1xH100, 2xH100
1xH200, 2xH200, 2xL40S, 4xL40S |
| nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xA10G, 1xL4OS |
| nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 1xA100, 2xA10G, 1xH100, 2xL40S |
| nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200
2xH200, 4xH200, 8xL40S |
| nvcr.io/nim/meta/llama-3.3-70b-instruct:1.8.5 | 8xA100, 8xH100, 4xH200, 8xL40S |
| nvcr.io/nim/meta/llama3-70b-instruct:1.0.3 | 4xA100, 4xH100, 8xH100 |
| nvcr.io/nim/meta/llama3-8b-instruct:1.0.3 | 1xA100, 2xA100, 1xA10G, 2xA10G, 1xH100, 2xH100
1xL40S, 2xL40S |
| nvcr.io/nim/microsoft/phi-3-mini-4k-instruct:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/mistralai/mistral-7b-instruct-v0.3:1.3.0 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100
1xL40S, 2xL40S |
| nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xA100, 8xH100 |
| nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:1.3 | 2xA100, 4xA100, 8xA10G, 2xH100, 4xH100, 4xL40S |
| nvcr.io/nim/nv-mistralai/mistral-nemo-12b-instruct:1.2.2 | 1xA100, 2xA100, 8xA10G, 1xH100, 2xH100 |
| nvcr.io/nim/nv-mistralai/mistral-nemo-minitron-8b-8k-instruct:1.2.3 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100
1xL40S, 2xL40S |
| nvcr.io/nim/nvidia/cosmos-predict1-7b-text2world:1.0.0 | 1xH100 |
| nvcr.io/nim/nvidia/cosmos-predict1-7b-video2world:1.0.0 | 1xH100 |
| nvcr.io/nim/nvidia/domino-automotive-aero:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/genmol:1.0.0 | 1xA100, 1xA10G, 1xA6000, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-content-safety:1.0.0 | 1xL40S, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-topic-control:1.0.0 | 1xL40S, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-70b-instruct:1.2 | 4xA100, 8xA100, 4xH100, 8xH100 |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xA100, 2xA100, 2xA10G, 1xH100, 2xH100, 1xH200
2xH200, 1xL40S, 2xL40S, 4xL40S |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 4xB100, 4xH100, 8xH100 |
| nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:1.5.0 | 1xA100, 1xA10G, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.4 | 4xA100, 8xA100, 8xA10G, 1xH100, 2xH100, 4xH100
8xH100, 1xH200, 2xH200, 4xH200, 4xL40S, 8xL40S |
| nvcr.io/nim/nvidia/molmim:1.0.0 | 1xA10G, 1xL40S |
| nvcr.io/nim/nvidia/nemoguard-jailbreak-detect:1.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-e5-v5-pb24h2:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-e5-v5:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-embedqa-mistral-7b-v2:1.0.1 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-rerankqa-mistral-4b-v3:1.0.2 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nv-yolox-page-elements-v1:1.1.0-rtx | 1xA100, 1xA10G, 1xL40S |
| nvcr.io/nim/nvidia/nvclip:2.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 |
| nvcr.io/nim/qwen/qwen-2.5-7b-instruct:1.0.0 | 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/snowflake/arctic-embed-l:1.0.1 | 1xA100, 1xA10G, 1xH100, 1xL40S |
| nvcr.io/nim/tokyotech-llm/llama-3-swallow-70b-instruct-v0.1:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S |
| nvcr.io/nim/tokyotech-llm/llama-3.1-swallow-70b-instruct-v0.1:1.3.2 | 4xA100, 4xH100, 2xH200, 8xL40S |
| nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH200, 1xH20, 1xL20, 1xL40S |
| nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH20, 2xH200, 1xH200, 2xL40S, 1xH100, 2xL20, 1xL20 |
| nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 1xH200, 2xH200, 4xH200, 2xH100, 4xH100, 8xH100, 2xH100, 4xH100, 8xH100, 4xA100, 8xA100, 4xL40S |
| nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 1xH100, 1xL40S, 2xA10G |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xH100, 2xH100, 1xH200, 2xH200, 2xA100, 1xA100, 1xL40S, 2xL40S, 4xL40S, 2xA10G |
| nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 8xH100, 4xH100, 4xB100 |
| nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xL4OS, 1xA10G |
| nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 2xA10G, 2xL40S, 1xA100, 1xH100 |
| nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xH200, 8xH100, 4xH100, 8xA100, 4xA100, 8xH100, 4xH100, 8xA100, 4xA100, 8xL40S |
| nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.6 | 4xL40S, 8xL40S, 8xA10G |
| nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xH100, 8xA100 |
| nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S |
| nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA10G, 1xL40S, 1xA100, 1xH100 |
| nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 |
| nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 |
| nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 |
| nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 2xH100, 1xH100 |