Skip to content
For Self-Managed AI Platform users running v11.1, see the on-premise platform documentation

NIM Containers – GPU Support Matrix

NIM Containers – GPU Support Matrix

See the Generic configuration guide.

This table maps each NVIDIA NIM container image to the set of GPU hardware profiles that have been validated by NVIDIA for that exact model version.

  • The smallest-count, lowest-tier device in each row (e.g., 1xT4 for PaddleOCR) is the least hardware you can run the container on without exceeding memory limits.
  • The "2x", "4x", "8x" prefixes indicate how many identical GPUs must be present in the node.
  • Any GPU shown in a row can host that model, letting you match performance, memory, or cost targets with the hardware you already own.

Consult the first (smallest) configuration to see the minimum requirement, and pick any other listed configuration when you need higher batch sizes, faster latency, or have newer GPUs available.

| 名前 | GPU Requirements | |------|--------| | nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 | | nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 1xH100, 2xH100 | | nvcr.io/nim/black-forest-labs/flux.1-dev:1.0.1 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S | | nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S | | nvcr.io/nim/deepseek-ai/deepseek-r1-distill-llama-8b:1.5.2 | 1xA100, 2xA10G, 1xH100, 4xL40S | | nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH20, 1xH200, 1xL20, 1xL40S | | nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH100, 1xH20, 1xH200, 2xH200, 1xL20, 2xL20
2xL40S | | nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-7b:1.1.0 | 1xA10G, 1xL4OS | | nvcr.io/nim/defog/llama-3-sqlcoder-8b:1.2.3 | 1xA10G, 2xA10G, 4xA10G, 1xH100, 2xH100, 1xL40S
2xL40S | | nvcr.io/nim/google/gemma-2-2b-instruct:1.4.0 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/google/gemma-2-9b-it:1.4.0 | 1xA100, 1xA10G, 1xH100 | | nvcr.io/nim/ipd/proteinmpnn:1 | 1xA10G, 1xL40S | | nvcr.io/nim/ipd/rfdiffusion:2 | 1xA100, 1xH100, 1xL40S | | nvcr.io/nim/meta/codellama-13b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100 | | nvcr.io/nim/meta/codellama-34b-instruct:1.2.2 | 2xA100, 4xA100, 4xA10G, 8xA10G, 2xH100, 4xH100
4xL40S | | nvcr.io/nim/meta/codellama-70b-instruct:1.2.2 | 4xA100, 8xA100, 8xA10G, 4xH100, 8xH100 | | nvcr.io/nim/meta/llama-2-13b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S | | nvcr.io/nim/meta/llama-2-70b-chat:1.0.3 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 4xL40S | | nvcr.io/nim/meta/llama-2-7b-chat:1.0.3 | 1xA100, 2xA100, 1xH100, 2xH100, 1xL40S, 2xL40S | | nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200
2xH200, 4xH200, 4xL40S | | nvcr.io/nim/meta/llama-3.1-70b-instruct:1.8.4 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200
2xH200, 4xH200, 4xL40S | | nvcr.io/nim/meta/llama-3.1-8b-base:1.1.2 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100
2xL40S | | nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 2xA10G, 1xH100, 1xL40S | | nvcr.io/nim/meta/llama-3.1-8b-instruct:1.8.4 | 2xA10G, 1xH100, 1xL40S | | nvcr.io/nim/meta/llama-3.2-11b-vision-instruct:1.1.1 | 1xA100, 2xA100, 4xA10G, 8xA10G, 1xH100, 2xH100
1xH200, 2xH200, 2xL40S, 4xL40S | | nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xA10G, 1xL4OS | | nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 1xA100, 2xA10G, 1xH100, 2xL40S | | nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xA100, 8xA100, 2xH100, 4xH100, 8xH100, 1xH200
2xH200, 4xH200, 8xL40S | | nvcr.io/nim/meta/llama-3.3-70b-instruct:1.8.5 | 8xA100, 8xH100, 4xH200, 8xL40S | | nvcr.io/nim/meta/llama3-70b-instruct:1.0.3 | 4xA100, 4xH100, 8xH100 | | nvcr.io/nim/meta/llama3-8b-instruct:1.0.3 | 1xA100, 2xA100, 1xA10G, 2xA10G, 1xH100, 2xH100
1xL40S, 2xL40S | | nvcr.io/nim/microsoft/phi-3-mini-4k-instruct:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/mistralai/mistral-7b-instruct-v0.3:1.3.0 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100
1xL40S, 2xL40S | | nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xA100, 8xH100 | | nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:1.3 | 2xA100, 4xA100, 8xA10G, 2xH100, 4xH100, 4xL40S | | nvcr.io/nim/nv-mistralai/mistral-nemo-12b-instruct:1.2.2 | 1xA100, 2xA100, 8xA10G, 1xH100, 2xH100 | | nvcr.io/nim/nv-mistralai/mistral-nemo-minitron-8b-8k-instruct:1.2.3 | 1xA100, 2xA100, 2xA10G, 4xA10G, 1xH100, 2xH100
1xL40S, 2xL40S | | nvcr.io/nim/nvidia/cosmos-predict1-7b-text2world:1.0.0 | 1xH100 | | nvcr.io/nim/nvidia/cosmos-predict1-7b-video2world:1.0.0 | 1xH100 | | nvcr.io/nim/nvidia/domino-automotive-aero:1.0.0 | 1xA100, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/genmol:1.0.0 | 1xA100, 1xA10G, 1xA6000, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-content-safety:1.0.0 | 1xL40S, 4xL40S, 8xL40S | | nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-topic-control:1.0.0 | 1xL40S, 4xL40S, 8xL40S | | nvcr.io/nim/nvidia/llama-3.1-nemotron-70b-instruct:1.2 | 4xA100, 8xA100, 4xH100, 8xH100 | | nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xA100, 2xA100, 2xA10G, 1xH100, 2xH100, 1xH200
2xH200, 1xL40S, 2xL40S, 4xL40S | | nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 4xB100, 4xH100, 8xH100 | | nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:1.5.0 | 1xA100, 1xA10G, 1xH100, 1xL4, 1xL40S | | nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.4 | 4xA100, 8xA100, 8xA10G, 1xH100, 2xH100, 4xH100
8xH100, 1xH200, 2xH200, 4xH200, 4xL40S, 8xL40S | | nvcr.io/nim/nvidia/molmim:1.0.0 | 1xA10G, 1xL40S | | nvcr.io/nim/nvidia/nemoguard-jailbreak-detect:1.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S | | nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S | | nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xA100, 1xA10G, 1xB200, 1xH100, 1xL4, 1xL40S | | nvcr.io/nim/nvidia/nv-embedqa-e5-v5-pb24h2:1.2.3 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nv-embedqa-e5-v5:1.6.0 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nv-embedqa-mistral-7b-v2:1.0.1 | 1xA100, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nv-rerankqa-mistral-4b-v3:1.0.2 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nv-yolox-page-elements-v1:1.1.0-rtx | 1xA100, 1xA10G, 1xL40S | | nvcr.io/nim/nvidia/nvclip:2.0.0 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 | | nvcr.io/nim/qwen/qwen-2.5-7b-instruct:1.0.0 | 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/snowflake/arctic-embed-l:1.0.1 | 1xA100, 1xA10G, 1xH100, 1xL40S | | nvcr.io/nim/tokyotech-llm/llama-3-swallow-70b-instruct-v0.1:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S | | nvcr.io/nim/tokyotech-llm/llama-3.1-swallow-70b-instruct-v0.1:1.3.2 | 4xA100, 4xH100, 2xH200, 8xL40S | | nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:1.1.2 | 4xA100, 8xA10G, 4xH100, 8xL40S | | nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-14b:1.1.0 | 1xH200, 1xH20, 1xL20, 1xL40S | | nvcr.io/nim/deepseek-ai/deepseek-r1-distill-qwen-32b:1.1.0 | 1xH20, 2xH200, 1xH200, 2xL40S, 1xH100, 2xL20, 1xL20 | | nvcr.io/nim/meta/llama-3.1-70b-instruct-pb24h2:1.3.6 | 1xH200, 2xH200, 4xH200, 2xH100, 4xH100, 8xH100, 2xH100, 4xH100, 8xH100, 4xA100, 8xA100, 4xL40S | | nvcr.io/nim/meta/llama-3.1-8b-instruct-pb24h2:1.3.6 | 1xH100, 1xL40S, 2xA10G | | nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:1.8.3 | 1xH100, 2xH100, 1xH200, 2xH200, 2xA100, 1xA100, 1xL40S, 2xL40S, 4xL40S, 2xA10G | | nvcr.io/nim/nvidia/llama-3.1-nemotron-ultra-253b-v1:1.8.4 | 8xH100, 4xH100, 4xB100 | | nvcr.io/nim/meta/llama-3.2-1b-instruct:1.8.5 | 1xL4OS, 1xA10G | | nvcr.io/nim/meta/llama-3.2-3b-instruct:1.8.4 | 2xA10G, 2xL40S, 1xA100, 1xH100 | | nvcr.io/nim/meta/llama-3.2-90b-vision-instruct:1.1.1 | 4xH200, 8xH100, 4xH100, 8xA100, 4xA100, 8xH100, 4xH100, 8xA100, 4xA100, 8xL40S | | nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.8.6 | 4xL40S, 8xL40S, 8xA10G | | nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:1.2.2 | 8xH100, 8xA100 | | nvcr.io/nim/colabfold/msa-search:1.0.0 | 1xA100, 1xH100, 1xL40S | | nvcr.io/nim/nvidia/nemoretriever-graphic-elements-v1:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 | | nvcr.io/nim/nvidia/nemoretriever-page-elements-v2:1.3 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 | | nvcr.io/nim/nvidia/nemoretriever-parse:1.2 | 1xA10G, 1xL40S, 1xA100, 1xH100 | | nvcr.io/nim/nvidia/nemoretriever-table-structure-v1:1.3.0 | 1xL4, 1xA10G, 1xL40S, 1xA100, 1xH100, 1xH100, 1xB200 | | nvcr.io/nim/openfold/openfold2:1.0 | 1xA100, 1xH100 | | nvcr.io/nim/baidu/paddleocr:1.3.0 | 1xA100, 1xA10G, 1xT4 | | nvcr.io/nim/bigcode/starcoder2-7b:1.8.1 | 2xH100, 1xH100 |