Nvidia opens 120B agent model that runs on one GPU

The latest:

A 120-billion-parameter open-weight model that runs on a single H200 or DGX Spark is Nvidia’s new bid for enterprise agents. Nemotron 3.5 Super VL, released October 15, 2026 under the OpenMDW-1.1 license, activates just 12 billion parameters per token and handles text, images, documents and video. Nvidia also unveiled two edge models under the Nemotron 3.5 Bolt line.

Details:

• The architecture: Nvidia said the model interleaves Mamba-2 layers, mixture-of-experts blocks and attention, with 512 routed experts per expert layer and a C-RADIO vision encoder. Context runs to 1 million tokens, and reasoning can be switched on or off through the chat template. Inputs include MP4 clips up to two minutes; outputs are text only.
• The efficiency claim: According to Nvidia, Super VL approaches the accuracy of Nemotron 3 Ultra on multimodal agentic tasks at roughly one-fifth the parameters; Ultra holds 550 billion total and 55 billion active, about 4.6 times larger. Nvidia also claims up to 5 times the throughput of comparable open models on a single GPU.
• The benchmarks: On the NVFP4 checkpoint, Nvidia reported GPQA-Diamond 89.90, SWE-bench Verified 71.07, Terminal-Bench 2.1 60.80, BrowseComp 77.5, IFBench 71.50, CharXiv reasoning 79.88, MMLongBench-Doc 62.82 and OSWorld-Verified 52.3. The company says it ranks among the top open models on OCRBench v2 without publishing a figure.
• Against Ultra: In agent frameworks, Nvidia put Super VL at 63.44% on Hermes Bench versus 56.33% for Nemotron 3 Ultra, and 87.05% versus 84.3% on LangChain. On CharXiv it reported 80.8%, rising to 84% with Python code interpretation, against 63% for Nemotron 3 Nano Omni.
• The training: Nvidia said the model was distilled from Nemotron 3 Ultra using a SkewReverse loss aimed at coding accuracy, across roughly 26 trillion tokens of pretraining and continued pretraining. Post-training used about 3.8 million samples with reinforcement learning across dozens of environments, including 10 billion tokens in Indic languages.
• The open release: Beyond weights, Nvidia published training recipes, reinforcement learning environments in NeMo Gym, and datasets on Hugging Face covering tool use, finance, instruction following, long context, fairness and 45 vision-language sets. A separate 2.8-trillion-token dataset spanning 22 Indic languages was built with IIT Madras and its AI4Bharat lab.
• The speed work: Nvidia said LiLiCorr speculative decoding lifts token generation up to 2.3 times on DGX Spark, alongside multi-token prediction and context caching that preserves attention memory and Mamba states between agent turns. With Dynamo and KV-aware routing on 16 B200 GPUs, it claims 8 times more concurrent users and 85% lower time to first token.
• Where to get it: Weights are on Hugging Face in NVFP4 and BF16 checkpoints, plus OpenRouter, ModelScope and an NVIDIA NIM microservice updated every two weeks. Runtimes include vLLM, SGLang and TensorRT-LLM on Blackwell and Hopper, with llama.cpp and Unsloth locally. Customization runs through NeMo AutoModel and NeMo RL.
• The edge models: Nemotron 3.5 Bolt 74B-A9B ships now with native 2-bit quantization to fit 24GB of memory, built for the RTX Spark laptops arriving October 16, 2026. Nvidia places it first among open models in its class on PinchBench and second on BrowseComp. A 4GB Bolt E4B follows on October 23, 2026 for physical and conversational agents.
• Early customers: Nvidia listed Scale AI post-training the model for a supplier credit agent at Skydance, citing 40 times lower inference cost and 5 times lower latency, and for an M&A due-diligence agent at EY. AfterQuery tuned it for credit-agreement analysis at Hebbia; AI21 and Alice reported 40% higher accuracy in threat analysis. Ramp expects up to 50% higher throughput converting receipts to JSON.

Background:

Nemotron 3.5 is split into three tiers: Lightning, released in August for high throughput; Ultra for the hardest reasoning tasks; and Bolt for edge devices. Super VL sits between them as the vision-language agent tier.

Between the lines:

The entire pitch rests on parameter efficiency rather than raw capability. By pairing a 12-billion active-parameter footprint with single-GPU deployment and claimed 5-times throughput, Nvidia is targeting the cost of running many agents concurrently, not the top of any leaderboard. The customer figures it cites, including 40 times lower inference cost at Skydance, are framed in cost and latency terms rather than accuracy. No named executive accompanied the announcement.

What’s next

The RTX Spark laptops Bolt 74B was built for arrive October 16, 2026, and the 4GB Bolt E4B lands October 23. Independent benchmark results, including Artificial Analysis agentic scores, will test Nvidia’s throughput and accuracy claims.