What emerges when AI agents require to run constantly without sending every task to an expensive frontier model? NVIDIA’s answer is a integration of specialized models and intelligent routing.
The company has released Nemotron 3.5 Lightning, a lightweight open model designed for high-volume agentic workloads, along NeMo Switchyard, an open-source library that routes requests between models based on factors including capability, latency, and cost.
Nemotron 3.5 Lightning Targets Always-On AI Agents
Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-specialists model built for specialised tasks inside larger multi-agent systems. Instead of needing a frontier reasoning model to manage each step, developers can assign Lightning duties tasks such as code review, tool use, security monitoring, and customer-assist queries.
NVIDIA says the model supply up to 4 times faster output and 30% faster completion of agentic tasks compared with other models in its class. Organizations can also post-train the open model using NVIDIA NeMo and their own domain-specific data, tools, and workflows.
That specialization could become an increasing essential as agent architectures shift towards systems of models. A larger reasoning models might plan a workflow at the same time as smaller models handle repetitive or closely described steps, reducing the amount of compute needed for every interaction.
Nemotron 3.5 Lightning can also run across local and enterprise infrastructure, consisting of RTX PCs, DGX systems, Jetson devices, workstations, data centers and cloud environments. This offers organizations additional options when workloads need low latency, local execution or tighter control over data.
NeMo Switchyard Brings Model Routing Into the Agent Stack
Managing numerous model formed another issue: deciding which model should get each request.
NeMo Switchyard addresses that challenge by automatically routing individual steps to models ideal to the task. Developers can tune routing approaches around priorities such as accuracy, latency, and cost while integrating NVIDIA, open, and proprietary models without rewriting the underlying agent application.
NVIDIA reports that internal benchmarks decreased task-completion costs to almost one-third of using Opus 4.8 alone while the same time as maintaining frontier-level accuracy. Partner testing also points to potential savings. LangChain reported 74% lower costs throughout 145 multi-flip Deep Agents tasks, while Ramp said a 58% cost reduction and 33% lower runtime in its SWE-Bench testing. These outcomes shows specific benchmark environments instead of guarantees for every workload.
Why Model Routing Could Matter for Agentic AI
The launches toward a wider change in AI engineering. Building effective agents may increasingly rely less on selecting one universal model and more on orchestrating numerous models as per to the requirements of each step.
