Modal said Monday, October 19, that it plans to introduce Nvidia’s Vera central processor into its AI sandbox service after internal tests showed up to twice the task throughput of a comparable Intel system under heavy load. The company’s report describes an expansion beyond its existing x86 infrastructure, not a completed fleet-wide conversion.
The finding concerns the computing environments where AI agents execute commands, rather than the models that generate their answers. Modal reported a much less pronounced advantage when tasks ran individually on an otherwise idle machine. Its results therefore support a specific claim about handling concurrent workloads, not a general conclusion that every AI application will run twice as fast.
Why AI agents need more than GPUs
AI agents alternate between generating instructions and carrying them out. In its architecture analysis, Nvidia describes CPU work that includes executing code, querying databases, retrieving information and processing tool results. Those operations can sit on the critical path between a model’s decisions. Faster graphics processors do not, by themselves, eliminate delays in the surrounding software.
The practical challenge is sharing a server without letting one customer’s activity slow another’s. Modal’s resource documentation says its sandboxes reserve a minimum amount of computing capacity but can use spare resources beyond that reservation. Billing reflects the higher of requested capacity or actual consumption. This arrangement makes available headroom useful, but also means that predictable behavior under simultaneous demand matters alongside speed on an empty server.
What the benchmark actually measures
Modal published its test code, called Agentless Agent Bench, making the methodology available for examination. The repository describes 50 tasks drawn from Terminal-Bench 2.1, executed using predefined solutions rather than waiting for a language model to decide what to do. Network-dependent work is excluded, and containers are prepared and warmed before measurement. A task counts as complete only after its verification step passes.
That design isolates host-computing performance from several other sources of delay. It measures both sequential task latency and the number of verified runs completed concurrently within a fixed interval. It does not measure how reliably an AI model chooses the right command, how long a remote service takes to respond or the full experience of starting a fresh environment. Those omissions are methodological boundaries, not evidence that the excluded delays have disappeared. The repository also provides a test that increases the available CPU count while varying concurrent task replicas. That makes scaling behavior part of the published method, rather than reducing the evaluation to a single lightly loaded run.
Memory and contention shape the result
According to Nvidia’s product specifications, Vera combines 88 custom Olympus cores with 176 hardware threads and memory bandwidth of up to 1.2 terabytes per second. Nvidia offers it as both a host for accelerated computing and a standalone processor. These are manufacturer specifications; they do not establish a uniform performance gain across customers or software.
Nvidia’s technical explanation emphasizes sustained progress when many tasks compete for resources. It describes an approach that partitions resources between two hardware threads to reduce interference, alongside a memory subsystem designed to keep active cores supplied with data. Modal attributes its stronger loaded-server results to that ability to withstand contention. The distinction is consequential: completing more jobs on a busy host is a different achievement from cutting every individual job’s execution time in half.
Independent testing also has limits
Earlier Phoronix testing provides a separate firsthand assessment of Vera, though not a replication of Modal’s experiment. Reviewer Michael Larabel tested preproduction hardware at Nvidia’s offices in May and reported strong results in the selected workloads. His disclosure also explains why that review cannot settle every question about production systems or operating costs. Testing used Ubuntu with a patched Linux kernel and a specific compiler configuration. Larabel did not test installation across different Linux distributions, another reason to distinguish a successful controlled demonstration from validation of every customer’s software environment.
Nvidia limited the permitted benchmark scope and asked that processor power and frequency monitoring not be used while tuning remained underway, Larabel wrote. He described the article as unsponsored, but the test conditions were still constrained. Neither that review nor Modal’s report supplies a general, independently established cost-per-task advantage for all deployments. Evaluating that question requires performance and power measurements, hardware costs and the workload actually being served, rather than a throughput figure alone.
A rollout plan, not an exclusive switch
Modal chief executive Erik Bernhardsson told Reuters that the company will retain Intel and AMD processors in a mixed fleet and place work on the hardware best suited to it. He also said the comparison involved chips already available, while newer competing processors remain to be tested. The decision gives Nvidia another CPU customer without establishing that existing suppliers have been displaced.
Modal says supporting the Arm-compatible processor requires changes to its infrastructure and image-building system. Its announcement expresses an intention to scale with Nvidia and cloud providers, but does not provide a general availability date. The immediate development is a planned hardware expansion backed by a disclosed benchmark. Whether customers receive consistently faster service or lower bills remains a separate deployment question.
