A big model takes dozens or hundreds of GPUs to train, and they have to remain perfectly synced as they work. At the end of every training iteration, they have to pass gradients and parameters between one another, meaning that GPUs can spend a lot of time sitting there waiting for the other end of the wire, instead of actually computing. The connection between GPUs can be the difference between finishing in a day versus a week.
It’s why InfiniBand versus Ethernet has always been a popular discussion. Our guide to GPU cloud computing covers the entire network stack. In this one post, we just focus on the question people ask and where it landed in 2026.
When the network is actually the bottleneck
So, let’s just tell it straight. Networking is only useful when communication is the bottleneck in your application. Whether it’s communication or computation is workload-dependent.
Training big models on many nodes is communication-bound. Tensor and pipeline parallelism, which synchronize at every step, can mean that on dozens of nodes with hundreds of GPUs, even a small delay per exchange adds up to hours lost in training. This is when the fabric makes its money.
Smaller training runs and most inference is the other side of the story. A job that fits within a single 8-GPU server is limited more by the fast connections between GPUs in that server, rather than by the fabric between nodes. And for inference, there’s not nearly as much GPU-to-GPU communication to move data around compared to distributed training, so it is able to perform on Ethernet in many cases.
So the more useful question isn’t whether InfiniBand is better; it’s whether my workload communication bound or is it compute bound?
The two fabrics at a glance
Stripped of the tribalism, each option comes down to a short list of strengths and trade-offs.
InfiniBand
- Delivers the lowest and most predictable latency, roughly one to two microseconds, on a lossless fabric that avoids the retransmissions that stall synchronisation. That consistency is why it still anchors many of the largest AI supercomputers.
- Comes at a premium. Per-port costs often run 1.5 to 2.5 times comparable Ethernet once the hardware and the specialist staff to run it are counted, and the fabric comes from effectively a single vendor.
Ethernet
- Is the technology your team already runs, sourced from many suppliers rather than one, at lower cost. With RDMA over Converged Ethernet (RoCE) it reaches the single-digit microseconds when tuned well, within striking distance of InfiniBand for most training.
- Needs careful configuration to stay efficient at scale. Standard Ethernet trails a purpose-built AI fabric until you move to an AI-optimised variant, which is exactly what changed in 2026.
The 2026 movement: the gap is closing
Until now, the choice was clear: InfiniBand for performance, Ethernet for cost. 2026 marked the beginning of an era where that binary was starting to collapse.
Much of that trend is due to a misperception. The narrative that Ethernet is five to 10 times slower than InfiniBand is almost entirely based on poorly-configured networks. When RoCE is properly optimized with priority flow control, adaptive routing, and other techniques, the disparity shrinks significantly and the numbers seem to back this up.
For example, Meta ran the training of its Llama 3 models on a 24,000-H100 RoCE Ethernet cluster, and NVIDIA reports its Spectrum-X Ethernet held around 95 percent throughput on xAI’s 100,000-GPU Colossus cluster. The number for standard Ethernet at that scale, according to NVIDIA, was about 60 percent.
The standards caught up too. The Ultra Ethernet Consortium released its 1.0 specification in mid-2025, backed by more than a hundred companies under the Linux Foundation, rebuilding the Ethernet stack for AI with intelligent congestion control and native RDMA. The most telling signal is NVIDIA’s own hand: the company behind Quantum InfiniBand now also sells Spectrum-X, an Ethernet platform engineered for AI.
When the leading vendor of InfiniBand introduces a competing Ethernet fabric, the game is clearly over. InfiniBand still controls about 80% of AI clusters as of 2023, but Dell’Oro analyst estimates suggest Ethernet will surpass it in AI backend networks by 2027.
None of that means InfiniBand is retired. The lowest latency in the most intense training is still its bailiwick. What has shifted is that the real divide in 2026 is less InfiniBand versus Ethernet than standard Ethernet vs Ethernet optimized for AI.
If you rent, you inherit the fabric
Most organisations do not build their own clusters. They rent capacity from a GPU cloud, which changes where this decision sits. You are not choosing switches yourself, but you are choosing a provider whose networking determines how your jobs scale.
That makes provider transparency the thing to check. Before committing to sustained distributed training, it is worth asking:
- What fabric connects the nodes: InfiniBand, an AI-grade Ethernet like Spectrum-X, or plain Ethernet that will struggle once a job spans many nodes?
- How does measured throughput hold up as the cluster scales, rather than the single-node benchmark that looks good in a sales deck?
For workloads that never leave a single node, or for inference, these questions matter far less. For large distributed training, they decide whether the GPUs you are paying for actually stay busy.
SkyBiometry’s managed GPU cloud is built around this: dedicated clusters with the interconnect matched to the workload, so distributed training runs on a fabric designed for it rather than whatever the cheapest tier happened to ship with.
The decision in short
InfiniBand still wins the largest, most latency-sensitive training, and it will keep that lead for a while yet. Below that frontier, the choice is now genuinely open. As a rule of thumb:
- Large-scale, latency-critical training across many nodes: InfiniBand remains the safe default.
- Mid-size training, inference, or AI folded into an existing data centre: modern Ethernet, ideally an AI-optimised fabric, is usually enough and costs less.
- Renting rather than building: the real question is which fabric your provider runs, not which one you buy.
The one answer that is always wrong is picking a fabric by reputation instead of by workload.
For the sibling decision, which accelerator to run on that fabric, see our comparison of H100, A100 and RTX Pro. For how these networking and hardware choices translate into cost, see our analysis of tokenomics and the compute economy.
