Leaf-spine bandwidth ratio planning
AI Network Oversubscription Calculator for GPU Fabrics
Oversubscription is the hidden ratio behind many AI network designs. A leaf switch can advertise hundreds of gigabits to each server while providing much less aggregate bandwidth toward the spine. This calculator makes that ratio explicit, shows the per-endpoint share of uplink capacity and estimates how many additional uplinks are needed to reach a target ratio.
Interactive calculator
AI Network Oversubscription Calculator
Enter your own topology and bandwidth assumptions. These planning calculators estimate raw links, capacity and ratios; they do not certify fabric goodput, rail mapping, cable reach, firmware interoperability, electrical design, cooling or OEM compatibility.
Live Amazon supporting hardware
Current supporting components and Price Options
Compare current Amazon listings relevant to this guide. Product availability and prices can change.
Quick answer
What is AI network oversubscription?
Oversubscription is the ratio of aggregate endpoint-facing downlink capacity to aggregate uplink capacity. A leaf with 32 x 400G server links and 16 x 400G spine links has 12.8 Tb/s of downlinks and 6.4 Tb/s of uplinks, or 2:1 oversubscription before other constraints. Lower ratios generally give synchronized AI workloads more predictable bandwidth.
Training fabrics usually need stricter ratios than ordinary enterprise networks
The acceptable ratio depends on traffic pattern. Large collectives and expert-parallel traffic can create simultaneous east-west demand that exposes shared uplinks quickly, while some inference workloads may tolerate controlled oversubscription. Choose the ratio from workload measurements and cost targets rather than copying a data-center default.
How to interpret common oversubscription ratios
The ratio alone does not define performance, but it makes the fabric’s shared-bandwidth assumption visible.
| Ratio | Meaning | AI workload implication | Typical action |
|---|---|---|---|
| 1:1 | Uplink capacity equals downlink capacity | Best starting point for nonblocking leaf design | Validate spine radix and traffic locality |
| 1.25:1 | 20% less uplink than downlink | Mild sharing may be acceptable | Measure collective tail latency |
| 1.5:1 | One-third less uplink than downlink | Contention can appear during synchronized bursts | Use workload traces before adoption |
| 2:1 | Half as much uplink as downlink | Significant sharing | Often risky for communication-heavy training |
| 4:1 | One quarter uplink capacity | Strong concentration | Better suited to lighter or localized traffic |
| >4:1 | Highly shared uplinks | Fabric can throttle many endpoints simultaneously | Revisit topology or workload placement |
Before you use the result for procurement
Draw the physical topology
Map every server-facing port, leaf uplink, spine link and network plane. Aggregate bandwidth alone can hide impossible port or lane assumptions.
Verify exact endpoints
Confirm NIC form factor, PCIe generation, host lane budget, port speed, connector and firmware support on the exact server platform.
Qualify optics and cables
Match OSFP/QSFP form factor, lane rate, breakout, reach, fiber type and both endpoint qualification lists. Do not treat equal headline speed as automatic compatibility.
Test failure and congestion behavior
Validate oversubscription, ECMP or multiplane path behavior, switch failure domains and recovery under the traffic patterns the AI workload will actually generate.
Oversubscription compares downlinks with uplinks
The numerator is the total capacity offered to endpoints on a leaf switch. The denominator is the capacity leaving that leaf toward the spine or next network stage. Dividing the two produces a ratio such as 1:1 or 2:1.
The ratio is a capacity model, not a prediction that every flow loses exactly the same percentage. Traffic locality, routing and message synchronization determine when the shared uplink becomes the bottleneck.
A 1:1 leaf is not automatically a nonblocking fabric
Equal leaf downlink and uplink bandwidth removes one obvious bottleneck, but the spine layer must also have enough radix and path diversity to carry the traffic. A poorly designed spine can reintroduce contention elsewhere.
Validate every stage of the topology. The calculator is intentionally leaf-centric because that is where oversubscription is often hidden in switch-port allocations.
Synchronized collectives punish shared uplinks
AI training frequently uses all-reduce, all-to-all or other communication where many accelerators transmit at the same time. Those bursts can line up on the same uplinks and turn a modest oversubscription ratio into long accelerator wait time.
Measure collective step duration and GPU idle time as the ratio is increased. The network cost saving from fewer uplinks is only worthwhile if the lost accelerator productivity is smaller than the infrastructure saving.
Expert parallelism can create variable all-to-all traffic
Mixture-of-experts models may send tokens to different experts across the cluster, creating traffic matrices that are less predictable than classic all-reduce. A link that looks lightly utilized on average can still experience severe short-lived contention.
Use percentiles and tail latency rather than average throughput. Spectrum-X adaptive routing and congestion control are designed for these difficult patterns, but no routing feature can create uplink capacity that was never provisioned.
Inference can tolerate different ratios depending on placement
Inference services may keep model shards, KV cache or expert groups within a rack or leaf domain, reducing traffic across the spine. Other serving designs continuously exchange context or route requests across many nodes and can be network intensive.
Benchmark the actual serving topology before choosing a relaxed ratio. A 2:1 network that works for one model may become the bottleneck when the serving engine changes parallelism strategy.
Utilization headroom protects burst capacity
A fabric planned around 100% continuous uplink use has no room for microbursts, retransmissions, failure reroutes or traffic growth. The calculator reports a utilization-adjusted view to show how much comfortable capacity exists below line rate.
Choose a headroom policy that reflects the service objective. Large synchronous clusters often justify lower normal utilization because the cost of a stalled accelerator fleet is high.
Higher-speed uplinks can reduce ratio without consuming more ports
If the switch supports faster spine-facing ports than server-facing ports, an operator can improve uplink capacity while preserving radix. For example, 800G uplinks can aggregate multiple 400G endpoints more efficiently than an equal number of 400G uplinks.
Breakout and lane constraints still apply. Verify that the switch can simultaneously operate the desired downlink and uplink modes and that suitable optics are available.
Multiplane changes where oversubscription is measured
In a multiplane fabric, each plane has its own leaf and spine capacity. The aggregate endpoint bandwidth is split across planes, so oversubscription should be checked inside each plane rather than only after summing all network links together.
A single undersized plane can become the slowest path if load balancing cannot fully avoid it. Keep plane-specific port counts and uplink ratios in the topology documentation.
Failure scenarios increase effective oversubscription
When one uplink or plane fails, surviving paths carry more traffic. A design that is 1:1 in normal operation can become oversubscribed during recovery unless spare capacity is reserved.
Run the calculator again with failed uplinks removed, then test the corresponding hardware fault. Failure-state ratios are often more important to service reliability than the normal-state ratio.
Optics cost can tempt designers into aggressive sharing
High-speed optical modules are expensive, so reducing spine links can look like an easy saving. The trade is that fewer optics also mean less network capacity and possibly more expensive GPU idle time.
Model total system economics. Compare the cost of additional uplinks with the value of accelerator-hours lost when communication slows. In very large AI clusters, a small percentage of stranded GPU time can outweigh many network components.
Telemetry should track congestion, not only port utilization
A port averaging 40% can still suffer queue spikes that hurt tail latency. Monitor buffer occupancy, ECN/CNP events, retransmissions, path distribution and collective performance in addition to byte counters.
The acceptance threshold should be tied to application goodput. If a 1.5:1 network meets job-completion goals with low tail congestion, it may be a better value than an overbuilt 1:1 design. If not, the ratio is too aggressive.
The ratio should be written into procurement documents
When a design simply lists switch models and cable counts, oversubscription can disappear from review. State the expected downlink capacity, uplink capacity and ratio for every leaf class.
That makes later changes auditable. If procurement substitutes a switch with fewer high-speed uplinks, the team can immediately see that the topology no longer meets the approved bandwidth assumption.
Methodology and sources
The calculator uses the standard downlink-bandwidth divided by uplink-bandwidth definition of oversubscription and adds a planning-utilization view. It does not estimate adaptive-routing gains or workload-specific performance. Use measured AI traffic and failure testing to choose an acceptable ratio.
- NVIDIA Spectrum-X Ethernet platform
- NVIDIA: Spectrum-X Multiplane technical architecture
- NVIDIA: Spectrum-6 arrives in gigascale AI factories
- NVIDIA Spectrum-6 SN6000 hardware specifications
- NVIDIA ConnectX-9 SuperNIC specifications
As an Amazon Associate, Cloudzat may earn from qualifying purchases. Marketplace listings on these pages are supporting networking hardware such as NICs, switches, optics and high-speed cables. A marketplace row is not represented as a Spectrum-6 switch, ConnectX-9 SuperNIC, Thor Ultra NIC or qualified NVIDIA fabric unless the exact listing evidence supports that identity. Verify model, speed, connector, firmware, warranty and OEM qualification before purchase.
Frequently asked questions
How is oversubscription calculated?
Divide aggregate endpoint-facing downlink bandwidth by aggregate uplink bandwidth. A result of 2 means a 2:1 oversubscription ratio.
Is 1:1 always required for AI training?
Not always, but communication-heavy synchronized workloads are sensitive to shared uplinks. Use workload benchmarks to justify any higher ratio.
Is 2:1 oversubscription bad?
It means endpoints collectively have twice the capacity of the uplinks. That can be acceptable for some localized or lighter traffic, but it can materially slow collective-heavy AI workloads.
Should I use port count or bandwidth?
Use bandwidth. Different port speeds can produce the same port-count ratio but very different oversubscription.
How do 800G uplinks help?
Faster uplinks increase spine capacity without necessarily consuming more switch ports, assuming the switch and optics support the desired mode.
Does Spectrum-X remove oversubscription?
No. Adaptive routing and congestion control can use available capacity more efficiently, but they cannot compensate for a topology that lacks enough aggregate uplink bandwidth.
How should I calculate a multiplane network?
Calculate the ratio inside each plane using that plane’s downlinks and uplinks, then evaluate the aggregate and failure-state behavior.
What happens after one uplink fails?
The remaining uplinks carry the traffic, increasing the effective oversubscription ratio. Recalculate the ratio with the failed link removed.
What utilization target should I use?
There is no universal number. Choose a target based on traffic burstiness, failure headroom and the cost of accelerator stalls.
What metrics validate the chosen ratio?
Track collective step time, GPU idle time, queue depth, congestion notifications, retransmissions, path balance and application goodput under normal and failure conditions.