AI clusters carry several traffic types with different latency, loss and burst characteristics. Treating them as one stream makes capacity calculations misleading. To address “Classify traffic before choosing ports”, list collective, storage, inference ingress, management, backup and telemetry flows with source, destination and peak rate. Test that result on NVIDIA AI Server Networking.

NVLink or other local interconnect handles tightly coupled accelerator communication inside supported systems; Ethernet or InfiniBand connects nodes and racks. Those domains solve different problems. To address “Understand scale-up versus scale-out”, draw the boundary of each scale-up island and calculate what actually leaves it during the workload. Test that result on NVIDIA AI Server Networking.

Data parallel, tensor parallel, pipeline parallel and expert parallel patterns move different amounts of data at different synchronization points. To address “Estimate east-west demand from the parallelism strategy”, profile the framework with representative model size and batch settings rather than deriving fabric bandwidth solely from GPU count. Test that result on NVIDIA AI Server Networking.

Dataset reads and checkpoints can overlap with compute communication. Shared links may become the hidden throttle even when average utilization looks low. To address “Protect storage traffic from collective bursts”, capture peak concurrent traffic and decide whether separate storage interfaces, VLAN/QoS policy or a separate fabric is warranted. Test that result on NVIDIA AI Server Networking.

A high-speed NIC can lose efficiency when it sits behind a constrained PCIe switch or crosses NUMA boundaries unexpectedly. GPUDirect paths also depend on supported topology and software. To address “Check NIC-to-CPU and NIC-to-GPU locality”, map NIC and GPU slots to PCIe roots, confirm ACS/IOMMU behavior where relevant, and benchmark the actual placement. Test that result on NVIDIA AI Server Networking.

RDMA can reduce CPU overhead and latency, but it introduces requirements around NIC support, drivers, fabric configuration and operational discipline. To address “Treat RDMA as an engineered feature”, use the vendor’s supported configuration and validate lossless or congestion-control behavior appropriate to the selected technology before production. Test that result on NVIDIA AI Server Networking.

Leaf-spine or multi-plane fabrics can have enough edge ports yet insufficient uplink capacity when many nodes communicate simultaneously. To address “Calculate oversubscription explicitly”, compute worst-case edge-to-uplink ratios per failure domain and verify that the intended collective pattern fits them. Test that result on NVIDIA AI Server Networking.

High-speed designs can be constrained by switch radix, OSFP/QSFP choices, breakout support, fiber type, DAC reach and power or thermal limits of optics. To address “Count ports, optics and cables early”, build a port-level cable schedule before ordering so the logical topology can actually be wired and serviced. Test that result on NVIDIA AI Server Networking.

When the main fabric fails, operators still need BMC, switch and orchestration access. Putting every control path on the same failure domain can make repair much harder. To address “Keep management independent enough to recover”, design an out-of-band management path and test recovery with the production fabric intentionally unavailable. Test that result on NVIDIA AI Server Networking.

Adding racks can consume spare spine capacity or require a new topology stage. The initial design should state its supported expansion point. To address “Plan for fabric growth without disruptive recabling”, reserve ports, power and physical pathways for the next increment, and document the trigger for adding another fabric plane or switch tier. Test that result on NVIDIA AI Server Networking.