Packet Pushers

OCTOBER 25, 2024

Ethernet competes with InfiniBand as a network fabric for AI workloads such as model training. And while Ethernet has kept up with increasing demands to support greater bandwidth and throughput, it was. Read more » Ethernet competes with InfiniBand as a network fabric for AI workloads such as model training.

Ethernet 83

Ethernet Infiniband Bandwidth Network 83

Packet Pushers

AUGUST 21, 2024

We discuss key considerations including bandwidth, the substantial power and cooling requirements of AI infrastructure, and GPUs. We also talk about InfiniBand and Ethernet as network fabrics for AI workloads, cabling considerations, and more. This is a sponsored episode. This is a sponsored episode. Read more »

Infiniband 52

Infiniband Ethernet Bandwidth Network 52

Engineering at Meta

AUGUST 5, 2024

Our paper, “ RDMA over Ethernet for Distributed AI Training at Meta Scale ,” provides the details on how we design, implement, and operate one of the world’s largest AI networks at scale. We opted for RDMA Over Converged Ethernet version 2 (RoCEv2) as the inter-node communication transport for the majority of our AI capacity.

Network 132

Network Networking Topology Data Centers 132

Engineering at Meta

MARCH 12, 2024

With this in mind, we built one cluster with a remote direct memory access (RDMA) over converged Ethernet (RoCE) network fabric solution based on the Arista 7800 with Wedge400 and Minipack2 OCP rack switches. The other cluster features an NVIDIA Quantum2 InfiniBand fabric. Both of these solutions interconnect 400 Gbps endpoints.

Infiniband 145

Infiniband Data Centers Server Network 145