Engineering at Meta

JUNE 12, 2024

Supporting GenAI at scale has meant rethinking how our software, hardware, and network infrastructure come together. Solving this problem requires a robust and high-speed network infrastructure as well as efficient data transfer protocols and algorithms. requires revisiting trade-offs made for other types of workloads.

Infiniband 141

Infiniband Data Centers Topology Network 141

Engineering at Meta

AUGUST 5, 2024

AI networks play an important role in interconnecting tens of thousands of GPUs together, forming the foundational infrastructure for training, enabling large models with hundreds of billions of parameters such as LLAMA 3.1 Distributed training, in particular, imposes the most significant strain on data center networking infrastructure.

Network 132

Network Networking Topology Data Centers 132

Engineering at Meta

MARCH 12, 2024

We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. Network At Meta, we handle hundreds of trillions of AI model executions per day. The other cluster features an NVIDIA Quantum2 InfiniBand fabric.

Infiniband 145

Infiniband Data Centers Server Network 145