Remove Fashion Remove Infiniband Remove Networking
article thumbnail

How Meta trains large language models at scale

Engineering at Meta

Supporting GenAI at scale has meant rethinking how our software, hardware, and network infrastructure come together. Optimal connectivity between GPUs: Large-scale model training involves transferring vast amounts of data between GPUs in a synchronized fashion. requires revisiting trade-offs made for other types of workloads.

article thumbnail

A RoCE network for distributed AI training at scale

Engineering at Meta

AI networks play an important role in interconnecting tens of thousands of GPUs together, forming the foundational infrastructure for training, enabling large models with hundreds of billions of parameters such as LLAMA 3.1 Distributed training, in particular, imposes the most significant strain on data center networking infrastructure.

Network 132
article thumbnail

Building Meta’s GenAI Infrastructure

Engineering at Meta

We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. Network At Meta, we handle hundreds of trillions of AI model executions per day. The other cluster features an NVIDIA Quantum2 InfiniBand fabric.