Engineering at Meta

JUNE 12, 2024

Data center deployment Once we’ve chosen a GPU and system, the task of placing them in a data center for optimal usage of resources (power, cooling, networking, etc.) There are two leading choices in the industry that fit these requirements: RoCE and InfiniBand fabrics. Both of these options had tradeoffs.

Infiniband 141

Infiniband Data Centers Topology Network 141

Engineering at Meta

AUGUST 5, 2024

Distributed training, in particular, imposes the most significant strain on data center networking infrastructure. Constructing a reliable, high-performance network infrastructure capable of accommodating this burgeoning demand necessitates a reevaluation of data center network design.

Network 132

Network Networking Topology Data Centers 132

Engineering at Meta

MARCH 12, 2024

Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta. Custom designing much of our own hardware, software, and network fabrics allows us to optimize the end-to-end experience for our AI researchers while ensuring our data centers operate efficiently.

Infiniband 145

Infiniband Data Centers Server Network 145

Router-switch

SEPTEMBER 25, 2024

In today’s high-speed networking world, optimizing and troubleshooting performance is crucial, especially with high-performance equipment like NVIDIA Infiniband switches. Whether you’re a data center admin or network engineer, mastering effective techniques is key.

Infiniband 40

Infiniband Network Networking Routers 40