Engineering at Meta

JUNE 12, 2024

There are several reasons for this failure, but this failure mode is seen more in the early life and settles as the server ages. HW network cable: In the general category of unreachable servers, these failures are also seen most often in the early life of the server. Both of these options had tradeoffs.

Infiniband 141

Infiniband Data Centers Topology Networking 141

Engineering at Meta

MARCH 12, 2024

The other cluster features an NVIDIA Quantum2 InfiniBand fabric. Through careful co-design of the network, software, and model architectures, we have successfully used both RoCE and InfiniBand clusters for large, GenAI workloads (including our ongoing training of Llama 3 on our RoCE cluster) without any network bottlenecks.

Infiniband 145

Infiniband Data Centers Server Networking 145

Router-switch

SEPTEMBER 25, 2024

In today’s high-speed networking world, optimizing and troubleshooting performance is crucial, especially with high-performance equipment like NVIDIA Infiniband switches. In this blog, we’ll share top tips for debugging and optimizing NVIDIA Infiniband networking performance.

Infiniband 40

Infiniband Networking Network Routers 40

Engineering at Meta

AUGUST 5, 2024

The scheduler does this by learning the position of GPU servers in the logical topology to recommend a rank assignment. Routing The scaling of compute power and network topology discussed above led to the question of how to efficiently balance and route the massive training traffic. Thus, we took two steps to improve the performance.

Networking 132

Networking Network Topology Data Centers 132