Cloudflare has announced the general deployment of its next-generation servers, the Generation 12 (Gen 12), powered by AMD EPYC 9684X processors (codenamed "Genoa-X"). This new generation focuses on delivering exceptional performance across all Cloudflare services, enhancing support for artificial intelligence and machine learning workloads, achieving significant advances in energy efficiency, and improving security features.
Among the key improvements offered by this generation compared to the previous one are:
- Doubled performance: Through close collaboration between Cloudflare and AMD, Gen 12 servers can handle more than double the requests per second (RPS) than Gen 11 servers, resulting in lower infrastructure costs.
- Improved energy efficiency: RPS per watt efficiency has improved by over 60% compared to the previous generation. This helps reduce operational costs and Cloudflare’s carbon footprint.
- Enhanced support for AI and ML: The thermal-mechanical design of Gen 12 servers has been updated to support more powerful GPUs, allowing for handling larger language models and increasing performance for smaller models.
- Security improvements: Hardware root of trust (HRoT) capabilities have been integrated to ensure the integrity of boot firmware and board management controller. Additionally, the Data Center Secure Control Module (DC-SCM) is modular and provider-agnostic, allowing for a unified openBMC image and faster prototyping.
- Multi-vendor strategy: To ensure continuity and resilience in infrastructure deployment, Cloudflare has adopted a robust multi-vendor strategy to mitigate risks in the supply chain.
Technical Details of Gen 12 Servers
The Gen 12 server has the following specifications:
- Form Factor: 2U1N – Single socket
- Processor: AMD EPYC 9684X Genoa-X 96-core
- Memory: 384 GB DDR5-4800, x12 memory channels
- Storage: 2x NVMe E1.S 7.68 TB
- Network: Dual 25 GbE OCP 3.0
- System Management: DC-SCM 2.0, ASPEED AST2600 (BMC) + AST1060 (HRoT)
- Power Supply: 800W – Titanium Grade
CPU Models Comparison
Cloudflare evaluated several CPU options for Gen 12, focusing on three main candidates from the AMD EPYC 4th generation series: Genoa 9654, Bergamo 9754, and Genoa-X 9684X. Here are the differences in specifications between these models and the AMD EPYC 7713 used in Gen 11 servers:
| Model | AMD EPYC 7713 | AMD EPYC 9654 | AMD EPYC 9754 | AMD EPYC 9684X |
|---|---|---|---|---|
| Series | Milan | Genoa | Bergamo | Genoa-X |
| CPU Cores | 64 | 96 | 128 | 96 |
| Threads | 128 | 192 | 256 | 192 |
| Base Frequency | 2.0 GHz | 2.4 GHz | 2.25 GHz | 2.4 GHz |
| Max Frequency | 3.67 GHz | 3.7 GHz | 3.1 GHz | 3.7 GHz |
| Boost Frequency | 2.7 GHz* | 3.55 GHz | 3.1 GHz | 3.42 GHz |
| Total L3 Cache | 256 MB | 384 MB | 256 MB | 1,152 MB |
| L3 Cache per Core | 4 MB | 4 MB | 2 MB | 12 MB |
| Max TDP | 240W | 400W | 400W | 400W |
*Note: The 2.7 GHz all-core boost frequency of the AMD EPYC 7713 is not an official specification but based on data collected in Cloudflare’s production fleet.
Performance and Efficiency
During testing, each of these CPUs outperformed the previous generation by at least 2 times. The AMD EPYC 9684X "Genoa-X" with 3D V-cache technology offered the greatest improvement, with performance 2.45 times higher than the AMD EPYC 7713 "Milan" of Gen 11 servers.
Comparing performance between Genoa-X 9684X and Genoa 9654, a difference of approximately 22.5% was observed. The primary difference lies in the amount of available L3 cache; the Genoa-X 9684X has 1,152 MB of L3 cache, three times more than the 384 MB of Genoa 9654. Cloudflare workloads benefit from increased low-level cache, avoiding latency penalties associated with data retrieval from memory.
Commitment to Energy Efficiency
While the Gen 12 server consumes 600 watts at a typical 25°C ambient temperature (a 50% increase from Gen 11), it offers a 145% increase in performance. Energy efficiency improved by 63%, significantly reducing operational costs and Cloudflare’s carbon footprint.
Memory and Storage
- Memory: Gen 12 servers use DDR5 memory with twelve fully utilized channels, offering increased bandwidth and efficiency. 384 GB of memory was configured based on optimal resource usage and allocation.
- Storage: The transition from M.2 form factor to E1.S was made. Servers are equipped with two 8 TB NVMe units, totaling 16 TB of storage. This change allows for greater capacity and improved cooling.
Network and Connectivity
Upgrades were made to OCP 3.0 compatible network cards, using Intel Ethernet Network Adapter E810 and NVIDIA Mellanox ConnectX-6 Lx adapters. These cards offer more MSI-X queues, which can be assigned to the higher number of cores of the AMD EPYC 9684X CPU. Additionally, they provide offload support improvements, reducing the load on the kernel and enhancing performance.
Security Improvements
Gen 12 servers integrate Project Argus, one of the first implementations of Data Center Secure Control Module 2.0 (DC-SCM 2.0). This decouples server management and security functions from the motherboard. The BMC, HRoT, Trusted Platform Module (TPM), and dual flash memories for BMC/BIOS are installed in the DC-SCM.
These updates make Gen 12 servers more secure and resilient against firmware attacks.
Prepared for AI and the Future
With the increasing demand for AI and ML workloads, Cloudflare has designed the Gen 12 server to accommodate more powerful GPUs, supporting their Workers AI service. This allows for strategically deploying GPUs in key regions to reduce latency for their clients.
Looking Ahead
Gen 12 servers are already deployed and operational in multiple Cloudflare data centers worldwide, processing millions of requests per second. The company is already exploring the next generation, with AMD EPYC 5th generation CPUs (codenamed "Turin") available for testing and Gen 13 server architecture planning.
Source: CloudFlare