Intel has followed up on AMD's Data Center AI performance claims for its 5th Gen EPYC Turin CPUs, saying 5th Gen Xeon is faster with the right optimizations.
AMD Showed Its 5th Gen EPYC Turin CPUs With Zen 5 Cores Beating 5th Gen Xeon Chips In AI Benchmarks But Intel Says That 5th Gen Leads Against EPYC When Right Optimizations Are Used
[Update - 6/18/24] - AMD has offered us an update regarding the Intel benchmarks and shared that the endnotes did provide the relevant data regarding what hardware and software stack was used in their tests comparing Turin with Intel's 5th Gen Xeon chips. They also state that the "Intel Xeon" platform used the most up-to-date and publicly available software at the time while Intel itself had published figures based on a software optimization library that was released on the 7th of June. Following is the full quote from AMD:
At Computex, AMD highlighted the leadership performance of 'Turin' on AI workloads compared to 5th gen Intel Xeon processors using the most up-to-date, publicly available software available at the time for both AMD and our competitor. The data for “Emerald Rapids” that Intel highlighted in their blog uses a software library that was released on June 7, after our Computex event. It is also important to note the new software stack was only used for the Intel platform and the performance optimizations were not applied to the AMD platform. 4th Gen EPYC CPUs continue to be the performance leader, and we expect “Turin” to remain deliver leadership performance across a broad range of workloads when we launch later this year.
AMD PR to Wccftech
At Computex 2024, AMD officially announced its 5th Gen EPYC CPU family codenamed Turin which utilizes the latest Zen 5 core architecture. The company put out some big numbers against the Intel 5th Gen Xeon family codenamed Emerald Rapids, specifically in AI throughput workloads but Intel has now clarified that those benchmarks were conducted without the proper optimizations for its Xeon family, and with those implemented, even 5th Gen Xeons should easily surpass 5th Gen EPYC Turin in AI performance.
The benchmark that was specifically highlighted by Intel is the Llama2-7B Chatbot which is based around the INT4 inference throughput and was conducted at a 50ms latency. AMD showed that its 5th Gen EPYC CPUs in a 2S (dual-socket) configuration with 128 cores each offer up to 671 tokens/s of performance while Intel's 5th Gen Xeon Platinum 8592+ chips with 64 cores each running in the same dual-socket configuration offered just 125 tokens/s output. That was a huge 5.4x gain for the AMD EPYC Turin CPUs.
Intel states that the benchmarks were conducted without the right software suite for the 5th Gen Xeon SKUs and AMD didn't put out any details regarding the Intel configuration in its footnotes.
The blue team has now conducted its performance benchmarks in the same AI workloads & it looks like the results are radically different.
Using the Intel Extension for PyTorch (P99 Latency), the 5th Gen Emerald Rapids Xeon CPUs can output 5.4x better performance than what AMD showcased. The 686 Tokens/s output exceeds the performance of AMD's 5th Gen EPYC Zen 5 CPUs. This is not only thanks to Intel's software optimizations for Llama2 but also the added AI hardware accelerators within the 5th Gen Emerald Rapids family which provide a nice boost.
Intel doesn't stop there, the company also states that in the other two workloads, Translation, and Summarization, the company saw a 1.2x to 2.3x gain in performance versus the results that AMD had used in its Computex 2024 presentation.
Furthermore, the more crucial aspect is that these benchmarks heavily depend on memory and the next-gen EPYC Turin family will feature a 12-channel DDR5 interface. While Intel's current-gen Xeon "Emerald Rapids" CPUs peak out at 8 DDR5 memory channels, the next-gen Granite Rapids "Xeon 6700P/6900P" CPUs will not only come with the same "Up To" 12-channel memory interface but also feature the same 128 core count with P-Cores and up to 288 E-Cores with the Xeon 6900E family coming in early 2025. The Granite Rapids Xeon 6700P CPUs will be available with up to 86 P-Cores in Q3 2024.
It looks like all major firms that are currently involved in the AI business is very serious about not only their performance claims but also what others are showing off. NVIDIA and AMD engaged in a similar battle a few months ago when the red team announced its MI300X figures against the Hopper lineup and it was a back-and-forth from there onward.
Similarly, on the AI PC side of things, the companies, AMD, Intel, Qualcomm, and NVIDIA are engaged in a very fierce fight where AI TOPs are the talk of the town. With the rising demand for AI across the tech sector, all of these tech firms are trying to attract more customers to use their products and ship as many units to fulfill the ever-increasing need for capable hardware.
