Nvidia has reported a staggering $14.5 billion in revenue from data center hardware sales during the third quarter of fiscal 2024. The highlight of this financial triumph is the unprecedented success of Nvidia’s H100 GPUs, designed explicitly for artificial intelligence (AI) and high-performance computing (HPC).
According to a recent report from market tracking company Omdia, the tech giant has sold nearly half a million H100 GPUs, marking a significant milestone in the realm of advanced computing. The soaring demand for Nvidia’s H100 GPUs is attributed to major players in the tech industry, with Meta and Microsoft emerging as the largest purchasers.
Omdia reveals that both Meta and Microsoft have acquired an impressive 150,000 H100 GPUs each, surpassing the procurement of other industry giants such as Google, Amazon, Oracle, and Tencent, who secured 50,000 units each.
The overwhelming demand has resulted in an extended lead time for H100-based servers, ranging from 36 to 52 weeks, underlining the industry’s struggle to keep pace with the fervor for advanced AI and HPC capabilities.
Omdia’s analysis sheds light on the pivotal role played by hyperscale cloud service providers, who constitute the primary recipients of the majority of server GPUs. This has created a challenging landscape for server Original Equipment Manufacturers (OEMs) like Dell, Lenovo, and HPE, as they grapple with fulfilling their orders amid the escalating demand for AI and HPC GPUs. The report suggests that server OEMs are facing a significant shortfall in meeting the burgeoning market requirements.
The report further anticipates that the combined sales of Nvidia’s H100 and A100 compute GPUs will surpass the half-million mark in the fourth quarter of 2023. This projection is a testament to the sustained and robust demand for Nvidia’s cutting-edge GPU technology in the global market.
Simultaneously, the report highlights the extended lead time for GPU servers, indicating a waiting period of up to 52 weeks, underscoring the challenges faced by companies striving to incorporate these advanced technologies into their infrastructures.
Despite the surge in demand, Omdia’s report presents a nuanced perspective, noting a decline in server shipments for 2023, projected to range between -17% and -20% year-over-year. However, server revenue is expected to see positive growth, ranging between +6% and +8% year-over-year. This dynamic reflects the industry’s transition and recalibration in response to the changing landscape of data center hardware requirements.
(image credit: Omdia)
An intriguing aspect highlighted by the report is the dual nature of companies purchasing Nvidia’s H100 GPUs in large quantities. While these enterprises are driving the current surge in demand, they are simultaneously investing in the development of custom silicon for AI, HPC, and video workloads. Omdia suggests that this dual strategy may lead to a gradual decline in their reliance on Nvidia hardware as they transition to proprietary chips, signaling a transformative phase in the industry’s hardware landscape.
Looking forward to 2027, Omdia paints a compelling picture of the server market, estimating its value to reach a staggering $195.6 billion. This remarkable growth trajectory is attributed to a paradigm shift toward servers tailored for specific applications, featuring a diverse array of co-processors.
Examples include Amazon’s AI inference servers with 16 Inferentia 2 co-processors and Google’s video transcoding servers equipped with 20 custom VCUs. Meta is also actively participating in this trend, deploying servers with 12 custom processors designed for video processing. Nvidia’s exceptional success in selling half a million H100 AI GPUs in Q3 2024 underscores the insatiable appetite for advanced computing capabilities. This, coupled with the industry’s evolving landscape and the imminent rise of custom silicon, sets the stage for a transformative era in data center hardware.
Related:
Is Nvidia G Sync Worth It? [2023]
Nvidia is hinting more VRAM for 3060 Ti and 3070 Ti to encounter VRAM deficiency in Next-Gen GPUs
Journalist
