Artificial Intelligence Will Transform the Data Center Industry
Artificial Intelligence Will Transform the Data Centers Industry
ChatGPT quickly captured the imagination of 100 million users and the technology press. Executives and boards are accelerating investments in AI as they focus on catching the next wave and avoid getting left behind as this technology accelerates change. These new applications present the opportunity to be creative and capture value, but also the risk that large capital assets could become obsolete or stranded. They also present the opportunity for new entrants to take share and create substantial value. We are in exciting times that require renewed focus for those making capital allocation decisions.
AI Driving Uptick in Data Center Demand
The AI boom is powered by GPUs, specialized server processors that consume about twice as much power as a typical chip. To train AI models, companies pack thousands of GPUs into data centers and run them at full capacity for weeks at a time, consuming tremendous amounts of electricity. For instance, to train GPT-4 Microsoft and OpenAI used more power than it would take to supply 10,000 homes for two months.(1)
We looked at expectations for Nvidia’s data center GPUs sales for the coming years, and the numbers are staggering: in five years these processors will be using about as much electricity as big states like Illinois or New York.(2) To house these servers, the industry will need to spend about $80 billion on building data centers, or nearly one quarter of data center growth will go to these applications.(3)
[Contact us for detailed projections]
AI Will Transform Data Center Design
Engineers are constantly working to pack GPUs into tighter configurations to improve performance and reduce cost. While a typical data center deployment is 8-10 kilowatts per rack, a standard Nvidia system for AI workloads uses 26kw today and will likely be more than twice that in two years.(4) In addition to higher power density, this change requires operators to rethink cooling systems, as the heat from tight clusters of processors exceeds the limits of traditional air cooling.
Large data center builders are already adjusting their plans. In December, 2022 Meta Platforms paused three in-progress data center projects and is rethinking its data center strategy as the company pivots to designs that support AI applications. Meta/Facebook is reportedly evaluating moving to liquid cooling systems.(5) These systems have been evaluated for over a decade but have not gained mainstream adoption due to the operational cost and risk from bringing liquid into a controlled IT environment.
Independent data center developers need to ensure that their facilities are capable of delivering the power density and cooling required to support both current and future AI workloads, or risk having stranded capital when leases roll over and customers migrate to more modern buildings.
AI Presents Opportunities for New Entrants With Different Skill Sets
As noted above, the design and operation of AI-specific data centers is quite different from traditional facilities. Buildings need to have 4-10x the power density which requires new approaches to power distribution and cooling.
One set of potential entrants are hosting companies with experience running Bitcoin mining facilities. Bitcoin mining is a competitive, capital intensive, commodity business that requires efficiency at every step to remain profitable through market cycles. Developers have built significant capacity optimized for Bitcoin mining’s unique characteristics: high capacity and power density with intermittent power and high network latency. Given these lower specifications, mining facilities can be built for approximately 95% lower capital cost compared to traditional data centers, but lack power redundancy, air handling/cooling, or security that would be considered table stakes for most enterprise users.
Recommended by LinkedIn
Certain AI workloads have similar characteristics to Bitcoin mining: training AI models requires many weeks of running high wattage GPUs with nominal network activity. This realization has led to several Bitcoin hosting companies exploring the opportunity to upgrade their facilities to add the minimal redundancy and security required to attract enterprise and Cloud customers. For instance, Applied Digital (NASDAQ: APLD) was founded as a Bitcoin hosting company, but is rapidly pivoting to build for the High Performance Computing / AI market.
The economics as presented are extremely compelling: capital cost for these purpose-built AI facilities are about half of traditional data centers and total cost of ownership including electricity is approximately 55-60% lower. There remain meaningful questions about the ability of these smaller developers to deliver to the needs of enterprise customers, but they have attracted interest from universities and startups eager to build large AI models at a lower price point.
Additional Areas of Research
We will continue to explore the impact of AI application growth on communications infrastructure. Future topics include:
Potential Winners
Potential Losers
Wild Cards
(1) https://news.microsoft.com/source/features/ai/how-microsofts-bet-on-azure-unlocked-an-ai-revolution/
(2) https://www.eia.gov/state/seds/data.php?incfile=/state/seds/sep_fuel/html/fuel_use_es.html&sid=VA
(3) https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/investing-in-the-rising-data-center-economy
(4) https://www.datacenterfrontier.com/featured/article/11427653/data-center-of-the-future-equinix- test-drives-new-power-cooling-solutions and https://docs.nvidia.com/nvidia-dgx-superpod-data- center-design-dgx-a100.pdf
(5) https://www.datacenterdynamics.com/en/news/metas-kuna-idaho-data-center-latest-to-be-paused-in-ai-redesign-rethink/
Higher densities for sure...and liquid cooling probably required in order to support those higher densities. Have heard some hyperscalers wanting facilities with flexible densities which seems like a big engineering challenge not solved yet. We'll need to see significant innovation in power efficiency (prob at the chip level), cost of chips/equipment and cost to build a DC in order to truly scale the AI industry. If we thought power rationing in Asia and Europe (and now NoVA) was bad before, it's only getting worse so something has to break/change/evolve. Interesting point on Crypto architecture being more efficient.