Race for AI - Capex, Energy, Jensanity

In the fast-paced race for artificial intelligence (AI) dominance, hyperscalers, and large tech companies face growing pressure to manage escalating capital expenditures (CapEx) while struggling with the challenge of securing the energy needed to meet surging demand. As AI continues to evolve and reshape the tech landscape, these companies are investing heavily to stay ahead of the competition. The need for advanced, high-performance infrastructure to support AI development and implementation is driving this race. Companies are pouring billions into building and maintaining massive data centers, acquiring cutting-edge hardware, and finding innovative ways to power these facilities.

Let’s explore the latest trends, financial impacts, and innovations shaping this field.

A) Capex Investments and the Return on Investment (ROI) Question

Beyond the AI hype, CFOs are now increasing pressure to address investor concerns about the return on investment (ROI) for these significant capital expenditures.

Source: Cloud and Hyperscale Capital Expenditures (CapEx) in 2024 - Dgtl Infra

The scale of CapEx investments is staggering. Microsoft, for example, has quadrupled its quarterly CapEx from $5 billion to $20 billion over the last three fiscal years. These investments span four primary areas– land, energy, network, and engineering hardware (racks, computer, chips). Approximately 50% of Microsoft’s CapEx is allocated to land and construction, with the remainder allocated to computing resources, chips, and storage. 

As a result of these massive investments, the data center market is experiencing unprecedented growth, which is reflected in several key trends such as:

1. Very Low Vacancy Rates: The data center industry in North America continues to experience significant growth, driven by increasing demand for digital infrastructure. In the third quarter of 2023, the average vacancy rate across North American data center markets was just 2.7%, with some markets, such as Northern Virginia, having a vacancy rate of about one percent. This low vacancy rate highlights the ongoing trend where demand for data center space is outstripping supply.
2. Booked in Advance: In addition to low vacancy rates, most under-construction data centers are already pre-leased. More than 85% of these facilities have been booked in advance, indicating strong demand and the need for more data center capacity.
3. Construction Spending: The growing demand for data centers is fueling a construction boom across towns in the U.S., Europe, APAC, and other regions. In its recently released, AWS Economic Impact Study (EIS), Amazon Web Services states that since 2011, it has invested over $108 billion into its cloud computing infrastructure in the United States, including, connectivity, maintenance, and operations. Microsoft is spending roughly $4 to $5 billion solely on construction. As a result, these companies are also trying to bring synergy and cost control across DC engineering, construction, and functions like procurement and sourcing to address the high-cost overruns often associated with construction.
4. Private Equity / Investment Action: The data center industry has witnessed significant changes in ownership and investment patterns over the past 18 months. Ownership of four out of the top six U.S. data center operators has changed hands in quick succession. Meanwhile, the two biggest names in the industry, Equinix and Digital Realty, are increasingly turning to joint ventures to help fund their growth. This shift highlights a notable trend where private equity investors are becoming more active than traditional data center operators. Beyond North America, regions such as Europe, APAC, and Australia are also experiencing a surge in data center deal activity. This global trend underscores the growing demand for data center infrastructure and the strategic importance of these facilities in the digital economy.
   • U.S. tech firms, for instance, have announced plans to invest more than $8 billion in U.K. data centers amid the ongoing AI frenzy. According to Britain’s Department for Science, Innovation and Technology, the investment pledges come from several prominent companies, including KKR and Co.-backed data-center operator CyrusOne, cloud-software company ServiceNow, Washington D.C.-headquartered data-center company CloudHQ, and cloud-computing provider CoreWeave.
   • Additionally, Digital Realty, the largest global provider of cloud- and carrier-neutral data center, colocation, and interconnection solutions, and Blackstone Inc. announced a joint venture in late 2023. Together, they plan to develop four hyperscale data center campuses across three metro areas on two continents.
   • Meanwhile, Malaysia is also emerging as a major data center hub in Asia, as demand continues to surge across the region.

5. Firstly, higher CapEx leads to increased depreciation, which directly affects profitability. As companies invest more in building and maintaining data centers, the depreciation of these assets rises, thereby reducing overall profitability.

   Secondly, while most large tech companies have substantial cash reserves, making it easier for them to invest, this expenditure will inevitably influence FCF growth—a key metric for Wall Street and investors. Significant data center investments will affect FCF, which in turn can influence investor sentiment.

(B) The Energy Conundrum: Innovations and Investments in the Data Center Industry 

1. In the end, this arena favors companies with deep pockets. The substantial financial resources required to invest in data center infrastructure create a significant barrier to entry for smaller players in this race. 

In the fast-paced world of hyperscalers, waiting for regulated utilities to catch up is not an option. These companies are determined to control their own power, leading to a surge in energy-related investments, engineering innovations, and strategic deals worldwide.

One of the key trends is the accelerated shift towards cleaner energy sources such as wind, solar, and nuclear power. This shift goes hand-in-hand with redesigning data centers to improve efficiency, incorporating advancements like liquid-cooled systems and direct-to-chip cooling for GPUs, while traditional servers still rely on air cooling. Additionally, there is a focus on designing data centers to handle AI-dedicated versus hybrid workloads, with rack-level density expected to grow 5-10 times compared to traditional data centers.

Innovative concepts like floating data centers on retrofitted oil rigs, are also being explored. Continuous innovation is a hallmark of this industry, with companies constantly seeking more efficient ways to operate. The rise of small modular nuclear reactors (SMRs) is another significant development, reducing reliance on state or government-owned utility providers and enabling direct power connections to data centers.

Private investments and partnerships are playing a crucial role in this transformation. Hyperscalers and tech companies are making venture capital investments in SMRs, microreactors, nuclear fusion, enhanced geothermal, renewable energy, and energy storage startups. These investments help keep hyperscalers at the forefront of energy innovation, reinforcing their leadership in sustainable infrastructure and future-ready power solutions.

(C) The Rise of Custom Chips

Nvidia has been one of the standout winners of this AI race, with demand for their GPUs to power data centers showing no signs of slowing down. Tech companies are eagerly vying to procure as much NVIDIA gear as possible, making Jenson Huang's company one of the most valued firms in the industry. 

A notable development is the rise of xAI and its "100,000" H100 GPU clusters. xAI built its supercomputer, Colossus, from a cluster of 100,000 Nvidia H100 GPUs. This achievement has garnered significant attention, with similar projects underway at Meta and other firms. 

A notable recent development is the debut of xAI's Colossus supercomputer, built with 100,000 NVIDIA H100 GPUs for many major customers. Most tech companies, hyperscalers included, have started to develop their own internal chips for specific needs. Doing so helps reduce energy consumption and can potentially shrink the cost and time to design. Currently, custom silicon design for data centers is dominated by Broadcom and Marvell.

Data center operators have a large appetite for co-processors—microprocessors that complement the primary processor. These co-processors not only increase computing capacity but also improve efficiency. Co-processors like Nvidia H100 and AMD’s MI300 GPUs, Google Cloud’s Tensor Processing Units (TPUs), and other custom application-specific integrated circuits (ASICs) are popular. These components enable AI training and inferencing, database acceleration, the offloading of network and security functions, and video transcoding, expanding computing capacity and improving efficiency in data centers.

Conclusion

In the race for AI supremacy, increasing CapEx is becoming a significant concern for CFOs and CTOs of every cloud service provider. Balancing the need for rapid growth and innovation with the pressure to deliver a solid return on investment is a challenge that these leaders are grappling with as they strive to stay ahead in this competitive landscape.

To navigate these financial waters carefully, it is crucial to allocate resources strategically to achieve organizational goals. This involves making informed decisions about where to invest and how to manage the financial pressures that come with substantial CapEx.

Moreover, it is essential to be mindful of the environmental impact of these changes. Companies must be open to finding new avenues to make this transition sustainable for their business and customers alike. This includes exploring cleaner energy sources, innovative cooling solutions, and other environmentally friendly practices.