The U.S. grid is unable to handle the immense power and water demands of generative AI.

The rapid growth of data centers due to the AI boom has resulted in a massive demand for electricity to power and cool the servers. As a result, there are concerns about whether the U.S. can produce enough electricity to support the widespread adoption of AI and whether our aging grid can handle the increased load.

Dipti Vachani, head of automotive at Arm, stated that if we do not begin to approach the power issue differently, we will never achieve our dream. The low-power processors offered by Arm have gained popularity among hyperscalers such as Amazon, Microsoft, Oracle, and others due to their ability to reduce power consumption by up to 15% in data centers.

Grace Blackwell, NVIDIA's latest AI chip, is said to run generative AI models on 25 times less power than its predecessor, thanks to the inclusion of Arm-based CPUs.

"Designing for power conservation is fundamentally different from maximizing performance," Vachani stated.

The strategy of reducing power use by improving compute efficiency, commonly known as "more work per watt," is one solution to the AI energy crisis. However, it is not a comprehensive solution.

A Goldman Sachs report states that one ChatGPT query consumes almost 10 times the energy of a typical Google search. An AI image generation can require the same amount of power as charging a smartphone.

The issue of carbon emissions from training large language models isn't novel; studies from 2019 revealed that the carbon footprint of training one such model is equivalent to the entire lifetime of five gas-powered vehicles.

The construction of data centers by hyperscalers to accommodate massive power draw is leading to a rise in emissions. Google's latest environmental report revealed that greenhouse gas emissions increased by nearly 50% from 2019 to 2023 due to data center energy consumption, despite the company's data centers being 1.8 times more energy-efficient than typical data centers. Similarly, Microsoft's emissions rose nearly 30% from 2020 to 2024 due to data center energy consumption.

In Kansas City, the high power needs for Meta's AI-focused data center have caused plans to close a coal-fired power plant to be put on hold.

Chasing power

The number of data centers worldwide is expected to increase significantly by the end of the decade, with the U.S. having the highest concentration. AI is driving this growth, and Boston Consulting Group predicts that demand for data centers will rise 15%-20% annually through 2030. By then, data centers are expected to consume 16% of total U.S. power, which is equivalent to the power used by about two-thirds of the total homes in the U.S. This growth is due in part to the release of OpenAI's ChatGPT in 2022.

In Silicon Valley, CNBC explored a data center to understand how the industry can manage its rapid expansion and where it will obtain the necessary energy.

According to Jeff Tench, Vantage Data Center's executive vice president of North America and APAC, we anticipate that the demand for AI-specific applications will be as high or greater than the demand we've historically seen from cloud computing.

Vantage's data centers can handle more than 64 megawatts of power, which is equivalent to the energy consumption of tens of thousands of homes, as many big tech companies contract with them to house their servers.

Single customers are taking up many of the leased space, and the number of AI applications can grow significantly beyond that into hundreds of megawatts, according to Tench.

In Santa Clara, California, where CNBC visited Vantage, there has been a long history of data center clusters serving data-hungry clients. Nvidia's headquarters can be seen from the roof. Tench stated that there is currently a "slowdown" in Northern California due to a "lack of availability of power from the utilities in this area."

Vantage is building new campuses in Ohio, Texas and Georgia.

Tench stated that the industry is seeking locations with close proximity to renewable energy sources, such as wind or solar, and other infrastructure that can be utilized, whether it is through an incentive program to convert coal-fired plants into natural gas or by exploring ways to offload power from nuclear facilities.

On-site electricity generation is being tested by some AI companies and data centers.

Sam Altman, CEO of OpenAI, has invested in three startups that aim to revolutionize the energy sector. He recently invested in a solar startup that produces shipping-container-sized modules with panels and power storage, a nuclear fission startup that aims to create mini nuclear reactors housed in A-frame structures, and a nuclear fusion startup that seeks to develop fusion reactors.

Microsoft signed a deal with Helion last year to start buying its fusion electricity in 2028. Google partnered with a geothermal startup that says its next plant will harness enough power from underground to run a large data center. Vantage recently built a 100-megawatt natural gas plant that powers one of its data centers in Virginia, keeping it entirely off the grid.

Hardening the grid

Although the aging grid can generate enough power, it often struggles to handle the load due to inadequate infrastructure. The issue lies in the transmission of power from the generation site to the consumption point. One possible solution is to construct hundreds or thousands of miles of new transmission lines.

According to Shaolei Ren, an associate professor of electrical and computer engineering at the University of California, Riverside, the process of implementing smart grid technology is both expensive and time-consuming, and often the cost is simply passed on to residents through an increase in their utility bills.

The $5.2 billion project to extend lines to "data center alley" in Virginia faced opposition from local residents who were against the increase in their bills to finance the project.

One way to minimize failures in the grid's weakest area, the transformer, is by employing predictive software.

According to VIE Technologies CEO Rahul Chaturvedi, all electricity generated in the U.S. must pass through a transformer, with an estimated 60 million to 80 million of them in use.

The average transformer is 38 years old, making them a common cause for power outages. Replacing them is costly and time-consuming. However, VIE has developed a small sensor that can attach to transformers and predict failures, allowing for the redistribution of load to prevent future outages.

Since the release of ChatGPT in 2022, Chaturvedi stated that business has tripled and is expected to double or triple again next year.

Cooling servers down

According to Ren's research, generative AI data centers will need 4.2 billion to 6.6 billion cubic meters of water withdrawal by 2027, which is more than the total annual water withdrawal of half of the U.K.

Tom Ferguson, managing partner at Burnt Island Ventures, stated that the energy intensity of AI is a concern for everyone, but we can solve this issue by being more intelligent about nuclear power. He emphasized that nuclear power is the key to overcoming the fundamental limiting factor of water in AI development.

Every 10-50 ChatGPT prompts can consume the equivalent of a standard 16-ounce water bottle, according to Ren's research team.

Vantage's Santa Clara data center utilizes large air conditioning units for cooling the building without any water withdrawal, which is different from much of that water used for evaporative cooling.

Another solution is using liquid for direct-to-chip cooling.

At Vantage, six years ago, we deployed a design that would enable us to access the cold water loop on the data hall floor, eliminating the need for retrofitting.

On-device AI has been promoted by companies such as Samsung and others as a way to keep power-intensive queries away from the cloud and data centers, thereby conserving power.

The amount of AI we'll have is limited by the capacity of data centers, which may not meet people's aspirations. However, many individuals are working on finding solutions to overcome these supply constraints.