The data centers that make generative AI products like ChatGPT possible will soon reach size limits, according to Microsoft Azure Chief Technology Officer Mark Russinovich, necessitating a new method of connecting multiple data centers together for future generations of the technology.

The most advanced AI models today need to be trained inside a single building where tens (and soon hundreds) of thousands of AI processors, such as Nvidia’s H100s, can be connected so they act as one computer.

But as Microsoft and its rivals compete to build the world’s most powerful AI models, several factors, including America’s aging energy grid, will create a de facto cap on the size of a single data center, which soon could consume multiple gigawatts of power, equivalent to hundreds of thousands of homes.

Already, some parts of the national grid are overwhelmed on hot days, when air conditioners are running full blast, forcing rolling blackouts and brownouts.

Microsoft has been working furiously to help add capacity to the grid, inking a deal to reopen the Three Mile Island Nuclear power plant, launching a $30 billion fund for AI infrastructure with BlackRock and inking a $10 billion deal with Brookfield for green energy, among other projects.

Overhauling the US’ energy infrastructure was a big part of the 2022 Inflation Reduction Act, which provided $3 billion in incentives for building out transmission lines, among other priorities. But companies like Microsoft can’t afford to wait around for more money from Washington, on top of the time it would take to deploy those funds.

Microsoft has also innovated on how GPUs are utilized to help data centers run more efficiently.

Given their AI ambitions, a solution could be building data centers in multiple locations to avoid overloading any one region’s power grid. It would be technically challenging, but it may be necessary, Russinovich told Semafor.

“I think it’s inevitable, especially when you get to the kind of scale that these things are getting to,” he said. “In some cases, that might be the only feasible way to train them is to go across data centers, or even across regions,” he said.

Connecting data centers that are already pushing the limits of modern computer networking will be no small feat. Even linking two of them is a challenge, requiring fiber optic speeds that, until recently, were not possible over long distances. For this reason, Russinovich said it is likely the data centers would need to be near each other.

He wasn’t sure exactly when the effort would be required, but it would involve several Microsoft teams as well as OpenAI. It could be years before the effort is necessary. “I don’t think we’re too far away,” he said.