How AI Models Affect Infrastructure Requirements

AI’s Biggest Bottleneck Isn’t Code. It’s Concrete, Copper, and Cooling. Let’s get real for a second. Everyone’s obsessed with the next big AI model, but almost nobody wants to talk about the hard limits: Power. Heat. Space. You can’t ship intelligence if you can’t plug it in. According to Goldman Sachs, global data center power demand is set to rise 165% by 2030, with AI workloads as the primary driver. https://lnkd.in/gKcsRuxj In major regions, data center vacancy rates are below 1%. That means, even if you have the hardware and the talent, your biggest challenge is often finding enough megawatts, enough cooling, and enough floor space to actually run your workloads. From my vantage point—deploying AI at scale—the constraints are physical, not theoretical. Every breakthrough in model design gets matched by an even bigger jump in energy and cooling requirements. No grid, no cooling, no go. What’s shifting right now? Direct-to-chip and immersion cooling are turning waste heat from a liability into an asset, doubling compute density per rack. Infrastructure leaders are designing for sustainability and modular deployment—not just patching legacy hardware. The next leap in AI won’t come from a new algorithm. It’ll come from infrastructure that’s actually ready for it. Here’s my challenge to every operator, investor, and AI team: Are you tracking your megawatts and thermal loads as closely as your training parameters? Are you planning for true density, or just hoping the power and space show up? Bottom line: The future of AI will be won by teams who master both the software and the physical world it runs on. Code matters. But so does concrete, copper, and cooling.

The Unpriced Infrastructure Bet Most bank CTOs I have spoken to have an AI transformation plan. Almost none of them have modelled what happens when the infrastructure underneath it gets constrained, contested, or repriced Here’s why that matters Hyperscalers are on track to spend $600-690 billion on infrastructure in 2026 alone - 75% of it earmarked for AI. That’s more than the entire US interstate highway system cost to build, in a single year. Your AI roadmap - every copilot deployment, every fraud model, every agentic workflow - depends on that buildout going to plan A new Arthur D. Little Blue Shift report, AI’s Hidden Dependencies, based on 50+ expert interviews, maps exactly where the vulnerabilities sit. Three stand out for financial services: 1️⃣ The energy constraint is physical, not financial. Data center electricity hit 415 TWh in 2024 and is projected to approach 1,000 TWh by 2030. In hubs like Virginia and Dublin, data centers could consume 30-40% of local electricity within five years. Grid connection queues already stretch to seven years. Your cloud provider’s capacity plan is only as good as the substation it connects to 2️⃣ Inference costs are the iceberg. Training gets the headlines, but inference - the actual running of models accounts for emissions an order of magnitude higher than training. As banks scale from chatbots to agentic AI, a single human query can trigger hundreds of background model calls. ADL’s own modelling shows that shifting to agentic architectures could multiply energy consumption by 50x 3️⃣ Compute concentration is a geopolitical risk. Nvidia, ASML, and TSMC dominate the upstream AI supply chain from just three countries. Cloud remains a persistent choke point. JP Morgan Chase has already reclassified AI spending as core infrastructure - alongside payments systems and risk controls. That’s not symbolism. That’s a bank pricing in dependency risk The report outlines four scenarios, from a bubble burst to full-blown compute wars. In none of them does AI get cheaper without consequence If you’re a bank CTO building a three-year AI transformation plan, the question isn’t whether AI works, it’s whether you’ve modelled what happens when the infrastructure it depends on gets constrained, contested, or repriced. My colleagues at Blue Shift make a pointed observation: we are not yet paying the true cost of AI What assumptions sit underneath your AI business case that you haven’t pressure-tested? #AIinfrastructure #FinancialServices #DigitalTransformation

Enterprise AI does not succeed because of better models alone. It succeeds because of the infrastructure underneath. Models are only one layer. Real-world AI requires orchestration, compute, networking, storage, observability, security, and cost controls working together as a unified system. This guide breaks down the Enterprise AI Infrastructure Stack (2026) — showing how data, GPUs, pipelines, serving, monitoring, governance, and optimization come together to move AI from experiments into reliable production systems. Here’s what’s actually happening under the hood: - Platform & Orchestration Coordinates containers, workloads, and ML pipelines so training and inference scale across clusters. - Distributed Compute & Scheduling Manages GPU-heavy workloads, batch jobs, and large-scale preprocessing with predictable performance. - Networking & GPU Communication Enables low-latency data transfer between nodes so models train faster and serve responses in real time. - Storage & Data Access Powers high-throughput access to datasets, embeddings, checkpoints, and feature stores. - Model Serving & Inference Deploys models efficiently, scales traffic dynamically, and keeps latency under control. - Experiment Tracking & MLOps Tracks runs, versions models, compares metrics, and makes results reproducible. - Observability & Performance Monitors GPU usage, latency, drift, and system health before issues impact users. - Security, Governance & Access Applies role-based access, secrets management, audit trails, and compliance by default. - Cost Management & Optimization Keeps GPU spend visible, prevents resource waste, and aligns infrastructure with business outcomes. Key takeaway: Enterprise AI is a systems problem - not a model problem. Winning teams don’t just pick tools. They design end-to-end platforms that balance scale, reliability, security, and cost from day one. If you’re building production AI, think in stacks - not shortcuts.

The Largest AI Data Center Ever Built Changes the Rules of Infrastructure Meta is building what may become the largest AI data center campus ever constructed. The project is called Hyperion, and its scale forces the industry to rethink how AI infrastructure is designed. At full buildout the campus is expected to span roughly five miles long and one mile wide and support multiple gigawatts of compute capacity dedicated primarily to training large-scale AI models such as the LLaMA family. What makes Hyperion different? 1. Compute at unprecedented scale The campus is expected to host tens of thousands of high-density AI racks built around NVIDIA’s next-generation GPUs along with Meta’s custom MTIA silicon. Meta is moving toward a hybrid model: • NVIDIA GPUs for large-scale model training • Meta-designed MTIA chips for optimized inference workloads Owning both hardware and software layers allows Meta to control performance, cost, and supply chain risk in ways that renting compute cannot. ⸻ 2. Energy infrastructure at gigawatt scale AI training clusters are now pushing power requirements into territory previously seen only in heavy industry. To support the campus, Meta and regional utilities are planning new large-scale generation capacity, including natural gas and renewable sources. The reality is becoming clear: Frontier AI development now requires energy infrastructure on the scale of power plants. The companies building the largest models are rapidly becoming energy planners and grid partners. ⸻ 3. Cooling and water demand Hyperscale AI campuses require enormous thermal management capacity. Liquid cooling, large chilled-water systems, and heat rejection infrastructure must handle massive continuous loads from dense GPU clusters. But the real water footprint of AI infrastructure is often misunderstood. It’s not only the data center itself. The power generation supporting the facility can consume far more water depending on the cooling technologies used by those plants. This is where AI infrastructure begins intersecting directly with regional water planning and energy policy. ⸻ What Hyperion signals about the future of AI Two major shifts are now visible. 1. AI development is becoming infrastructure-driven The competitive advantage is no longer just algorithms. It is who can build and operate the largest, most efficient compute infrastructure. ⸻ 2. The next AI race will be about energy Training frontier models now requires: • gigawatt-scale power • advanced cooling technologies • massive capital investment • deep integration with utilities and energy markets The companies leading AI will increasingly look like energy companies and infrastructure developers. ⸻ The age of AI supercomputers has arrived. And they are being built at a scale the digital infrastructure industry has never seen before. ⸻ #AIInfrastructure #DataCenters #ArtificialIntelligence #Hyperscale #EnergyTransition #DigitalInfrastructure

A federal court recently ruled that documents created using the AI system Claude were not protected by the attorney-client privilege. For enterprise leaders deploying AI, this ruling signals that infrastructure design is becoming inseparable from governance and risk. In United States v. Bradley Heppner, the court found that confidentiality could not be assumed when AI inputs might be retained by a third-party provider. For CTOs and CIOs building enterprise AI systems, the implication goes beyond one case. As AI moves into core workflows, architectural decisions now influence legal exposure, governance obligations, and operational risk. As CEO of an infrastructure company that operates AI environments at scale for enterprises, I see this shift every week. The constraint is no longer model capability. It is architecture, specifically where AI runs and how it is governed once agents become operational. When AI moves from chat to autonomous agents acting on sensitive enterprise data, the exposure surface expands quickly. It is no longer just documents hitting an endpoint. It is prompts, traces, tool calls, and decision logic. In real deployments, this quickly becomes millions of interactions forming a new stream of high-value intellectual property. If that data crosses your trust boundary, you introduce confidentiality, compliance, and legal exposure. The cost model shifts as well. Always-on agents turn token usage into an infrastructure line item. In the wrong architecture, inference spend becomes unpredictable when leadership wants financial control. This is why the private cloud renaissance is real. Not because cloud is going away, but because enterprises are re-calibrating where trust and control must live. Leaders getting this right focus on three layers: - Trust boundary. Where sensitive agent data persists and who controls it. - Control plane. The governance, monitoring, and policy enforcement over agent execution. - Cost discipline. Guardrails that prevent inference from scaling faster than business value. You do not have to abandon the cloud operating model. But you do need to decide which workloads require tightly governed environments, whether private cloud, sovereign infrastructure, on premises, or isolated inference. AI is becoming operational infrastructure. When something becomes infrastructure, the standard changes. You optimize for control, accountability, and economic discipline. This is exactly the shift we are building for at Rackspace Technology as enterprises move from AI experiments to operational systems. The leaders moving fastest are not chasing models. They are redesigning where AI runs before scale, cost, legal discovery, or a board question forces the decision. Where are you drawing the trust boundary for your AI agents today?

Venture capital and media attention fixate on foundation model capabilities, but the competitive battleground in AI has shifted to the unsexy, boring parts of AI - things like orchestration layers, retrieval systems and connective infrastructure. Organisations do not deploy “a model”. They deploy workflows integrating models with proprietary data, existing software systems, human review processes, compliance controls and operational monitoring. The sophistication of this second-order infrastructure increasingly determines who wins in AI deployment. The Model Context Protocol exemplifies this shift. By providing a standardised interface for AI systems to connect with external tools and data sources, MCP solves the “M times N” problem that plagued earlier integration efforts. Connecting M models to N tools previously required M times N custom integrations, each demanding bespoke engineering, testing and maintenance. MCP reduces this to M plus N by providing a common protocol. The seemingly technical detail of interoperability standards enables the ecosystem effects that allow agentic AI to scale across organisations and use cases. Retrieval-Augmented Generation represents another critical infrastructure layer. Generic models know only what appears in their training data. Enterprise value requires grounding AI responses in current, proprietary organisational information. RAG systems retrieve relevant context from document stores, databases and knowledge graphs, then inject that context into the model’s reasoning process. The engineering required to make this work reliably encompasses vector databases, embedding models, semantic search, ranking systems, access controls and cache management. These components are invisible to end users but determine whether an AI system produces valuable insights or expensive nonsense. The orchestration market has grown explosively as organisations recognise that managing multiple specialised models and tools requires sophisticated coordination. Rather than forcing every query through a single expensive frontier model, orchestration systems route requests intelligently. Simple queries go to fast, cheap models. Complex reasoning tasks go to sophisticated models. Specialised tasks go to fine-tuned domain models. This arbitrage across model capabilities and costs determines the unit economics of AI deployment. These systems sit between enterprise users and external AI providers, enforcing usage policies, managing costs, logging interactions for audit and blocking potentially harmful outputs. Deploying AI without a gateway has become as negligent as deploying web servers without firewalls. The governance, compliance and risk management capabilities embedded in these infrastructure layers determine whether enterprises can scale AI deployment while maintaining controle. The companies building superior connective tissue will matter more than those training marginally better models.

As I continue sketching my “AI in 2026” observations, this one keeps surfacing in boardrooms: Autonomous AI forces a decision about sovereignty. Once AI systems act continuously inside core workflows, questions that once sat in the background move to the foreground very quickly: - Who controls the model? - Where does the data live? - Who has access in transit and at rest? - What jurisdiction governs failures, audits, or disputes? - What happens if an external provider becomes unavailable, restricted, or non-compliant? These are now baseline design questions. This is why sovereign AI has moved from policy discussions into enterprise architecture. For governments, this shows up as a geopolitical concern. For enterprises, it shows up as operational and legal exposure. In both cases, dependence on externally controlled AI infrastructure becomes more consequential once systems are embedded deeply enough to affect outcomes. As AI agents become persistent, they generate decisions, actions, and institutional memory. Data sovereignty expands beyond storage into behavior, accountability, and control over outcomes. Edge-native deployment fits naturally into this picture. In regulated industries, critical infrastructure, healthcare, manufacturing, and logistics, organizations are placing inference closer to where data is generated and decisions are made. Local execution reduces dependency on external networks, limits data movement, and simplifies governance boundaries. Energy efficiency enters through the same operational path. Persistent agents run continuously. Over time, energy usage and inference cost surface directly in operating models. Smaller, specialized models running locally become common. Larger models are used selectively, where cost and risk are justified. What emerges is a new deployment reality. Autonomous systems operate across a continuum: edge environments, private infrastructure, regional hubs, and centralized platforms. Each layer is chosen deliberately based on jurisdiction, cost, latency, and control requirements. By 2026, these choices shape competitiveness. Organizations that treat sovereignty, locality, and efficiency as first-order design inputs gain resilience and flexibility. Organizations that assume AI infrastructure will remain centralized, inexpensive, and universally accessible encounter constraints after systems are already embedded. AI strategy increasingly becomes a question of infrastructure, governance, and geopolitical alignment.

Telcos Will Build Networks for AI, Not Humans By 2030, autonomous AI agents are expected to exceed one billion active instances globally. These agents will interact with APIs, execute transactions, perform inference at the edge, and operate continuously without human intervention. In parallel, AI-generated content is projected to surpass human-generated content in volume across digital platforms. This shift alters the operating assumptions of human mobile networks. Agents require deterministic latency, persistent low-jitter sessions, verifiable identity, and secure orchestration. Their traffic is structured, continuous, and increasingly upstream heavy. Interaction is machine-to-machine, with growing demands for localized compute and context-awareness. Traditional user-based billing models and best-effort routing are not the best for this profile. Telco infrastructure must expose core capabilities as programmable services. Network slicing must prioritize agent-critical traffic. eSIM and IMSI infrastructure must issue and verify agent identity. Edge compute nodes must support real-time model inference. Session orchestration must scale to persistent A2A and B2A traffic patterns. ( Agent-to-X economics). Foundation model providers and AI platforms will require what Telcos uniquely provide: proximity, mobility context, deterministic routing, and national compliance. The network will shift from moving packets to executing intelligence. Operators that adapt will become essential infrastructure in the AI stack.

The phrase “AI-ready infrastructure” appears everywhere today. But what does it actually mean? It’s too often used as marketing language rather than a reflection of real capability. AI workloads place fundamentally different demands on infrastructure compared to traditional enterprise IT. Higher compute density, significantly greater power requirements, advanced cooling systems, high-speed networking, and storage architectures capable of handling enormous data volumes are all part of the equation. Preparing for AI doesn’t equate to adding more servers. Rather, it requires rethinking how facilities are designed and operated. It also requires something less visible but equally important: the ability to run these environments with discipline, reliability, and skilled teams. AI infrastructure is defined by whether the systems, facilities, and people behind it can support the scale and complexity that AI truly demands. Being labelled as AI-ready is not quite the same as being engineered to call yourself so.

U.S. Data-Center Buildouts Are Running Into Structural Limits The latest figures from MSCI Real Assets (via JPMorgan) show an extraordinary surge in U.S. data-center construction and planning: more than 80 gigawatts of capacity are now in the pipeline — a scale unimaginable even five years ago. But what stands out most in the chart is not the growth. It is the stalling. A meaningful portion of planned capacity is now classified as stalled, reflecting growing bottlenecks in land availability, permitting, electricity supply, and utility interconnection queues. The U.S. grid cannot expand fast enough to meet hyperscaler demand, and power-delivery timelines in several key states are now measured in years, not quarters. This has two important implications: 1. The AI buildout may face physical constraints, not just financial ones The industry often focuses on capex and debt levels, but the binding constraint may ultimately be power, not capital. Without grid upgrades, even well-funded projects cannot move forward. 2. AI infrastructure costs could rise faster than expected Stalled projects imply scarcity. Scarcity drives prices. Land, power-purchase agreements, cooling infrastructure, transformers, and even long-lead electrical components are already becoming more expensive. This raises the long-term cost curve for AI training and inference — and increases the risk that companies’ future cost assumptions are too optimistic. In short: AI infrastructure demand is real and accelerating, but the hard limits of the U.S. power grid are becoming impossible to ignore. Source: JPMorganChase and MSCI Inc.