On-Prem is the New Cloud: Why Enterprises are Reversing Stance in the Age of AI Remember when cloud was the battleground for innovation, agility, and scale? Everyone was tracking the % of on-prem compute moving to the cloud and it was growing every year? Well now there’s a reversal. The reality today? On-prem is the new cloud. Here’s why. AI’s Dark Side: Legal Risk and Data Governance Headaches Enterprises are excited about AI, but also uneasy as vendors like OpenAI face intensifying legal pressure. In a high-stakes lawsuit, a judge ordered OpenAI to preserve all ChatGPT chats (even deleted ones), overriding its 30-day deletion policy. Imagine that: you delete sensitive conversations, but they’re still sucked into legal limbo. The precedent is clear: AI prompts and outputs are discoverable records, demanding integration into enterprise ESI (Electronic Stored Information) policies. For large organizations handling sensitive data, the risk is simple: you can’t afford your AI tools to turn into legal liabilities. Especially when global regulations (GDPR, etc.) and internal governance collide with unpredictably broad discovery orders. Sovereignty, Control, Trust Many enterprises still ban ChatGPT or impose strict rules on usage. Even when allowed, token limits are throttled. Enterprises are cautiously letting AI in (as they should). Enterprises need to have control and trust. SAP is leaning into this with its Sovereign Cloud On-Site, deployable on customer premises with SAP-managed infrastructure. It is built to deliver data, operational, technical, and legal sovereignty in one package. Doubt it will be the last. Scaling Pains Talking to founders, three reasons stand out why cloud isn’t scaling with AI: 1/ The rate at which AWS/GCP lets you provision compute and the compute it wants you to provision both falter as AI and AI driven developers deploy a lot more code more frequently than before. So sometimes it’s not about data governance, but actually about performance and functionality. 2/ Shadow AI is rampant—employees paste sensitive info into models despite restrictions—creating demand for better permissioning tools. 3/ Data leakage remains a live risk, like the August 2025 Grok incident where hundreds of thousands of private chats became publicly accessible and indexed. One thing is for sure - founders selling into the enterprise are scaling faster if they offer the ability to run on-prem. The Future is Hybrid Don’t get me wrong - cloud is not going anywhere. But neither is on prem. A few years agoI thought we were headed for a full cloud shift, but now I think the future is truly hybrid.
Why Enterprises are Choosing On-Prem for AI
More Relevant Posts
-
Impressive collaboration between Deutsche Bank and Google Cloud. By developing db-Lumina, an AI-powered research agent, they are empowering their financial analysts to generate reports in minutes instead of hours. This is a powerful example of how AI can enhance efficiency and drive a competitive edge in the fast-paced world of finance, all while upholding strict data privacy. #ArtificialIntelligence #FinancialServices #DigitalTransformation #Gemini #VertexAI https://lnkd.in/e9t2y_T7
-
SAP and OpenAI partner to launch sovereign ‘OpenAI for Germany’ SAP SE and OpenAI announced 24.09.2025 the launch of OpenAI for Germany, a partnership to bring SAP’s enterprise applications expertise and OpenAI’s leading AI technology to Germany’s public sector. To ensure sovereignty, OpenAI for Germany will be supported by SAP’s subsidiary Delos Cloud, running on Microsoft Azure technology. The collaboration will enable millions of public sector employees to use AI safely and responsibly while meeting strict data sovereignty, security, and legal standard. https://lnkd.in/eZQ2Dsbg https://lnkd.in/etem9uCf
-
Seekr launches SeekrFlow Enterprise AI Platform in AWS GovCloud, providing federal agencies with secure, one-click access for agentic AI in sovereign cloud environments. "As customers accelerate AI adoption, they face mounting requirements to protect Controlled Unclassified Information (CUI), handle sensitive data, and enable accurate and explainable AI," said Rob Clark, President of Seekr. Read the full news: https://lnkd.in/dzVp54J3 #SeekrFlow #AWSGovCloud #AgenticAI #FederalAI #DoDInnovation #SecureAI #CDAO #TechIntelPro
-
ProCogia unveils Data Science in a Box, enabling secure generative AI deployment across Amazon Web Services (AWS), Microsoft Azure, and Google Cloud in just one week - https://lnkd.in/gVCkJsZX "Our clients want the speed of LLM adoption without sacrificing governance or security," said Brian Carter, Delivery Manager at ProCogia. #GenerativeAI #MultiCloud #AIGovernance #SecureAI #TechIntelPro
-
"To ensure sovereignty, OpenAI for Germany will be supported by SAP’s subsidiary Delos Cloud, running on Microsoft Azure technology." It would have been funny but they are saying that with a straight face.
Adjunct Lecturer (2022-08) - Stanford Engineering :: European|Asian Innovation Ecosystems :: Industrial AI • Semiconductors • Displays • Space :: IxD UX DesignT :: Innovation Sociology :: ST&I Policy :: Long-COVID Rehab
“Sovereignty” ?? Or lipstick on a pig?? “WALLDORF, Germany / SAN FRANCISCO, CA – 24 September 2025 – SAP SE and OpenAI today announced the launch of OpenAI for Germany, a partnership to bring SAP’s enterprise applications expertise and OpenAI’s leading AI technology to Germany’s public sector. To ensure sovereignty, OpenAI for Germany will be supported by SAP’s subsidiary Delos Cloud, running on Microsoft Azure technology. The collaboration will enable millions of public sector employees to use AI safely and responsibly while meeting strict data sovereignty, security, and legal standards.”
-
OpenAI for Germany will be supported by SAP’s subsidiary Delos Cloud, running on Microsoft Azure technology. The collaboration will enable millions of public sector employees to use AI safely and responsibly while meeting strict data sovereignty, security, and legal standards. #OpenAI #SAP #Technology
-
OpenAI for Germany will be supported by SAP’s subsidiary Delos Cloud, running on Microsoft Azure technology. The collaboration will enable millions of public sector employees to use AI safely and responsibly while meeting strict data sovereignty, security, and legal standards. #OpenAI #SAP #Technology
-
One of the biggest stories in the AI space is that OpenAI reportedly inked one of the biggest cloud contracts ever, a $300 billion agreement with Oracle for compute power over five years, beginning in 2027. https://bit.ly/4gqVLcO
-
🚀 Rethinking Enterprise AI: Introducing the LLM Service Stack (LLM-IaaS, LLM-PaaS, LLM-SaaS, and LLM-Marketplace) The enterprise technology world has evolved in service layers: IaaS (Infrastructure as a Service) gave companies on-demand compute and storage. PaaS (Platform as a Service) simplified application development and deployment. SaaS (Software as a Service) delivered ready-to-use apps for end users. Today, as Large Language Models (LLMs) become the core of digital transformation, we need a similar structured model. At STAR I.T Consulting, we’re proposing a new framework: The LLM Service Stack. 🔹 1. LLM-IaaS (Infrastructure as a Service) Just like VMs in the early cloud days, enterprises will consume GPU clusters and AI hardware on-demand to train or run LLMs. Azure, AWS, and Google Cloud already offer H100/A100/TPU clusters. Enterprises can rent this infra instead of investing millions in on-prem GPUs. Example: A hospital rents Azure ND H100 nodes to run a healthcare LLM. 🔹 2. LLM-PaaS (Platform as a Service) Managed platforms where enterprises fine-tune, secure, and deploy LLMs without touching the underlying GPU complexity. Azure AI Studio, AWS SageMaker, and Google Vertex AI are early steps here. Enterprises can build domain-tuned models (HealthcareGPT, FinanceGPT, InsureGPT). Example: A bank uses LLM-PaaS to fine-tune an LLM for fraud detection. 🔹 3. LLM-SaaS (Software as a Service) End-user facing AI applications, powered by LLMs, delivered like Salesforce or Office 365. Example: AI Medical Scribes, AI Legal Assistants, AI Insurance Claims Analyzers. Doctors, lawyers, and insurers just log in and use the app — no GPU knowledge needed. 🔹 4. LLM-Marketplace (Ecosystem Layer) A library of industry-ready LLMs available like software licenses. Enterprises can “purchase” or subscribe to HealthcareGPT, LegalGPT, FinanceGPT the way they license SAP, Oracle, or Microsoft software today. Providers can bundle LLM + Hardware Appliances or deliver via thin clients (Citrix-style publishing) for efficient enterprise rollouts. 🌍 Why This Matters Predictable Costs → License LLMs instead of building from scratch. Compliance Ready → Industry-specific guardrails (HIPAA, GDPR, SOX). Scalable Delivery → Publish AI apps securely to thin clients, even in regulated industries. Enterprise Familiarity → CIOs/CTOs already understand IaaS/PaaS/SaaS — this is the AI-native extension. ⚡ The Future In the same way cloud services redefined IT, the LLM Service Stack will redefine how enterprises adopt AI. LLM-IaaS → Rent GPUs. LLM-PaaS → Tune & deploy models. LLM-SaaS → Use AI apps. LLM-Marketplace → Shop industry-ready intelligence. At STAR I.T Consulting, we see this as the next wave of digital transformation — where every industry gets its own ready-to-use LLMs bundled with hardware, compliance, and enterprise support. #AI #LLM #Cloud #DigitalTransformation #STARITConsulting #Innovation
-
That's what I call a bold move: SAP SE and OpenAI today announced the launch of OpenAI for Germany, a partnership to bring SAP’s enterprise applications expertise and OpenAI’s leading AI technology to Germany’s public sector. To ensure sovereignty, OpenAI for Germany will be supported by SAP’s subsidiary Delos Cloud, running on Microsoft Azure technology. The collaboration will enable millions of public sector employees to use AI safely and responsibly while meeting strict data sovereignty, security, and legal standards. https://lnkd.in/dbwDY-w3
More from this author
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development
The legal discovery risk with AI chats becoming permanent records changes enterprise governance requirements entirely. Shadow AI creating data leakage concerns shows why on-prem control appeals to risk-conscious organizations. How do you balance AI innovation with discoverable records management?