A VentureBeat survey of 107 enterprises with over 100 employees uncovers a critical compute gap: organizations are purchasing AI infrastructure at a pace that far outstrips their ability to monitor and control actual costs. This phenomenon, termed the AI compute gap, poses a major challenge for AI governance.
Production AI adoption remains a minority
Only 21% of surveyed enterprises run AI workloads in production at scale. 76% are still experimenting or running only some workloads in production. This matters because infrastructure decisions are being made by organizations early in deployment, meaning their compute footprint and costs are about to grow.
Current providers are hyperscalers and APIs, but the future points to specialized clouds
Today, the vast majority use Google Cloud (48%), Microsoft Azure, AWS, and major model APIs (Gemini, OpenAI, Anthropic). Specialized GPU providers like CoreWeave, Lambda, or Crusoe have almost zero presence. Yet 45% of enterprises plan to evaluate AI-specialized clouds in the next twelve months, a category they barely use today. This signals a clear re-platforming intent.
Sponsored Protocol
64% of enterprises will switch providers within a year
A striking finding is the high churn intent: 64% plan to switch or add an infrastructure provider within twelve months, and 38% within the next quarter. Although new entrants attract interest, most near-term movement involves reshuffling among incumbents: Microsoft Azure and Google Cloud top the switching consideration list, followed by OpenAI and Gemini.
Token price is not decisive; integration and total cost of ownership matter
When choosing a provider, enterprises prioritize integration with the existing stack (41%) and total cost of ownership (35%). Only 8% consider cost per million tokens as the deciding factor. This is rational, but a contradiction emerges: the same enterprises that cite TCO as key often cannot measure it accurately.
Sponsored Protocol
Underutilized GPUs and loose accounting
83% of enterprises operating GPUs report utilization at 50% or less, and 49% run at 25% or below. Fewer than half (44%) rigorously track their AI compute costs. Expensive GPUs sit idle most of the time while investment continues without clear economic visibility. As noted in a recent article on AI security risks, Jamie Dimon compared Anthropic's Claude Mythos AI to ballistic missiles, highlighting challenges in governing fast-evolving technologies.
The next bottleneck: from compute power to memory bandwidth
As large-scale inference grows, the constraint is shifting from GPUs to memory capacity (KV-cache). However, about one in five enterprises does not yet recognize this shift or has not started addressing it. For IT professionals, understanding these dynamics is essential: approaches like Infrastructure as Code with Terraform can help manage complexity, but economic visibility remains the Achilles' heel.
Sponsored Protocol
The VentureBeat study, conducted in June 2026, offers a directional but consistent sample: enterprises are spending on AI infrastructure faster than they can measure it. Closing the AI compute gap requires not just more hardware, but better cost monitoring and governance tools. For further reading, see the definition of GPU on Wikipedia.
