Enterprises orchestrating multiple AI models in the hope of covering each other's blind spots may be falling into a mathematical trap. A new study evaluating 67 frontier models from 21 providers reveals that the rate of simultaneous failure, known as the co-failure ceiling, is systematically underestimated by a factor of 2.25 when relying on traditional pairwise correlation metrics.
The co-failure ceiling: when all models fail together
The core idea behind multi-model orchestration is simple: combining a coding specialist, a logic specialist, and a generalist model should create a safety net. However, lead author Josef Chen explains that the real limit is not how often models disagree, but the percentage of prompts where every model in the pool gives the wrong answer at the same time. This limit is called the co-failure ceiling, and no router, voting system, or cascade can surpass it.
Sponsored Protocol
Hidden costs of the multi-model strategy
To orchestrate multiple models, developers typically use three architectures: routers, cascades, and Mixture-of-Agents (MoA). Each introduces hidden costs such as added latency, infrastructure complexity, and governance risks across multiple API providers. The study shows that engineers rely on pairwise error correlation to select their model pool, but this metric is misleading. When models have unequal quality, weaker ones tend to band together and outvote the smarter model, causing an average accuracy drop of up to 10 points.
The free test to calculate true accuracy ceiling
Fortunately, teams can calculate their maximum accuracy ceiling for free using the Clopper-Pearson bound. With a sample of 200 complex support tickets, a fintech company can estimate that if all models fail together on only 2 prompts, the true co-failure rate could be as high as 12%. This zero-cost test prevents investing in expensive routing infrastructure to chase non-existent performance gains. As Chen notes, "the measurement costs nothing, and any team can track its own co-failure rate as new models drop."
Sponsored Protocol
The study also found that task format affects co-failure: switching from multiple-choice to free-response questions increased the co-failure rate to 12.7%. The recommendation is clear: convert generation into verification or constrained selection (structured outputs, verifiable answers, execution tests) to reopen the ceiling. In ceiling-bound environments like open-ended math, co-failure is high and no orchestration helps; in realizability-bound environments like graduate-level science, the challenge is picking the right model among disagreeing experts.
Sponsored Protocol
For more on how frontier models are currently being deployed, read our article on OpenAI GPT-5.6 Sol, Terra, and Luna. For a broader perspective on global tech strategies, see China Eyes Nvidia Chips, US Bets on Nuclear.
For an authoritative reference on language models, visit Wikipedia Language Model.
