Enterprise AI is facing an ROI paradox. While throwing more compute at the strongest foundation model works well in product experiments, the costs become unbearable when the product is deployed in production. A new paper from researchers at Writer provides a concrete solution accessible to engineering teams. The study systematically examines optimizing the different components of the orchestration layer that wraps around the foundation model, known as the AI harness.
The Tokenmaxxing Crisis in AI Systems
The practice of tokenmaxxing, where developers rely on massive context windows and brute-force token consumption as a substitute for good system design, is causing an ROI crisis. Waseem AlShikh, CTO and co-founder of Writer, told VentureBeat that teams tokenmax because it is the cheapest fix in the moment, but dropping per-token prices mask underlying inefficiency. Unmanaged agentic loops cause cost per task to compound exponentially, negating any savings. By optimizing the harness, the researchers demonstrated a 38% reduction in tokens per task, a drop in cost-per-successful-task by up to 61%, and steady quality, all without changing the underlying model.
Sponsored Protocol
Harness Levers for Efficiency
The harness, the orchestration layer that routes, formats, and turns the LLM into a working system, has become a first-class software artifact. Key intervention points include system prompt caching, interaction history compaction, tool management, retrieval strategies, and error management. Historically, developers treated the harness as disposable glue code, but the study signals it must now be treated as a primary artifact requiring its own testing, versioning, and rigorous design. Whoever owns the harness owns the unit economics of the application. Engineering teams can apply these findings to build highly cost-efficient AI applications without model fine-tuning.
Sponsored Protocol
Experiments Across Six Foundation Models
To isolate the impact of the orchestration layer, researchers ran experiments on six foundation models spanning multiple vendors and weight classes: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Writer's own Palmyra X6. They compared a frozen conventional production agent loop against the optimized Writer Agent Harness on the same 22 locked enterprise tasks. The results were significant: blended cost per task dropped 41%, from $0.21 to $0.12. Tokens per task fell 38%, from 14,200 to 8,800. Task success rates held steady, moving from 78% to 81%, a gain the researchers describe as directional. End-to-end latency also improved, with median wall-clock time dropping 44%, from 48 seconds to 27 seconds.
The harness is designed to delegate tasks like search to specialized sub-agents. A sub-agent receives only the tool and the specific query, retrieves exact data, and returns a clean summary to the main agent, keeping the primary context window lean. However, researchers found limits to multi-agent orchestration with smaller models. Gemini Flash 3.5 and Qwen 3.6 scored well below usable reliability (0.45 and 0.42), while only the two strongest models, Palmyra X6 (0.86) and Claude Sonnet 4.6 (0.85), crossed the threshold.
Sponsored Protocol
Actionable Strategies for Harness Optimization
From a practical perspective, the playbook for developers includes implementing the Two-Zone Prompt and Context Offloading. Separating the stable zone (core rules, tool schemas) from the volatile zone (user query, conversation state) enables prompt caching, avoiding paying for the same instructions on every step. Moving history and intermediate artifacts to retrievable storage, pulling back only what is needed for the current step, reduces context bloat. AlShikh emphasizes that the biggest line item in agent spend is not reasoning but re-sending things the model has already seen.
Sponsored Protocol
To prevent uncontrolled agent loops, hard constraints must be enforced at the code level: hard per-task token budgets, generation fencing with caps on steps and tool calls, and failure-spend governance. Never ask the model to police its own spending. Teams should also start tracking Completions Per Million (CPM) to understand true task costs. However, adding structural scaffolding requires the model to be large enough to support it; otherwise, accuracy drops and tokens rise. For further reading, check the internal articles on AI confidence drops and an autonomous agent breaching Hugging Face. The original paper is available on VentureBeat.
