At VB Transform 2026, Toby Roberts, SVP of Engineering at Zillow, and Arvind Jain, co-founder and CEO of Glean, detailed the AI architecture they built to handle the complex customer journey in real estate. Zillow's customers interact with multiple professionals over months or years, moving from a phone screen to a loan officer to a real estate agent, expecting the context to follow them. A single chatbot cannot carry that thread. Roberts explained that the hardest problem was not data but creating a persistent context layer that can support customers wherever they are in their journey.
Zillow chose to own the context layer rather than relying on external chatbot interfaces
Roberts noted that Zillow started with a strong data foundation using a data mesh approach, clear data lineage, and governance with permissions and identity attached to the data itself. However, the real challenge was building something that remembers where the customer is and carries that context forward, no matter which surface they show up on next. For this reason, Zillow decided to own the context layer internally instead of depending on a single external chat interface, a decision reached quickly after examining the shape of a real transaction rather than a single conversation.
Sponsored Protocol
AI architecture with small models and Glean integration
Zillow built its own AI harness rather than routing customers through a single model API. The company leveraged 20 years of machine learning history, using smaller, task-specific fine-tuned models instead of a general-purpose model. Internally, that harness runs alongside Glean. Roberts revealed that Zillow now has thousands of Glean agents in production, handling tens of thousands of repetitive tasks across the company. Glean's pitch, according to Jain, is centralizing that integration work once through the Glean MCP gateway, rather than letting finance, legal, and marketing each rebuild their own connections to the same systems. This centralization also acts as a cost lever: model routing sends most tasks to smaller, cheaper models, and precomputed context avoids an agent burning tokens to assemble context from scratch. Jain cited an example where Claude is slow in assembling initial context, while routing through Glean can cut token consumption by half.
Sponsored Protocol
Measure AI ROI before building: Zillow's lesson with DORA metrics
A key takeaway was the importance of establishing a measurement baseline before AI implementation. Roberts attributed a 40% increase in shipped code to AI adoption precisely because of a DORA metrics baseline set years earlier, not to the AI rollout itself. Other practical advice includes centralizing context once instead of letting every team rebuild it, not assuming permission inheritance is enough for regulated data (Zillow added hard rules and a standing compliance check for its most sensitive categories), and treating context as a cost lever, not just a capability. Additionally, a breach of Hugging Face by an AI agent shows how lack of persistent context and guardrails can compromise security. Meanwhile, AI confidence is dropping, but robust governance can reverse the trend.
Sponsored Protocol
For more on context architecture, see the Wikipedia page on Zillow describing its scale. In conclusion, Jain stated that models alone are not enough to bring automation inside the enterprise; they must be connected with enterprise context. Companies aiming to implement agentic AI should follow Zillow's example by measuring first and building a persistent context layer.
