Zillow Urges Companies to Measure AI Effectiveness Before They Build
Source: VentureBeat AI
Summary
- Zillow, a real estate technology company, has been using AI for years to improve its services.
- At the recent VB Transform 2026 conference, Zillow's SVP of Engineering Toby Roberts shared the company's experience with building AI architecture that can carry context across different customer interactions.
- The company found that the hard part of building AI was not collecting data but rather creating a persistent context layer that remembers where a customer is in their journey.
- Zillow chose to build its own architecture rather than relying on a single external chat interface and has integrated it with Glean, a platform that centralizes integration work and reduces costs.
Why It Matters
- Measuring AI effectiveness is crucial for businesses, and Zillow's experience highlights the importance of setting a baseline before implementing AI.
- This approach allows companies to accurately attribute the impact of AI adoption and make informed decisions.
- Companies should also consider centralizing context and avoiding duplicated integration work, which can be a hidden cost.
- By following these best practices, businesses can maximize the benefits of AI and improve their overall performance.
GenAI EXPLAINED
Return on Investment (ROI) numbers refer to the financial gains or losses resulting from a particular investment or decision. In the context of AI, ROI numbers measure the impact of AI adoption on a company's performance. A data mesh approach involves breaking down a large dataset into smaller, more manageable pieces, making it easier to analyze and understand. This approach helps ensure that data is accurate and reliable. Machine learning history refers to the experience and knowledge gained from previous AI projects. In Zillow's case, the team drew on 20 years of machine learning history to build its AI architecture. Glean agents are small, task-specific models that can be fine-tuned to perform specific tasks. These agents are used to automate repetitive tasks and reduce costs.