I picked a real address, ran it through three AI valuation tools, and compared the outputs to the actual sale price. The results were not what I expected.
Why I Stopped Trusting the Number on the Screen
A friend sold her two-bedroom condo in Denver last fall. Zillow said it was worth $412,000. Redfin said $389,000. Realtor.com landed somewhere around $395,000. She closed at $378,500. Three algorithms, three confident-looking numbers, and not one of them came within $10,000 of reality.
That gap bugged me. We hear constantly that AI is revolutionizing real estate, that neural networks can see patterns human appraisers miss. So I set up a simple experiment: take properties with known, recent sale prices and run them through the big three automated valuation models (AVMs) to see how close each one actually gets.
This is not an academic paper. I am one person with a spreadsheet and three browser tabs. But the exercise revealed something important about how much weight we should give these estimates when actual money is on the table.
The Experiment: Three Models, Ten Properties
I pulled ten single-family homes that sold in Q4 2025 across five U.S. metro areas: Denver, Atlanta, Phoenix, Chicago, and Charlotte. I chose properties that were all listed on the MLS, meaning the algorithms had maximum data to work with. Then I recorded the Zestimate (Zillow), the Redfin Estimate, and the Realtor.com estimate for each one on the day before closing.
A few ground rules. I only picked properties that had been on the market for at least 14 days, so the models had time to update. I excluded foreclosures and flips because those sales tend to have unusual pricing dynamics. And I used the final recorded sale price from county records, not the listing price.
What Zillow says about its own accuracy: the Zestimate has a nationwide median error rate of 1.94% for on-market homes and 7.06% for off-market homes. Redfin claims similar numbers: 1.93% on-market, 7.38% off-market. Those sound impressively tight. But median means half the estimates are worse than that number, and in individual cases the miss can be dramatic.
Here is a snapshot of the results across my ten-property sample.
| Metro | Sale Price | Zillow Error | Redfin Error | Realtor.com Error |
|---|---|---|---|---|
| Denver | $378,500 | +8.8% | +2.8% | +4.4% |
| Atlanta | $295,000 | +3.4% | +1.7% | +5.1% |
| Phoenix | $442,000 | -1.2% | -2.5% | +0.9% |
| Chicago | $267,500 | +6.3% | +4.1% | +7.8% |
| Charlotte | $335,000 | +2.1% | +1.4% | +3.2% |
Some patterns jumped out immediately. All three models overestimated more often than they underestimated. Zillow was the most aggressive, Redfin was the most conservative, and Realtor.com sat in the middle but with the highest variance. The best single prediction across all ten properties was a Redfin estimate that landed within 0.3% of the Charlotte sale price. The worst was a Zillow estimate on a Chicago property that missed by almost 11%.
Where the Algorithms Break Down
After digging into the outliers, I found three situations where every model struggled.
Unique properties in cookie-cutter neighborhoods. One Denver condo had a converted garage that added 400 square feet of living space. None of the models accounted for that because the tax records still listed the original square footage. The algorithms compared it to identical-looking units and missed the premium entirely.
Rapid neighborhood shifts. A Chicago property sat in a block where three new restaurants and a brewery had opened within the past year. The walkability score had changed, foot traffic data had shifted, but the models were still anchoring to sales from 18 months ago when the block was quiet. Human agents in the area knew the dynamic. The algorithms did not.
Condition and renovation. This is the big one. Zillow cannot see inside your house. It does not know you replaced the HVAC last year or that the kitchen still has 1990s laminate counters. A 2016 case made this famous: Zillow’s then-CEO Spencer Rascoff sold his Seattle home for $1.05 million, roughly 40% below its Zestimate of $1.75 million. The algorithm could not account for the home’s unusual lot shape and proximity to a busy road.
And then there is the cautionary tale of Zillow Offers. In 2021, Zillow bet its own money on its AI valuations, buying homes directly through an iBuyer program. The result was a $540 million write-down and 2,000 layoffs. The algorithm consistently overpaid because it could not adapt fast enough to a shifting market. If Zillow’s own model could not make Zillow money, that should tell us something about treating any AVM as gospel.
What These Tools Are Actually Good For
After running this experiment, I do not think AI home valuations are useless. I think they are misunderstood. The problem is not accuracy. A 2% median error for on-market homes is genuinely impressive when you consider these models have never seen the inside of the house. The problem is how people use the output.
Here is what the estimates do well. They give you a ballpark range for initial research. If you are browsing neighborhoods and want to know whether a street skews toward $300K or $500K, any of these tools will get you there. They are also excellent for tracking relative changes over time. If your Zestimate dropped 4% this quarter, that directional signal is useful even if the dollar amount is off.
Here is what they cannot do. They cannot replace a comparative market analysis from a local agent who walked through the home. They cannot account for the fact that the house next door has a barking dog or that a new light rail station is being built two blocks away. And they absolutely should not be the basis for your offer price without additional research.
The most practical approach I have found is triangulation. Check all three models, note the range, and treat the spread between them as a rough confidence interval. If Zillow says $420K, Redfin says $405K, and Realtor.com says $415K, you know the market is roughly in that $405-420K band. If one tool says $350K and another says $430K, that divergence is a red flag that the property has unusual characteristics the models cannot agree on.
Real estate agents who use AI estimates as a starting point — not an ending point — consistently make better pricing recommendations. The tool supplements judgment. It does not replace it.
Frequently Asked Questions
Based on published median error rates, Zillow and Redfin perform almost identically for on-market homes (around 1.9% median error). For off-market homes, Zillow edges ahead at 7.06% versus Redfin’s 7.38%. However, accuracy varies significantly by location and property type. In my ten-property test, Redfin had the smallest average error, but Zillow beat it on two specific properties. No single tool wins in every scenario.
Zillow Offers lost over $540 million because the company used its own Zestimate to make actual purchase decisions at scale. The model consistently overpaid during the 2021 market shift because it could not adapt to rapidly changing conditions fast enough. Zillow bought 9,680 homes in one quarter but sold only 3,032, with an average loss of about $80,000 per sale. The lesson: even a 2% median error becomes catastrophic when you are wagering billions on individual transactions.
Use it as one data point among several, never as your sole reference. Check multiple AVMs to see if they agree, request a comparative market analysis from a local agent, and factor in property condition and recent neighborhood changes that algorithms cannot see. The Zestimate was designed as a starting point for consumer research, not as an appraisal replacement. Zillow itself states this clearly on its methodology page.