Follow Our Tweets: @mris_Real_News Like Us On Facebook MRIS® - a Bright MLS

AVM Bake Off – Variances Still Unacceptable

February 20, 2013  |  by John Heithaus

Recently, Zillow’s CEO was on Bloomberg and this chart popped up. The analysis is done by Clarity Consulting and I’m not privy to their data methods or accuracy. But what jumped out to me, aside from the huge accuracy variances, is the assertion that only 72% of AVM estimates in the DC market are within 10% of the eventual sales price. What experience do you have with AVMs and related issues and how do you address them? What do you think MRIS should do about these issues?


Posted in Blog, MRIS CMO Insights

Next Post → ← Previous Post

22 responses to “AVM Bake Off – Variances Still Unacceptable”

  1. Dina says:

    They are way off most of the time. I wish they would add a disclaimer, so that clients do not call us and say: "but zillow shows….. " …. I think MRIS should help find a way to educate the public that these are NOT accurate!

  2. Collier says:

    When a client calls me about the "zillow value" I direct them to the discussion that zillow posts on the accuracy of their estimates. Once framed by that discussion they tend to put less emphasis on it.

  3. Janet says:

    The layman or general public believes sites like Zillow, Trulia,, and don't realize that their information is not accurate. If MRIS could help educate our customers and clients, it would help us REALTORS do their job.

  4. Bob Jurgensen says:

    Zillow is like a stopped clock – it's right twice a day – and the rest of the time it's off by second, minutes and hours – in other words, it's highly inaccurate most of the time. The bottom line on Zillow, Trulia and other AVM's is that they are computer geeks relying strictly on data and stale tax assessments. For example, in PWC, next month we will get a new tax assessment in the mail. This assessment will be as of January 1, 2013 – however reality is that that assessment was completed sometime in 2012 – perhaps on January 2, 2012. BUT think about this: what about a tax assessment TODAY (Feburary, 2013) – it could be as old as January 2, 2011 !!!!! What a different two years makes in this market? OMG, it's huge! Perhaps 20-30% or more!

    Here is the bottom line: Zillow and other AVM's are a prime example of our "instant" access to data and information but there is simply no replacement for a human in a dynamic market place – which is the very description of real estate markets – dynamic. It's always changing and it's always in need of tweaking – to give you a more perfect example, Zillow is just a computer formula programmed by a person based on available information that can be hard coded. What canNOT be hard coded are things like "LOCATION" or "VIEWS" or bad news neighbors, or overall condition of a home, deferred maintenance or MOTIVATION. I've seen people sell for much less due to motivation (which can be a wide variety of things!)

    The best example of an AVM's incompetence is beach property: I know because I own a place at the beach – Zillow has no ability to distinquish between a home that is 2 lots from the ocean versus a home 10 or 15 lots back from the water… but each lot further from the water devalues a similar property by as much as $20-40,000 or more, depending on how far from the ocean (and also VIEWS!) Other good examples of such AVM incompetence would be backing to a commercial area or busy road (appeal) or living near the end of an airport runway or downwind of a landfill. AVM's just are not that smart – nor will they ever be that smart – it takes a human to determine those nuances.

    I always take a copy of the Zillow report with me because 7 out of 10 times I will hear "…but Zillow says…" of course if Zillow if too low, which is happening now do to the rising market trends (check back in 2015 or 2016, Zillow may have caught up by then) – and I pull out the Zillow "report" and we discuss it versus my CMA – mine always makes much more sense.

    NOW, to give kudos where they are due: the new RPR system recently gifted to us as agents by NAR is a near perfect tool for valuations – far better than anything MRIS offers us, BUT one must learn how to use it – it's complicated and it's sophisticated but in the right hands of a patient and savvy agent, it's a wonderful tool – and honestly I was shocked that NAR could ever come up with anything this accurate. Of course it needs human tweaking and it's not available to the general public (why? because it takes a professional to know how to use it property – I mean would you give a patient a scalpel and tell him to go to WEB MD and then operate on himself? Of course not.

    AVM's, in general, serve only to confuse the public and sell advertising to make computer geeks wealthy in CA so they can build mansions and live happily ever after. They are confusing and we, as Realtors, should never contribute to their databases in any way, shape or form. Just like we should not encourage people to represent themselves in court when they have a serious legal issue, or perform surgery on themselves, we should not suggest that people determine their own values using automated valuation models. It's just another worthless Internet gimmick used primarily to sell ads – not help people.

  5. Denis Rose says:

    This is what I tell clients: Zillow's model uses proximity of properties, county tax assessment and square footage for their Zestimates. There is no variance for neighborhood, improvements in a home, schools, additions or major changes that aren't in the tax assessment, etc. May be relevant for a large subdivision with a small number of models and a high number of transactions. It is a thoroughly untrustworthy source for pricing evaluation

  6. Consumers seem to prefer 3rd party sites for their information because they don't distrust the source, the way they do information from real estate "sales people". I think MRIS has the potential to create this same perception as a trusted resource for information by stressing in their marketing message that all MRIS isting and statistical data is the original source of information relied upon by 3rd party sites as well as REALTORS, appraisers, and consumers AND emphasizing the "up-to-date" benefit of listing data on your site and the websites of member brokers and agents. As to automated values, I think you should stay out of that game other than to indicate that the ONLY way to accurately estimate the value of a property is through a personal inspection of the property by licensed agents and/or appraisers.

  7. Falls Church Jim says:

    10%?! I don't think that's anything to brag about. On a $500,000 home being off by $50,000 is a lot to most hard working Americans. Of course an algorithm is useless in determining the true market value of anything. The public concensus is slow to come to realizations, but eventually the sham that is Zillow will be revealed as more people understand how it (doesn't) works. I have yet to really come across anyone who takes it seriously.

  8. Valerie says:

    I tell my buyer clients to ignore those sites since the market value information they provide is neither valid nor "value-able." The section of our listing agreement that allows a seller to approve or veto putting their property on sites that have inaccurate data sets the stage for that discussion with them. I use MRIS data to show them current comparables and GCAAR stats to show them historical growth or decline data and recommend that they still include their properties on those other sites for nationwide visibility. It's not hard to explain that being roughly 90% right 72% of the time is nothing to crow about. Think stealth missiles.

  9. Frank Locke says:

    My two cents is that it doesn't make any difference to have the conversation when our own NAR pretty much sold us agents down the river when they established the "Realtors Property Resource" application and basically come up with the same valuation that these other sites have. Anytime you get to the point where a CMA is done by pushing a button, all you need REALTORS for is to open up doors and print brochure. Remember travel agents?

  10. DavidR says:

    -setting folks straight needs to be handled case-by-case
    -as others point out, we should refer folks to zillow's (e.g.) own quality assessment
    -I also tell folks that lenders PAY for appraisials and do not rely on free AVM baloney
    -further, the AVM crew (nor the local tax assessor) has ever been to MY home / yours?
    -by using tax assesments in their models, they are doomed / garbage in, garbage out
    itz kinda late for MRIS who should have been pushing homesdatabase to the local public BIG time from it's inception / in not doing so the 3rd party sites have gained a foothold (and using our listing data too!) / hoping to see more homesdatabase publicity in the future!

    nevertheless, I am a BIG fan of MRIS / DavidR

  11. M. Morris says:

    Zillow is amazing. I don't know how they can be so inaccurate, so way out in left field, so arrogant that they believe they are doing the public a service. I can't believe that someone or some organization hasn't tried to stop them. To test the accuracy of their estimates, I entered my house info. If I could get that price for my house I wouldn't be living there now.

  12. PR in VA says:

    I turn off the AVM for all listings because the clients want me to do that once they understand how inaccurate it is. I would hope that all other brokers and agents would start doing the same.
    Perhaps MRIS can publish statistics on how many listings have this feature on or off. Once the topic is discussed, maybe the tide will change. When I see the default for AVM, Comments, Addresses set to NO instead of yes, I will know things are heading in the right direction.

  13. MRIS_CMO says:

    Thanks to everyone for their insightful and valuable comments. Some really good suggestions here — MRIS has an AVM tool in development intended for use by our CUSTOMERS only, not the public. Stay tuned for more and we'll likely reach out to each of you for feedback prior to launch. David R — we do have a big marketing push in the works for to be rebranded We have tv, radio, print and outdoor advertising planned …. real time local listings is the primary theme.

  14. Diane says:

    All Realtors should take at least two seminars in appraisal. Then it is not too difficult to help the owner or buyer learn how to do a "quick and dirty" appraisal. They see immediately what the problem is with those automated versions. The only houses that present any issue tend to be the "singletons". Those are normally in locations that buyers with children would not want to consider because they lack friends and playmates for their children. In the Greater Washington Metropolitan Area the school pyramid is the biggest factor in a home's value and desirability…so start with that when doing an "appraisal" or BPO. This has been the case for the last 50 years….and should continue to be so. The school standings do not seem to change much…..alas.

  15. Suzanne says:

    I just had this discussion yesterday with clients – and he is an attorney and she is our CPA. I just tell customers that only provides accurate consumer information in our area.

  16. Lina Bader says:

    The main problem is caused by the data bases — to be precise, Zillow takes into account ALL neighborhood "Sales," so that when the bank "buys" at auction a 100K townhouse for its $235K loan balance, Zillow includes that $235K "Sale" to derive the neighborhood's average and median prices AND a particular property's "estimated" value.

  17. Mark Novak says:

    I think we should leave well enough alone, it is fine for web sites to provide a ballpark and these inaccuracies further enhance our position as the professional. How about we take the time to prove our worth and educate our client's.

  18. Why aren't our associations, NAR VAR FAAR as well as MRIS doing more to disclaim these valuation sites? They are so busy playing big PAC to government that I think they have forgotten we pay to have them assist – advertise – disperse the knowledge that Realtors are professionals who can greatly aid them in their search and sale of Real Estate. A web site in no way replaces a Realtor.
    We pay a lot of money I don't feel we get a lot of support.

  19. Bob Jurgensen says:

    MRIS sells our data to sites like Zillow and Trulia via IDX so they can then turn around and sell the leads to us – so we sold our soul to the devil and the devil is US!

  20. […] There’s quite a discussion thread over on the MRIS blog about how useful the Zillow estimates … Zillow recently announced that 72% of their estimates in the DC market are within 10% of the eventual sales price and the realtors are weighing in on those Zestimates and how it impacts their business. [MRIS] Tweet Pin It […]

RSS Feed