Overgeneralised feedback

Description

  • We listened, here's our improved all-steel umbrella, now impervious to 6-inch hailstones based on user feedback! Only 25 pounds!

Disproof

  • Quantification how much of your core user group actually holds any given opinion

Consequences

  • You improve for something that’s really not important
    • You burn time, money and possibly drive the majority of users away
    • Congratulations, you played yourself

Causes

  • In B2B situations, whoever kicks their account manager the hardest will get them to shout the loudest, with predictable outcomes
  • You may have hired people who look really good when it comes to designing and building, but actually haven’t been around for the full product lifecycle and don’t really know what to do at this point
    • Leadership roles self-select for serial job hoppers with nothing to their name but a half-dozen shipments they didn’t stick around to evaluate/improve
  • Else, you created unnecessary time pressure that will come back to bite you; rushed evaluation is on par with no evaluation
  • Some more banal causes also include; insufficient segmentation, unqualified focus on quantified data, confirmation biases and unspecific feedback collection methods

Approaches

  • If users tell you something’s not working for them, it’s a good idea to listen
  • If users tell you what they want it to do, it’s a good idea to take it under advisement and then actually figure out what the problem with the solution was
  • Either way, create conditions where everyone understands that feedback is successfully implemented when it produces results you want, not when it passes unit tests