Beware of averages
Why averages can fool product managers and how to fix it
In product management we deal a lot with numbers and data, trying to improve our understanding of our users and how they use our products.
Often, to make sense of the data, we calculate averages, like the average amount of purchases per user, or the average time a support ticket is being answered.
So, whats the problem with this?
Consider diapers. The average diapers user is around 33 years old. But the real distribution is obviously very different, you gonna have lot’s of baby’s using diapers and lots of seniors using them, thus the average number becomes useless and distorts what’s really going on.
Look at distributions instead!
So instead of trusting a single aggregated number (33 years), look at the actual distribution and you will be able to identify different segments.
Moreover you might discover some interesting insights, such as I did when I wrote this article.
Example from a real project
For one product I worked on, the average response rate was around 50%. You could fall into the trap of averages and believe a typical user responds on average every second time. But, when we digged deeper and looked at the distribution, we saw what was really going on: once users responded, they kept almost always responding, but clearly we had an activation problem: many users never responded at all.
So my practical tip for PM’s: look at distributions. They give you a much clearer picture on what’s actually happening and let’s you see patterns.
