How the Simpson Paradox can play tricks on you
March 17th, 2021. This is a normal day at ORGO, a producer of edible organic oil. 9 o’clock in the morning, John sends the below chart to Alex, head of marketing.
According to this chart, men like sesame oil more than flaxseed oil, so do the women.
Alex also wants the aggregated view.
No problem. John adds a chart in his SAP Analytics Cloud story. Here is the result obtained by John.
As usual when a difficulty occurs, John rubs his neck. There must be something wrong somewhere.
Let’s define a table to see the values that hide behind the calculated measure: %Likes.
The values for the calculated ratio % Likes seem correct.
John cannot send to the CMO of ORGO the current story. His reputation is at stake. The aggregated numbers contradict the detailed ones. He needs a cappuccino …
The calculations in the table are correct, and at the same time the results don’t make sense. Who could help him? Margaret maybe (Margaret studied at Berkeley in the mid-seventies, she loves statistics and Jimmy Hendrix). This phenomenon is called the Simpson Paradox, she said. It can happen whenever you calculate a percentage of total between groups of different sizes. To work around the issue Margaret suggests averaging the ratios.
John creates the following calculated measure:
And obtains this chart:
She is right, the paradox vanished.
This is a quick and dirty fix though. A better solution would be to have the same number of answers in each group, so that a Like has the same weight regardless of gender and oil type. Having equal sized groups will prevent the issue John ran into, that is (Margaret writes on the whiteboard):
John nods his head. Margaret explains to him that, with balanced groups, the ratio of the sums will equal the average of the ratios. Here is an example:
Thank you so much, Margaret.
Half past twelve. John’s stomach rumbles. He goes for lunch with a sense of relief.
Will lunch be with Flaxseed or Sesame oil? 😉
John Leggio , sesame for me - John's reputation is at stake 🙂
Short and sweet 🙂
PS. I love paradoxes!
Great blog on a tricky topic, Statistics is fun! 🙂
Well done Marc! Very interesting and a good read
Great blog! Statistics is silent and doesn't tell us whether to aggregate or not when we encounter Simpson's paradox. Did Margaret use the domain expertise of Jimmy Hendrix? 🙂 Let us work together on a follow-up article on when not to aggregate. It is important for decision making and can have legal implications like the Berkeley case.