The Survivorship Bias - "Impact of False Negatives"

A good read to understand the impact of false negatives:

During World War II, numerous fighter planes were getting hit by anti-aircraft guns.

Air Force officers wanted to add some protective armor/shield to the planes. The question was "where"?

The planes could only support few more kilos of weight. A group of mathematicians and engineers were called for a short consulting project.

Fighter planes returning from missions were analysed for bullet holes per square foot. They found 1.93 bullet holes/sq. foot near the tail of the planes whereas only 1.11 bullet holes/sq. foot close to the engine.

The Air Force officers thought that since the tail portion had the greatest density of bullets, that would be the logical location for putting an anti-bullet shield.

A mathematician named Abraham Wald said exactly the opposite: "more protection is needed where the bullet holes aren't"! that is, around the engines.

His judgement surprised everyone. He said, "We are counting the planes that returned from a mission. Planes with lots of bullet holes in the engine did not return at all".

Debrief

If you go to the recovery room at the hospital, you’ll see a lot more people with bullet holes in their legs than people with bullet holes in their chests. That’s not because people don’t get shot in the chest, it’s because the people who get shot in the chest don’t recover.

A quote from Albert Einstein: Not everything that counts can be counted, and not everything that can be counted, counts.

References:

From the book: "How Not To Be Wrong", by Jordan Ellenberg.

More on Survivorship Bias: https://en.wikipedia.org/wiki/Survivorship_bias