THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS

''Statistics can fool you. In fact it is fooling your government right now. It can even bankrupt the system...
...
For us the world is vastly simpler in some sense than the academy, vastly more complicated in another. So the central lesson from decision-making (as opposed to working with data on a computer or bickering about logical constructions) is the following: it is the exposure (or payoff) that creates the complexity —and the opportunities and dangers— not so much the knowledge ( i.e., statistical distribution, model representation, etc.). In some situations, you can be extremely wrong and be fine, in others you can be slightly wrong and explode. If you are leveraged, errors blow you up; if you are not, you can enjoy life.
...
A Turkey is fed for a 1000 days—every days confirms to its statistical department that the human race cares about its welfare "with increased statistical significance". On the 1001st day, the turkey has a surprise.
...
If small probability events carry large impacts, and (at the same time) these small probability events are more difficult to compute from past data itself, then: our empirical knowledge about the potential contribution—or role—of rare events (probability × consequence) is inversely proportional to their impact.
...
What Is Wise To Do (Or Not Do) In The Fourth Quadrant...
- Avoid Optimization, Learn to Love Redundancy...
- Beware the "atypicality" of remote events...
- Metrics. Conventional metrics based on type 1 randomness don't work. Words like "standard deviation" are not stable and does not measure anything in the Fourth Quadrant. ...'' [source]

A very stimulating article!