Cognitive Biases at Work: What Replicated and What Reduces Them

Cognitive biases are systematic errors in judgment: mistakes that are predictable in direction, not random noise. In Tversky and Kahneman's classic 1974 experiment, people who saw a rigged wheel of fortune stop at 10 estimated that 25% of United Nations members were African countries; people who saw 65 said 45%. The number was meaningless, and it still moved the answer by 20 points. This post explains what cognitive biases are, which ones show up in decisions at work, which famous findings held up in large replications and which did not, and what the evidence says about reducing them.

What cognitive biases are: errors with a pattern

A cognitive bias is a shortcut in thinking that usually works and sometimes fails in a known direction. The direction is the point. Random errors cancel out over many decisions; biased errors pile up on one side.

Tversky and Kahneman's 1974 paper in Science framed it this way. People judge uncertain things with a few heuristics, mental rules of thumb, and each heuristic leaves its own fingerprint:

• Representativeness: judging probability by resemblance ("she sounds like an engineer"), while ignoring how common engineers are in the first place.
• Availability: judging frequency by how easily examples come to mind, so vivid or recent cases count for more than they should.
• Anchoring and adjustment: starting from a number and adjusting away from it, but not far enough.

The paper's own anchoring examples are deliberately absurd, which is why they persuade. In the wheel experiment, paying people for accuracy did not remove the effect. The bias was in the starting point, not in effort.

A useful distinction follows from this. A bias is not the same as ignorance. A well-informed person can still anchor, because the mechanism runs before knowledge gets a vote.

The cognitive biases that matter most in work decisions

At work, the costly biases are the ones that distort a team's recommendation before it reaches the person who signs off. In their Harvard Business Review article, Daniel Kahneman, Dan Lovallo and Olivier Sibony built a 12-question review around exactly that situation. The biases they target are familiar names:

Confirmation bias. Teams look for evidence that the preferred option is right and stop searching once they find it. The authors' test is whether the team seriously considered credible alternatives.

Anchoring. Budgets copied from an earlier project, trends extrapolated by default, a first offer in a negotiation. Their advice is to ask where each number came from and, if needed, re-anchor with an independent estimate.

Optimism and the planning fallacy. Forecasts built bottom-up, step by step, run short. The fix they name is the "outside view": compare the plan with what actually happened in similar past projects, rather than with the team's own story about this one.

Sunk cost and the halo effect. Past spending keeps a failing project alive, and one success makes everything else about a person or a company look better than the data says.

Whether this matters for money is harder to prove, but the best large-scale attempt points one way. Lovallo and Sibony (McKinsey Quarterly, 2010) surveyed managers about 1,048 major business decisions. In their regression, the quality of the decision process explained outcomes about six times more than the amount of analysis did. For the 673 decisions with ROI data, moving from the bottom to the top quartile on process went with 6.9 percentage points more ROI. It is self-reported and correlational. It is also the closest thing to a field estimate of what a less biased process is worth.

Which cognitive biases replicated, and which did not

The core judgment biases have held up better than much of social psychology. That is the honest summary of the replication crisis as it applies to this field.

The first Many Labs project (Klein et al., 2014) re-ran 13 classic and newer effects in 36 samples with 6,344 participants. The set included anchoring and gain-versus-loss framing, as the project overview from BITSS lists. Ten effects replicated consistently. The two clear failures were priming effects: a flag nudging people toward conservatism and money images nudging them toward defending the system. Whether a lab or an online sample was used mattered little; what mattered was the effect itself.

Prospect theory, the source of loss aversion and framing effects, passed a bigger test. Ruggeri and colleagues (2020) ran Kahneman and Tversky's original 1979 choices with 4,098 people in 19 countries and 13 languages. The pattern replicated for 94% of items and for 12 of 13 theoretical contrasts, with somewhat smaller effects.

The wider picture is less flattering. Many Labs 2 (Klein et al., 2018) tested 28 published findings with 15,305 participants across 125 samples in 36 countries and territories. Only 15 (54%) replicated at the usual threshold, and the median effect size fell from d = 0.60 in the originals to 0.15. So a bias that replicates may still be smaller than the textbook says.

Then there is the bias that turned out to be a mistake about bias. For decades the "hot hand fallacy" was the standard example of people seeing streaks in random data. Miller and Sanjurjo (Econometrica, 2018) showed that the classic basketball analysis used a measure that is biased downward in short sequences. Once that is accounted for, the gap between shooting after three hits and after three misses was about 12 percentage points larger than chance would produce. The original conclusion reverses: belief in a hot hand was not simply an illusion.

The bias you cannot see in yourself

The most stubborn cognitive bias is the belief that you have fewer of them than other people. Psychologists call it the bias blind spot, and it is a failure of metacognition, your judgment about your own thinking.

Scopelliti and colleagues (Management Science, 2015) built a scale for it and tested it on 661 people across five samples. For each of 14 biases, people rated the average American as more susceptible than themselves. At the individual level, 85.2% showed a significant blind spot; one person in the whole sample showed the reverse.

Two findings from the same paper matter at work. The blind spot was independent of intelligence and of general decision-making competence, so being smart does not protect you. And people with a bigger blind spot were less receptive to useful advice, which means the most confident colleague is often the hardest one to debias.

Kahneman, Lovallo and Sibony reached the same conclusion from practice. Executives, they wrote, can do little about their own biases, but with the right tools they can spot and correct the biases in their teams' proposals.

Debiasing: what training can and cannot do

Training can reduce cognitive biases, which used to be the minority view. Early training attempts met with little success, so researchers and policy makers turned to incentives and nudges instead.

Morewedge and colleagues (2015) ran two experiments with about 240 lab participants each. People got a single session: either a computer game with personalised feedback or an instructional video. The first experiment targeted the bias blind spot, confirmation bias and the fundamental attribution error; the second targeted anchoring, representativeness and social projection. Each targeted bias fell by 31.94% or more right after the game and by 18.60% or more after the video. Two months later the reductions were still at least 23.57% for the game and 19.20% for the video. Feedback and practice beat explanation.

The harder question is transfer: does a lab lesson survive a real decision? Sellier, Scopelliti and Morewedge (Psychological Science, 2019) gave the same kind of training to 290 graduate students in three professional programmes. Later they solved an unannounced business case modelled on the Challenger launch decision, without knowing it measured bias. Trained students were 29% less likely to choose the inferior, hypothesis-confirming option. The authors are clear that this is one field experiment, not a settled law.

What the evidence does not support is the cheapest version of debiasing: a slide listing 20 biases. Knowing names is not the same as changing the move your mind makes under time pressure.

Fix the decision process as well as the person

Since you cannot reliably catch your own biases, the most practical defence is a process that catches them for you. This is the shared conclusion of the HBR checklist and the McKinsey data.

A good process makes biases visible at the moment they act. It asks for the source of every key number. It requires at least one real alternative before a choice. It compares the plan with a reference class of similar past projects. It gives someone the job of arguing the other side, so dissent is part of the procedure rather than a personal risk.

None of this needs a new tool. It needs the habit of reviewing how a recommendation was reached as well as what it says. Related habits show up in what critical thinking is and how it is measured, and in hiring specifically, in the evidence on hiring bias.

The individual side still counts. People differ in how they reason under uncertainty, and those differences are measurable. If you want to see your own thinking in numbers, NOOVA runs it as a 20-minute voice dialogue instead of hours of tests.

A checklist for catching cognitive biases in your next decision

1. Ask where each key number came from before you discuss whether it is right.
2. Write down one credible alternative to the option your team prefers.
3. Compare your timeline and budget with three similar past projects, not with the plan.
4. Name the money or effort already spent, then decide as if it were gone.
5. Assign one person to argue against the recommendation in the meeting.
6. Rate your confidence before you see the outcome, then check it afterwards.
7. Ask a colleague which bias they see in your proposal, and take the answer seriously.

Sources

• Judgment under Uncertainty: Heuristics and Biases, Tversky and Kahneman, Science, 1974
• Investigating Variation in Replicability: A "Many Labs" Replication Project, Klein et al., Social Psychology, 2014
• A replication example: The Many Labs Project, Berkeley Initiative for Transparency in the Social Sciences
• Many Labs 2: Investigating Variation in Replicability Across Samples and Settings, Klein et al., Advances in Methods and Practices in Psychological Science, 2018
• Replicating patterns of prospect theory for decision under risk, Ruggeri et al., Nature Human Behaviour, 2020
• Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers, Miller and Sanjurjo, Econometrica, 2018
• Bias Blind Spot: Structure, Measurement, and Consequences, Scopelliti et al., Management Science, 2015
• Debiasing Decisions: Improved Decision Making With a Single Training Intervention, Morewedge et al., Policy Insights from the Behavioral and Brain Sciences, 2015
• Debiasing Training Transfers to Improve Decision Making in the Field, Sellier, Scopelliti and Morewedge, Psychological Science, 2019
• Before You Make That Big Decision, Kahneman, Lovallo and Sibony, Harvard Business Review, 2011
• The Case for Behavioral Strategy, Lovallo and Sibony, McKinsey Quarterly, 2010