Should Research Priority Depend on the Consequences of Being Wrong?

What If the Current Answer Is Probably Right, but Being Wrong Would Matter Enormously?

Researchers often focus on how uncertain the evidence is. But uncertainty tells only part of the story.

Suppose existing evidence suggests that one conclusion is probably correct. There is still some chance that it is wrong. If being wrong would have only minor consequences, additional research may have limited value. If being wrong could expose people to substantial harm, waste major resources, misdirect policy, or distort an important scientific field for years, the same residual uncertainty becomes harder to ignore.

This leads to a different way of thinking about research priority: not merely “How likely are we to be wrong?” but also “What happens if we are?”

The Consequences of Error Should Matter, but They Are Only Part of the Decision

Research Value Depends on Both the Chance of Error and What That Error Would Cost

A Small Risk Can Matter When the Stakes Are Large

Consider two decisions for which the current evidence suggests a 90% chance that the preferred option is indeed better. The probability of choosing incorrectly appears identical.

In the first case, choosing incorrectly creates a minor inconvenience. In the second, it could cause substantial harm to thousands of people.

The evidential uncertainty may look similar, but the expected consequences of that uncertainty are plainly different.

This principle is central to value-of-information analysis. In formal decision settings, the value of reducing uncertainty depends partly on the opportunity loss associated with making a suboptimal decision. Research can therefore have high value not merely because we are uncertain, but because being wrong under that uncertainty would be consequential.

Probability of Error and Consequence of Error Are Different Questions

Neither quantity is sufficient alone.

A high probability of error attached to a trivial choice may generate little expected loss. A very severe possible consequence attached to an extraordinarily remote and poorly supported scenario may also provide a weak case for research. Priority emerges from their combination, together with the ability and cost of obtaining better information.

Think About Expected Consequences, Not the Worst Thing You Can Imagine

Researchers should be cautious when invoking catastrophic possibilities. Almost any decision can be made to sound urgent by describing the worst conceivable outcome.

A more disciplined assessment considers both the plausibility and magnitude of the consequences. Ask which outcomes remain reasonably possible under current evidence, how likely the current choice is to be suboptimal, and what would happen in those states of the world.

This prevents research priorities from being dominated by vivid but extremely improbable scenarios.

Being Wrong Can Have Different Kinds of Consequences

Consequences are not limited to physical harm or financial loss.

The appropriate consequences depend on the question. Researchers should avoid forcing every project into health, economic, or policy terms when its real contribution is scientific.

The Number of People or Decisions Affected Can Change the Stakes

A small loss repeated many times can become consequential. This is particularly relevant when evidence informs decisions applied repeatedly across a large population.

Formal population value-of-information analyses can account for the number of current and future people expected to face a decision. A modest per-person consequence may generate substantial aggregate value for better information when the decision affects many people over time.

The reverse is also important. A small affected population does not make research unimportant. A rare condition, specialized population, or unusual setting may involve very high individual stakes. Scale is one dimension of consequence, not a substitute for considering severity.

Consequences Can Be Asymmetric

Being wrong in one direction may be much worse than being wrong in the other.

Suppose researchers are uncertain whether an educational screening tool identifies students who need additional support. One kind of error might provide unnecessary additional support to some students. Another might systematically fail to identify students who genuinely need assistance.

The two mistakes may have different costs and implications.

Similarly, incorrectly concluding that an intervention works and implementing it can have different consequences from incorrectly concluding that it does not work and withholding it. Research prioritization should not assume that all errors are interchangeable.

The Consequences of Being Wrong Can Extend Beyond the Immediate Decision

A mistaken conclusion can sometimes propagate.

One study may influence subsequent hypotheses, systematic reviews, clinical recommendations, institutional policies, technological development, or future funding. If an incorrect assumption becomes embedded in later work, the cost of error may accumulate.

This does not mean researchers should inflate the importance of every study by imagining that it will transform an entire field. The pathway should be plausible. But when current evidence occupies an influential position in a chain of later decisions, verifying a consequential assumption can have substantial value.

Severe Consequences Do Not Automatically Mean More Research Is the Answer

This is a crucial limitation.

Suppose being wrong would be disastrous, but no feasible study could materially reduce the relevant uncertainty. Or suppose obtaining the evidence would take ten years while the decision must be made next month. Additional research may have little immediate decision value despite the high stakes.

Likewise, research itself may create risks, burdens, or opportunity costs that outweigh its expected informational benefit.

The appropriate response to uncertainty can sometimes be precaution, monitoring, reversible implementation, stronger safeguards, or choosing an option that performs reasonably well across several plausible scenarios rather than conducting another study immediately.

Research is one way of managing uncertainty, not the only one.

Consequences and Uncertainty Need to Be Considered Together

The relationship can be summarized conceptually:

This is why research priority should not depend exclusively on how uncertain the current answer is. Uncertainty becomes consequential through what can happen when the current conclusion is wrong.

The Same Probability of Error Can Justify Very Different Research Priorities

Common Mistakes When Using the Consequences of Error to Set Research Priorities

Ask What You Stand to Lose if the Current Answer Is Wrong

When evaluating a research question, do not stop after estimating how uncertain the evidence appears. Map the consequences attached to plausible errors.

Then return to the broader question of how much difference better knowledge could make. The value of knowing the answer depends partly on the mistakes that better information could help prevent.

Finally, compare that benefit with the cost of obtaining the answer. High stakes can justify substantial research investment, but they do not make resources unlimited.

Before Prioritizing Research Because Being Wrong Would Matter, Check:

Questions About Research Priority and the Consequences of Being Wrong

Research Priority Depends on What Could Happen When Current Knowledge Fails Us

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.