Should You Prioritize Missing Evidence or Unreliable Existing Evidence?

Which Deserves More Attention: No Evidence or Evidence You Cannot Trust?

Imagine that you are choosing between two potential research gaps. For one question, almost no relevant studies exist. For the other, several studies have been published, but their methods are weak, their findings conflict, or their evidence does not adequately answer the question you actually care about.

Which gap should receive priority?

The absence of research can appear more obviously novel. Unreliable existing evidence, however, may leave decision makers just as uncertain and sometimes create an additional problem: people may already be acting on conclusions that deserve less confidence than they receive.

The choice therefore cannot be made simply by counting studies.

Neither Type of Gap Automatically Has Priority

Missing Evidence and Unreliable Evidence Create Different Research Problems

First distinguish absence from inadequacy

AHRQ's framework for identifying research gaps from systematic reviews provides a useful way to see why "there is a gap" can mean several different things. The framework identifies reasons including insufficient or imprecise information, biased information, inconsistent results, and information that does not adequately address the relevant population, setting, outcomes, or other aspects of the question.

The categories are not perfectly separate. A field may have both too little evidence and serious weaknesses in the evidence that does exist. The distinction is nevertheless useful because different problems call for different research responses.

No evidence does not automatically mean high priority

A completely unstudied question looks like a clear research gap. Yet absence alone tells you nothing about the importance of the question.

Perhaps nobody has studied it because the issue is new or technically difficult. Perhaps the affected population has been overlooked. But another possibility is less exciting: the question may have little consequence.

This is why missing knowledge does not automatically deserve a study. Before prioritizing an evidence-free question, ask what would change if reliable evidence became available.

Existing studies do not mean the gap has been filled

The opposite mistake is to treat publication as resolution. A question can have many studies and remain uncertain.

Cochrane's GRADE guidance evaluates certainty using considerations that include risk of bias, inconsistency, indirectness, imprecision, and publication bias. These dimensions illustrate why evidence volume and evidence reliability are not interchangeable.

For example, ten observational studies with similar sources of bias do not necessarily provide the same confidence as ten well-designed studies suited to the causal question. Likewise, several small studies may leave an estimate too imprecise for a decision, and numerous studies conducted in one population may provide indirect evidence for a substantially different population.

Accordingly, a topic can have plenty of research and still contain an important gap.

Unreliable evidence can be especially consequential when people already trust it

An absence of evidence leaves an obvious uncertainty. Unreliable evidence can be more deceptive because it may create apparent certainty.

Suppose weak studies repeatedly suggest that an intervention is effective and the finding has begun influencing practice. If serious methodological limitations make the apparent effect uncertain, stronger research could do more than add knowledge. It could confirm, revise, or overturn an existing conclusion on which people are already acting.

That possibility can strengthen the priority of the gap, particularly when the decisions are consequential. It does not mean that every weak literature deserves replication. The potential consequences of being wrong still need to be established.

Different evidence problems imply different research needs

Once you identify why the evidence is inadequate, you can ask what type of research would actually improve it.

Cochrane's guidance on implications for research follows similar reasoning. For example, risk of bias can imply a need for methodologically better studies, indirectness can imply a need for studies better matched to the relevant question, and imprecision can indicate a need for additional information.

Do not confuse disagreement with unreliability

Conflicting findings deserve investigation, but disagreement among studies does not automatically mean that all existing evidence is unreliable.

Differences may reflect real variation. An intervention may work differently across populations or contexts. Measurements may capture different constructs. Effects may depend on implementation, dose, timing, or baseline conditions.

The research gap may therefore concern the source of heterogeneity rather than a need to conduct one more study asking for an overall average effect.

Priority should depend on consequences as well as certainty

Imagine two gaps:

• Question A has no published evidence, but the answer would probably have little effect on theory or practice.
• Question B has substantial existing evidence, but serious limitations leave an important policy decision uncertain.

Question B may deserve higher priority despite being less novel.

Reverse the consequences, however, and the priority could reverse as well. There is no general rule that unreliable evidence outranks absent evidence.

This is why research prioritization needs explicit criteria. AHRQ's work on research gaps identifies considerations such as potential impact, foundational importance, timeliness, and stakeholder perspectives. The type of evidence gap is one part of the judgment, not the entire judgment.

Ask what a new study would add to the evidence state

Before choosing either gap, compare the likely information gain from feasible research.

If no evidence exists, a well-designed first study may substantially improve understanding. Yet one small exploratory study may still leave the important question largely unresolved.

If unreliable evidence exists, a rigorous study designed specifically to address the dominant limitation may sharply improve confidence. On the other hand, if the field's problems cannot realistically be overcome, another study may contribute little.

The important unit of evaluation is therefore not just the gap. It is the combination of the gap, its consequences, and the study capable of addressing it.

When Weak Existing Evidence Deserves Priority Over a Completely Untouched Question

Now suppose the researcher cannot conduct a design capable of improving on the weaknesses of the early-warning studies. Question B remains important, but the particular proposed study becomes less attractive. This is the distinction between prioritizing a research problem and approving a specific research project.

The example also shows why the choice between a completely unstudied question and a poorly studied important question should not be decided by novelty alone.

Common Mistakes When Comparing Missing and Weak Evidence

Prioritize the Uncertainty That Better Research Can Most Meaningfully Reduce

If you are choosing between a question with no evidence and one with weak evidence, avoid asking which looks more novel. Compare what is uncertain, why it matters, and what your proposed research can realistically contribute.

When both gaps are important, other considerations become decisive: feasibility, ethics, resources, timeliness, stakeholder priorities, and the magnitude of the expected improvement in knowledge.

The result may sometimes be that neither proposed study should proceed in its current form. Research prioritization is not obligated to produce a winner from every pair of available gaps. The appropriate conclusion can be to redesign the project, seek a different evidence source, or leave a gap unfilled for the time being.

Compare Missing and Unreliable Evidence Before Choosing Your Research Gap

Questions About Missing and Unreliable Research Evidence

Prioritize the Consequential Uncertainty, Not the Emptiest Literature

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.