Can Lack of Real-World Evidence Be a Research Gap?

What If Something Works in a Study but We Do Not Know What Happens in Routine Use?

Some research questions appear well answered until you examine the conditions under which the evidence was produced.

An intervention may have been tested with carefully selected participants, specially trained personnel, unusually close monitoring, strong adherence support, additional resources, or procedures that differ substantially from ordinary practice. Under those conditions, the results may be convincing.

But what happens when the intervention leaves the research environment and encounters routine users, ordinary workloads, imperfect adherence, variable resources, and everyday implementation constraints?

Can the lack of evidence under those conditions constitute a research gap?

Yes, When Controlled Evidence Does Not Resolve What Happens Under Routine Conditions

When Does Missing Real-World Evidence Become a Genuine Research Gap?

First, “Real-World Evidence” Has a Specific Meaning in Some Fields

The phrase real-world evidence is used broadly across disciplines, but it also has formal meanings in regulatory health research. The U.S. Food and Drug Administration distinguishes real-world data from real-world evidence and maintains a dedicated program concerning their use in healthcare decision-making.

In regulatory contexts, real-world data concern routinely collected information relating to patient health status or healthcare delivery, while real-world evidence concerns clinical evidence about the use and potential benefits or risks of a medical product derived from analysis of real-world data.

Outside health research, researchers often use real-world evidence more loosely to refer to evidence generated under routine, naturally occurring, operational, or practice conditions. If you are working outside the regulatory health context, define what you mean rather than assuming the phrase has one universal disciplinary definition.

The Underlying Question Is Often “Can It Work?” Versus “Does It Work Here?”

Controlled studies can be designed to determine whether an intervention produces an effect under conditions that give it a strong opportunity to do so. Other studies are designed to inform decisions about what happens when the intervention is used under ordinary conditions.

PRECIS-2 describes this distinction as a continuum between explanatory and pragmatic trial intentions. Explanatory trials tend toward ideal conditions, whereas pragmatic trials are designed to support decisions under usual conditions. The framework emphasizes that these are not two mutually exclusive boxes. Trial features can be more or less pragmatic across several domains.

Both questions can be scientifically valuable. Problems arise when evidence answering the first is treated as though it completely answers the second.

“Real World” Is Not a Single Place

A hospital is real. A laboratory is real. A university research site is real. The phrase does not mean that controlled research somehow occurs outside reality.

The useful distinction concerns how closely study conditions resemble the circumstances in which the intervention, policy, technology, or practice is intended to be used.

PRECIS-2 makes this concrete through nine domains: eligibility, recruitment, setting, organization, flexibility of delivery, flexibility of adherence, follow-up, primary outcome, and primary analysis. A study can be pragmatic in one domain and relatively explanatory in another.

This is a more defensible way to discuss real-world relevance than simply labeling one study "real world" and another "artificial."

Routine Conditions Can Change More Than the Effect Size

When an intervention moves into ordinary practice, many aspects of its performance may become important.

A real-world evidence gap should specify which of these uncertainties matters. "There is little real-world research" is much less informative than explaining that effectiveness under routine staffing and adherence conditions remains uncertain.

A Controlled Study Is Not Inferior Because It Is Controlled

Controlled conditions can reduce noise, isolate causal effects, standardize delivery, and answer questions that would be difficult to resolve in routine practice. Those are strengths when they match the research objective.

Pragmatic designs involve their own trade-offs. Routine conditions may introduce heterogeneity, imperfect adherence, variation in implementation, missing data, and confounding that complicate interpretation.

Real-World Evidence Is Not the Same as Observational Evidence

Real-world data are often observational, such as electronic health records, registries, administrative records, routine platform data, or other information generated outside a traditional trial. But observational and real-world are not synonyms.

Randomized trials can be highly pragmatic and embedded within routine practice. Conversely, an observational study can use highly selected data or analytic procedures that do not necessarily make its conclusions broadly applicable.

Study design and degree of pragmatism should therefore be considered separately.

Real-World Data Are Not Automatically High-Quality Data

Routine data can offer large samples and reflect actual practice, but they were often collected for purposes other than the research question now being asked.

Important variables may be missing, inconsistently recorded, measured differently across sites, or affected by changes in systems and practice. Confounding and selection can complicate causal interpretation in observational analyses. Data provenance, completeness, measurement quality, and fitness for use therefore require careful evaluation.

More data do not automatically mean better evidence. That familiar methodological nuisance has survived the era of big data quite comfortably.

A Real-World Gap Can Overlap With a Missing Setting

If an intervention has been tested only in specialist research centers and is intended for routine community use, the literature may contain both a missing-setting gap and a lack of real-world evidence.

The distinction lies in emphasis. A missing-setting gap asks whether evidence applies in a particular environment that is underrepresented. A real-world evidence gap asks more broadly whether findings hold under the ordinary conditions of actual implementation and use.

Sometimes the same study can address both.

Real-World Evidence Can Also Test External Validity

A tightly controlled study may deliberately restrict participants, providers, or settings to improve internal validity or answer a focused explanatory question. Those choices can leave uncertainty about applicability beyond the study conditions.

Real-world studies can provide evidence about whether findings extend to broader users and routine environments. That does not guarantee generalizability everywhere. The target population and intended context still need to be specified.

If the research product is specifically a prediction model or algorithm, the more precise issue may instead be lack of external validation.

Implementation Failure and Intervention Failure Are Not the Same

Suppose an intervention performs poorly in routine practice. One interpretation is that the intervention itself is ineffective. Another is that it was rarely delivered as intended, users could not access it, staff lacked resources, or organizations could not sustain it.

Real-world research can help distinguish these possibilities, especially when outcome data are accompanied by evidence about implementation and exposure.

This matters because an intervention that fails under routine conditions may require adaptation, additional implementation support, or abandonment. Those are different conclusions, and outcome data alone may not distinguish them.

Routine Use Can Reveal Effects That Controlled Studies Are Poorly Positioned to Detect

Research conducted under selected conditions may involve relatively homogeneous participants, close monitoring, high adherence, or limited follow-up. Wider implementation can expose an intervention to more diverse users and circumstances.

That may reveal heterogeneity of effects, implementation problems, uncommon adverse outcomes, or resource demands that were not apparent during earlier studies.

Again, this does not make routine data intrinsically more truthful. It makes them informative about a different set of conditions.

How Do You Establish That Real-World Evidence Is Actually Missing?

First identify what kind of evidence already exists. Do not assume that every randomized trial is explanatory or that every observational study is real-world. Examine eligibility criteria, recruitment, setting, organization, delivery, adherence support, follow-up, outcomes, and analysis.

Search terms may include pragmatic, effectiveness, routine practice, usual care, implementation, registry, administrative data, electronic records, practice-based research, naturalistic, and discipline-specific terminology.

Then specify the unresolved question. A defensible statement might be: "Controlled trials demonstrate efficacy under intensive implementation support, but evidence concerning effectiveness and sustained use under routine institutional conditions remains limited."

That is considerably stronger than claiming that the intervention has never been studied in the real world.

When an Educational Technology Leaves the Research Trial

The research question is no longer simply whether AI tutoring can improve learning. It concerns what happens when the intervention must function without the unusually supportive conditions of the original studies.

Common Mistakes When Claiming a Real-World Evidence Gap

How to Decide Whether Lack of Real-World Evidence Justifies Your Study

Start with the decision that existing research cannot adequately inform. Then identify which features of routine use are missing from the evidence.

A persuasive real-world evidence gap therefore specifies what controlled evidence already establishes, which routine conditions remain inadequately represented, why those conditions matter, and what decision better practice-based evidence would support.

Before Claiming Lack of Real-World Evidence as Your Research Gap

Frequently Asked Questions About Real-World Evidence Gaps

Evidence That Something Can Work Is Not Always Evidence of What Happens in Routine Use

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.