03 · What You Need to Know
Why research can disappear before it reaches the literature
Where did the term "file drawer problem" come from?
The term is strongly associated with psychologist Robert Rosenthal's 1979 paper, The File Drawer Problem and Tolerance for Null Results. Rosenthal drew attention to a fundamental difficulty in interpreting a research literature: researchers generally cannot know how many studies have been conducted but never reported.
He illustrated an extreme possibility in which journals contain the relatively small proportion of studies producing statistically significant results while file drawers contain the much larger number producing non-significant findings. The illustration was intentionally stark. It captured a problem that remains central to evidence synthesis: the studies you can observe may be a selected subset of the studies that were actually conducted.
The physical file drawer is no longer essential to the concept. Today's missing research might remain on an investigator's computer, an abandoned manuscript, an old project folder, an undisclosed dataset, or an analysis that was completed but never written up.
The problem can begin before a manuscript even exists
It is easy to imagine the file drawer problem as a journal rejecting a paper with a null result. Research suggests that this explanation is incomplete.
A particularly informative study by Franco and colleagues followed social-science experiments from an earlier point in the research process. Projects producing strong results were substantially more likely to be published than those producing null results. Importantly, much of the difference arose before journal publication: researchers were less likely to write up and submit null findings.
A later study using a new cohort from the same research program again found evidence of a file drawer problem, although the gap was smaller. The researchers reported that the remaining pattern appeared largely related to investigators choosing not to write up statistically non-significant results rather than journals disproportionately rejecting such papers after submission.
This distinction matters because fixing the problem requires more than telling editors to accept null findings.
Researcher-side attrition
Research is analyzed but never written up, submitted, or otherwise disseminated, potentially because the findings seem uninteresting or difficult to publish.
Publication-stage selection
A completed manuscript is submitted but its probability of publication is influenced by characteristics of the findings or their perceived novelty or importance.
Both processes can contribute to selective evidence availability, but they occur at different stages.
Why are null results especially vulnerable?
Research cultures have often rewarded novel, statistically significant, theoretically supportive, or apparently consequential findings. Null or inconclusive findings may seem less exciting to researchers, reviewers, editors, institutions, and sometimes funders.
Researchers may consequently decide that a null result:
- is not worth the time required to write up;
- will be difficult to publish;
- does not tell a sufficiently interesting story;
- reflects a failed project rather than useful evidence;
- should be followed by additional analyses before dissemination;
- has lower priority than projects with clearer findings.
These decisions do not require misconduct. They can emerge from ordinary incentives and judgments about what counts as a worthwhile publication.
The cumulative consequence, however, can be serious. If many researchers independently decide that statistically non-significant findings are not worth reporting, the published literature becomes selectively enriched with statistically significant findings.
A null result does not mean that nothing happened
The language surrounding the file drawer problem can accidentally reinforce the very behavior that creates it. Researchers sometimes call statistically non-significant studies "failed studies" or say that they "found nothing."
That interpretation is often too strong.
A statistically non-significant result can arise because the true effect is small or absent, but it can also reflect limited precision, insufficient sample size, substantial variability, measurement problems, or an estimate compatible with several plausible effect sizes.
Statistically non-significant result
The analysis did not cross the chosen statistical-significance threshold. This does not, by itself, establish that the true effect is exactly zero.
Evidence of no meaningful effect
A stronger substantive conclusion that requires consideration of the estimated effect, uncertainty, study design, precision, and what magnitude would actually matter.
A well-designed study with an informative null or near-null estimate can substantially change what researchers should believe. Leaving it in the file drawer removes that information from everyone else's evidence base.
The file drawer problem is not identical to publication bias
The two concepts overlap closely, and the terminology is sometimes used interchangeably. It is nevertheless useful to distinguish them.
Publication bias is the broader problem in which the probability or visibility of dissemination depends on study findings. It can involve researcher submission decisions, journal publication decisions, publication timing, selective outcome reporting, or other mechanisms.
The file drawer problem focuses more specifically on research that remains undisclosed or insufficiently disseminated, traditionally because statistically non-significant or unexciting findings are left in researchers' files rather than entering the accessible scientific record.
File drawer problem
Research findings remain unseen or insufficiently disseminated, often because investigators do not pursue publication or other reporting of null or unremarkable findings.
Publication bias
The broader systematic distortion created when whether, when, or how research becomes available is related to what the research found.
The file drawer problem can therefore be understood as one important route through which publication bias develops.
Not every unpublished study belongs in the file drawer problem
This qualification is important.
Research can remain unpublished for reasons unrelated to its results. Investigators may leave academia. Collaborations may collapse. Funding may end. A student may graduate. Data problems may render a study uninterpretable. Researchers may lack time to prepare the manuscript. A project may become obsolete before publication.
Research on conference nonpublication has similarly found practical reasons such as lack of time among investigators' explanations for failing to publish completed work.
If studies disappear randomly with respect to their findings, the evidence base becomes incomplete, but this does not necessarily create the characteristic directional distortion associated with the file drawer problem.
Watch Out
Do not label every unpublished study a "file drawer study" or assume that it must contain a null result. The methodological concern is selective non-dissemination related to findings. The actual reason an individual study remained unpublished may be unknown.
How does the file drawer problem distort a literature review?
Suppose a research question has been tested repeatedly. Studies with larger or statistically significant effects are more likely to become visible, while studies with estimates close to zero disproportionately remain undisclosed.
A literature reviewer then encounters a selected evidence base.
The consequences can include:
- an exaggerated apparent effect;
- greater apparent consistency among studies;
- overconfidence that a phenomenon replicates;
- underrepresentation of uncertainty;
- an inflated impression that a hypothesis has repeatedly been supported;
- misleading meta-analytic estimates;
- unnecessary repetition of research whose hidden results already provide relevant information.
The published studies themselves do not need to be wrong for this distortion to occur. The problem is that they may not be representative of all the studies conducted.
A simple thought experiment shows why the problem matters
Imagine that 20 independent studies test an intervention that actually has little or no meaningful effect. By chance alone, some studies may nevertheless produce apparently impressive results.
If researchers preferentially write up and publish those unusual results while most unremarkable studies remain unseen, a later literature search can encounter what looks like a series of successful replications.
The apparent pattern comes partly from the selection process rather than from the underlying phenomenon.
This is close to the concern Rosenthal used the file drawer metaphor to illustrate. The extreme numerical example in his original discussion should not be interpreted as a literal estimate of how research publication works. Its value lies in showing how selective visibility can create a misleading scientific record.
The problem becomes especially important in meta-analysis
Meta-analysis combines quantitative estimates from multiple studies. This can increase precision and provide a clearer estimate of an effect, but only from the studies that enter the synthesis.
If the available studies are systematically selected according to their results, statistical precision does not repair the selection process. A highly precise pooled estimate can still summarize an unrepresentative evidence base.
This is why the file drawer problem became particularly prominent in discussions of meta-analysis. Combining published studies assumes that the available studies provide an informative representation of the relevant research. Selective disappearance challenges that assumption.
How do we know file drawers exist if the studies are hidden?
This is a genuinely difficult methodological problem. Research that leaves no public trace is, by definition, hard to count.
One strong approach is to identify studies before their results are known and then follow what happens to them. Sources can include research registries, approved research proposals, ethics records, conference cohorts, funded projects, or other inception points.
Franco and colleagues used this logic in social science by following accepted research proposals and examining whether the eventual findings predicted writing, submission, and publication. Their work found substantially greater publication among studies with statistically significant results and showed that much of the attrition occurred because investigators did not write up null findings.
A 2025 follow-up using later projects from the same program again found a statistically significant publication gap, but one smaller than in the earlier cohort. The authors suggested that increasing acceptance of null findings and open-science practices may be contributing to improvement, while cautioning against assuming the problem has disappeared.
Open science can make the invisible research record more visible
Several research practices can reduce dependence on eventual journal publication as the only evidence that a study existed.
These include:
- prospective study registration;
- preregistration of hypotheses and analyses;
- registered reports;
- public results reporting;
- preprints;
- institutional repositories;
- data and materials sharing;
- research registries that preserve project records.
These practices address different problems. A preregistration can reveal that a study was planned, while public results reporting can reveal what it found. A registered report can reduce result-dependent publication decisions by obtaining an in-principle publication commitment before the results are known.
No single practice solves the entire problem, but together they make it harder for completed research to disappear without leaving a trace.
Registered reports directly change the incentive structure
Traditional publication decisions often occur after the findings are known. Registered reports change the sequence.
In the registered-report model, a journal evaluates the importance of the research question and the quality of the proposed methods before data collection or before results are known. If the protocol is accepted in principle and the researchers follow the approved methodology, publication is not supposed to depend on whether the eventual results are statistically significant or support the hypothesis.
This does not guarantee perfect research. It does directly weaken one mechanism that can feed the file drawer: the fear that an informative null result will make the study unpublishable.
Preregistration alone does not empty the file drawer
Registering a study makes its existence more visible, but registration does not automatically make the eventual results available.
A researcher can preregister an experiment, complete it, obtain null results, and still never write up or publicly report those results. The difference is that an external observer may now be able to see that the study was planned.
For this reason, registration and results reporting should be distinguished. The first creates a trace of the research. The second contributes the findings to the evidence base.
Searching grey literature can uncover research that escaped journal publication
Some studies that look absent from the journal literature are not truly hidden. Their findings may survive in dissertations, institutional reports, working papers, conference materials, preprints, or other grey-literature sources.
Searching these sources can therefore recover some evidence that would otherwise function like file-drawer research from the perspective of a journal-only reviewer.
It cannot recover studies that were never disseminated anywhere. That is why grey-literature searching reduces the problem without guaranteeing a complete research record.
Study registries can expose the empty space in the evidence base
A registry may identify a completed study even when no results can be located.
That does not tell you what the missing study found. It does tell you something important: the set of studies with accessible results is not the entire set of studies known to have been conducted.
Reviewers can therefore search for unpublished and ongoing studies and trace completed records to conference abstracts, preprints, reports, journal articles, or investigators.
When results remain unavailable, the study should not simply vanish from the reviewer's understanding of the evidence landscape.
Do not guess what is inside the file drawer
This is perhaps the most important limitation.
Publication bias research may show that statistically non-significant findings are less likely to be disseminated in a particular research context. That does not allow you to infer that every individually unpublished study found no effect.
An unpublished study could have found:
- a null result;
- a favorable result;
- an unfavorable result;
- an imprecise result;
- contradictory outcomes;
- methodological problems that prevented interpretation;
- results that were simply never written up because the research team ran out of time.
At the level of a systematic review, missing studies can create a risk of bias. At the level of an individual missing study, the correct result is usually: unknown.
Can statistical methods estimate what might be missing?
Meta-analysts have developed methods intended to investigate possible publication bias or assess the sensitivity of findings to missing studies. Rosenthal's original paper itself proposed a calculation concerning tolerance for unpublished null results, an approach associated with the fail-safe N tradition.
Modern evidence synthesis generally treats such methods cautiously. Statistical approaches can explore how conclusions might behave under assumptions about missing evidence, but they cannot directly observe studies that left no record.
Funnel plots and related tests likewise cannot prove that a file drawer exists or tell you exactly what is inside it. Funnel-plot asymmetry can have explanations other than publication bias.
Statistical diagnostics are therefore supplementary evidence, not a substitute for searching for missing research and understanding the publication process.
The file drawer problem wastes research as well as distorting it
The consequences extend beyond biased effect estimates.
If null or inconclusive findings remain inaccessible, later researchers may repeat approaches that have already been tried. Funders may support redundant work. Participants may contribute to studies that could have been better designed if earlier results had been visible. Theoretical claims may survive because contradictory evidence never enters the conversation.
Researchers proposing systems for more complete results reporting have therefore framed the file drawer not only as a publication-bias problem but also as a source of avoidable research waste.
Null results can be scientifically useful
The solution is not to publish every analysis merely because it produced a p-value above 0.05. Poorly designed research does not become informative simply by being null.
What matters is whether a study was capable of providing useful evidence about a meaningful question.
A rigorous study that produces an estimate close to no meaningful effect can constrain theories, challenge earlier findings, inform sample-size planning, prevent redundant research, and improve meta-analysis. A badly underpowered or methodologically compromised study may contribute far less regardless of whether its p-value is significant.
The case for opening the file drawer is therefore a case for making informative research available irrespective of whether its results are exciting.
