How Do You Keep Track of Why Studies Were Excluded?

Do You Really Need to Record Why Every Study Was Excluded?

During screening, exclusion decisions accumulate quickly. One study has the wrong population. Another examines the wrong intervention. A third turns out to be a commentary rather than empirical research. After several hundred decisions, it can become surprisingly difficult to remember why any particular paper disappeared from consideration.

Does every exclusion need to be documented?

The answer depends partly on the stage of screening and the type of review. In systematic reviews, detailed documentation becomes particularly important when a report has survived initial screening and is subsequently excluded after full-text assessment. PRISMA 2020 expects reviewers to report primary reasons for excluding reports that appeared potentially eligible, while Cochrane requires reviewers to document studies that might reasonably have been expected to meet the eligibility criteria but did not.

The objective is not paperwork for its own sake. A well-maintained exclusion record allows you to reconstruct how the final evidence base emerged from the literature you originally identified.

Record Exclusions at the Level of Detail the Screening Stage Requires

Exclusion Records Are Part of the Evidence Trail

Study Selection Produces More Than a List of Included Papers

A systematic search may identify hundreds or thousands of records, while the completed review includes only a fraction of them. Readers therefore need some way to understand what happened between those two points.

PRISMA 2020 asks systematic reviewers to describe the results of the search and selection process from the number of records identified through the number of studies ultimately included. Its flow diagram distinguishes records identified, records screened, reports sought for retrieval, reports assessed for eligibility, and studies included.

Exclusion documentation provides the explanation behind those numbers.

If 120 full-text reports were assessed and only 42 studies were included, readers should not have to take the disappearance of the remaining reports on faith. They should be able to see the principal reasons potentially eligible evidence did not enter the review.

Title and Abstract Exclusions Usually Need Less Detail

Initial screening can involve enormous numbers of records. Recording a customized reason for every title and abstract exclusion may add substantial work without improving the transparency of the review proportionately.

Cochrane states that during the initial title and abstract screening stage, only the number of records excluded needs to be recorded. Individual reasons for every clearly irrelevant record are not required.

This makes sense because many exclusions at this stage are straightforward. A search about educational interventions may retrieve articles about medical imaging, agricultural systems, or unrelated uses of the same terminology. Readers do not generally need a citation-by-citation explanation for every obvious mismatch.

Your review methodology or screening system may collect more detailed information during initial screening, and that can be useful internally. The important point is that the reporting expectations are not necessarily identical at every stage.

Full-Text Exclusions Need a Defensible Reason

Once a report reaches full-text assessment, it has already survived preliminary screening as potentially eligible. Excluding it therefore requires closer justification.

PRISMA 2020 item 16b asks reviewers to cite studies that might appear to meet the inclusion criteria but were excluded and to explain why they were excluded. Cochrane similarly requires reviewers to list studies that might reasonably have been expected to be included but were excluded after full-text assessment, together with the reasons for exclusion.

The reason should correspond to an actual eligibility criterion.

Good reasons might include:

• ineligible population;
• ineligible intervention or exposure;
• ineligible comparator, where required;
• ineligible study design;
• ineligible setting or context;
• publication type not eligible;
• outside a justified date criterion; or
• another explicitly defined eligibility requirement not met.

The wording should be specific enough that someone familiar with the criteria can understand what failed.

“Not Relevant” Is Usually Too Vague

A screening log containing hundreds of entries marked simply “not relevant” is not especially informative.

Why was the study not relevant?

Was the population wrong? Did it examine a different intervention? Was it a review article when only primary research was eligible? Was the context outside scope?

Specific categories make exclusion decisions easier to audit and easier to summarize later.

The exact categories should follow your review's criteria. Do not create generic exclusion labels that obscure the actual methodological reason.

Use One Primary Reason When Several Criteria Fail

A full-text paper may fail more than one eligibility criterion.

Suppose a study involves secondary-school students, uses a non-generative automated feedback system, and reports only a conceptual discussion rather than empirical data. You could record three reasons.

For study-selection reporting, however, assigning one primary reason can make the process clearer and prevent exclusion counts from exceeding the number of reports assessed.

Cochrane recommends assessing eligibility criteria in an order of importance so that the first failed criterion can serve as the primary reason for exclusion. The appropriate order depends on the review.

This is not the only possible internal workflow, but it provides a reproducible way to handle studies that fail several criteria.

Define Exclusion Categories Before Full-Text Screening Where Possible

If you invent exclusion categories one paper at a time, your screening log can quickly become chaotic.

You may end up with:

• wrong participants;
• wrong population;
• not our population;
• students too young;
• school students; and
• population mismatch.

All may describe essentially the same eligibility failure.

Instead, derive a manageable set of exclusion categories from the eligibility criteria before full-text screening. For example:

• Population;
• Intervention or phenomenon;
• Comparator;
• Outcome, only if outcome measurement legitimately determines eligibility;
• Study design;
• Context or setting;
• Publication characteristics; and
• Other prespecified criterion.

You can retain a brief note for unusual cases without turning every variation into a new category.

This also supports consistent application of the inclusion and exclusion criteria across the review.

Your Exclusion Reason Must Reflect Evidence You Actually Have

A reason should not be more certain than the evidence supporting it.

Suppose an abstract describes “students” but never states whether they are university students. You should not record “ineligible population” merely because you suspect they were school students.

If the full text confirms that participants were secondary-school students, the population reason becomes defensible. If the full text remains unclear, the problem is insufficient information rather than demonstrated population ineligibility.

This is why missing information should not be converted into an invented eligibility failure.

“Full Text Unavailable” Is Not the Same as “Excluded”

PRISMA 2020 distinguishes reports sought for retrieval from reports actually retrieved and assessed for eligibility. This distinction matters because a report you cannot obtain has not necessarily failed an eligibility criterion.

If a potentially relevant report cannot be retrieved, record the retrieval outcome accurately rather than inventing a substantive exclusion reason.

Depending on the review, you may try:

• institutional library services;
• interlibrary loan;
• author or institutional repositories;
• other reports of the same study;
• study registries;
• supplementary sources; or
• contacting the investigators.

If sufficient information still cannot be obtained, follow the procedure specified by your review methodology for unresolved or unavailable reports.

This distinction is part of knowing when and why full-text retrieval is necessary rather than treating access problems as eligibility decisions.

A Related Report Is Not Necessarily an Excluded Study

Another source of confusion arises when several publications describe the same underlying study.

Suppose a trial has a protocol, primary results article, secondary analysis, and follow-up publication. You may not need to count all four as separate included studies, but neither should the additional reports automatically be entered in an excluded-studies table as though they failed eligibility.

Cochrane treats the study, rather than each individual report, as the primary unit of interest and requires multiple reports from the same study to be collated.

If a report belongs to an included study, link it to that study. Do not manufacture an exclusion reason simply because it is not the primary publication.

This is why multiple reports from the same study need study-level organization.

Duplicate Publication Needs Careful Labeling Too

Similarly, “duplicate” can mean different things.

An identical bibliographic record retrieved twice can be removed during deduplication. A separate publication containing overlapping data may need to remain linked to the underlying study because it contains useful information.

Do not use “duplicate” as a catch-all exclusion category without knowing whether you are dealing with a duplicate search record, a secondary report, or problematic duplicate publication.

The distinction matters because overlapping publications can distort a review if they are mistakenly counted as independent evidence, while legitimate secondary reports can improve understanding of the study.

Keep the Original Decision and the Final Decision Separate

Screening decisions can change.

A title and abstract may initially be marked “include” and later be excluded after full-text assessment. Two reviewers may disagree and later reach consensus. A study may be temporarily uncertain while additional information is sought.

Your screening system should preserve enough information to distinguish the stage and status of each decision.

Not every review requires every field shown here. The principle is to preserve enough of the decision trail that you can reconstruct what happened without relying on memory.

Do Not Overwrite Disagreements Without Resolving Them

When two reviewers screen independently, they may assign different decisions or different exclusion reasons.

One reviewer might identify an ineligible population while another believes the population qualifies but the study design does not. That disagreement can reveal either a factual misunderstanding or an ambiguity in the criteria.

The appropriate procedure depends on the review methodology. Cochrane requires at least two people working independently to make final inclusion decisions for potentially eligible studies, with a predefined process for resolving disagreement. JBI similarly uses independent review and consensus or an additional reviewer in its evidence-synthesis methods.

When the disagreement is resolved, record the final decision according to the agreed procedure. Do not simply overwrite one reviewer's entry without preserving whatever audit information your review process requires.

Exclusion Reasons Should Follow the Criteria, Not the Findings

Imagine a study that meets your population, intervention, setting, and design criteria but reports no statistically significant effect.

“No effect” is not an exclusion reason unless your review has adopted a highly unusual and methodologically defensible eligibility rule based on something other than the direction of the result. In most systematic reviews, selecting studies according to their findings would introduce serious bias.

The same applies to:

• results contradict our hypothesis;
• effect too small;
• findings not interesting;
• authors reached the wrong conclusion; or
• study does not support the argument.

Eligibility should be determined independently of whether you like what the study found.

If the Criteria Change, Revisit the Exclusion Log

Suppose you initially exclude studies from vocational colleges because your population criterion is limited to universities. Later, you make a justified amendment that expands eligibility to recognized higher-education institutions.

Previously excluded studies affected by the change need to be identified and reassessed.

This is one reason detailed exclusion records are useful. If your log says only “not relevant,” finding the studies affected by a population amendment may require screening the entire excluded set again. If the reason is coded as “population: vocational college,” the relevant records can be identified much more efficiently.

When eligibility criteria change after screening has begun, the exclusion log becomes part of the mechanism for applying that amendment consistently.

Your Exclusion Log Feeds Directly Into PRISMA Reporting

PRISMA 2020 asks reviewers to report the flow of records and reports through the selection process and to provide primary reasons for reports excluded after full-text assessment.

If you record those reasons while screening, constructing the final flow diagram becomes largely an accounting task.

If you do not, you may reach manuscript preparation with 73 excluded full texts and no reliable memory of why each one was rejected. Reconstructing those decisions months later is both tedious and vulnerable to error.

The same records help when deciding whether and how the review's study selection should be represented in a PRISMA flow diagram.

Good Documentation Also Protects Against Accidental Re-Screening

Excluded studies have a habit of returning.

You may encounter the same paper through citation searching, an updated database search, a colleague's recommendation, or another report from the same project. Without a clear exclusion record, you can spend time reassessing it and perhaps even make a different decision because you no longer remember the original reasoning.

A searchable study-selection log lets you answer quickly:

Have we seen this before, and what happened to it?

That becomes increasingly valuable in large or long-running reviews.

Traditional Literature Reviews May Need Less Formal Documentation

Not every literature review requires a PRISMA-style excluded-studies table.

A traditional narrative review may use a more interpretive literature-selection process and may not formally document every source considered but not cited. Imposing systematic-review procedures on such a review can create an appearance of methodological formality that the review was never designed to provide.

Still, keeping notes about why apparently relevant papers were not used can be valuable, particularly for a thesis, dissertation, evidence-informed review, or collaborative project. The documentation may be less formal, but it can still improve consistency and make later revisions easier.

The level of documentation should therefore match the review methodology and the claims the review intends to make.

Turning Full-Text Decisions Into a Usable Exclusion Log

At the end of screening, the team can count how many reports were excluded for each primary reason and identify specific studies that readers might reasonably expect to have been included.

The log therefore serves two purposes simultaneously: it supports day-to-day screening decisions and provides the data required for transparent reporting later.

Common Mistakes When Recording Study Exclusions

Build the Exclusion Log Before You Need the Flow Diagram

The easiest time to document an exclusion reason is when the paper is open and the decision has just been made.

Create a small, controlled set of categories derived from your eligibility criteria. Keep them specific enough to explain the decision but broad enough that equivalent reasons do not fragment into dozens of labels.

A spreadsheet can be sufficient for many reviews. At minimum, consider fields for the citation or report ID, screening stage, decision, primary exclusion reason, optional explanatory note, and final consensus decision. Collaborative systematic review software may provide these fields automatically.

Use controlled terminology. If the category is “Ineligible population,” do not alternate among “wrong participants,” “population mismatch,” and “wrong sample” unless those distinctions actually matter.

By the time manuscript preparation begins, your exclusion log should already be able to answer three questions: how many reports were excluded after full-text assessment, why they were excluded, and which apparently eligible studies need explicit explanation for readers.

Before You Finish a Full-Text Exclusion

Questions About Recording Study Exclusion Reasons

Record Enough to Explain Where the Potentially Eligible Studies Went

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.