How Do You Decide Which Papers Actually Belong in Your Literature Review?

Your Search Found the Papers. Which Ones Should You Actually Keep?

A literature search can leave you with dozens, hundreds, or sometimes thousands of records. Some are obviously relevant. Others mention your topic but investigate something quite different. A few may seem too important to ignore even though they sit awkwardly at the edge of your intended scope.

This creates a deceptively difficult question: what makes a paper belong in your literature review?

The answer is not simply that the paper contains your keywords, was published in a respected journal, has many citations, or seems interesting. Searching and selecting are different tasks. A search is designed to find potentially relevant records. Study selection determines which of those records actually satisfy the intellectual and methodological boundaries of the review.

How formal that selection needs to be depends partly on the kind of review you are conducting. A systematic review normally requires explicit, predetermined eligibility criteria and a documented selection process. A narrative or traditional literature review may permit more interpretive judgment, but the underlying principle remains useful: you should be able to explain why the literature you included is appropriate for the question you are trying to answer.

Include Papers Because They Fit the Review, Not Because You Found Them

Study Selection Is About Matching Evidence to the Question

Start With the Review Question, Not With the Papers

One of the easiest ways for a literature review to lose focus is to allow the papers you happen to find to redefine what the review is about. A fascinating article appears, so the scope expands to accommodate it. Another uses a slightly different population, so that population is added too. Before long, the review question and the included literature no longer describe the same body of evidence.

A stronger approach works in the opposite direction. First establish what the review is intended to examine. Then determine what characteristics a source would need to have to contribute evidence relevant to that purpose.

For a focused review, these characteristics may be expressed as formal inclusion and exclusion criteria. Depending on the question, they might specify:

• the population or participants of interest;
• the intervention, exposure, phenomenon, concept, or topic being investigated;
• the comparator, when relevant;
• the outcomes or experiences of interest;
• the setting or context;
• eligible study designs;
• publication type;
• language;
• publication period; or
• other boundaries that are justified by the review objective.

The appropriate criteria vary considerably by review type and research question. JBI guidance, for example, emphasizes that inclusion criteria should be clear and aligned with the review objective and question. Different evidence-synthesis approaches use different frameworks to establish that alignment. The point is not to fill every possible criterion. It is to define the characteristics that actually determine whether a source can answer your question.

A Search Result Is Not the Same as an Included Study

Search strategies usually need some sensitivity. If your search retrieves only papers you already know you want to include, it may be so narrow that relevant evidence is being missed. As a consequence, database results often contain records that eventually prove ineligible.

This distinction explains why you generally should not treat every paper returned by your search as something that must be read in full. Screening progressively separates plausible candidates from material that does not belong in the final evidence base.

Relevance Is Usually the First Question

At its simplest, relevance asks whether a paper contributes evidence that can help answer the review question. Keyword overlap alone does not establish this.

Suppose your review examines university students' experiences of fully online laboratory courses. A paper about secondary-school virtual laboratories may contain nearly all of your search terms. Another may examine university students but compare laboratory software usability without investigating students' learning experiences. Both might be useful background reading, yet neither necessarily belongs in the evidence being reviewed if your eligibility criteria require university participants and evidence about student experience.

This is why researchers need to distinguish whether a study is relevant from whether it is methodologically strong. A beautifully conducted study that answers a different question does not become relevant because it is rigorous.

Eligibility Criteria Turn Scope Into Decisions

Statements such as “I will include papers related to online learning” sound reasonable until you have to decide whether a borderline paper qualifies. What counts as related? Does blended learning qualify? What about emergency remote teaching? University students only, or adult learners generally? Empirical research only, or conceptual papers too?

Eligibility criteria convert broad intentions into operational decisions. Instead of asking whether a paper “feels relevant,” you ask whether it satisfies criteria established for the review.

Criteria should be justified rather than convenient. Restricting studies to a particular country, publication period, language, or research design can change the body of evidence you ultimately see. Such restrictions may sometimes be entirely appropriate, but they should follow from the purpose and feasibility of the review rather than being imposed without consideration of what they exclude.

Selection Usually Happens in Stages

For systematic and other structured evidence reviews, selection commonly proceeds from broad screening to closer examination. Titles and abstracts can eliminate records that are clearly outside the scope. Potentially eligible records then move to full-text assessment, where the eligibility criteria can be applied using more complete information.

At the initial stage, the task is therefore not to understand every paper completely. You are deciding whether the available information is sufficient to exclude it. Knowing what to look for when screening titles and abstracts can make this stage considerably more efficient.

If the abstract leaves an important criterion uncertain, uncertainty itself is not necessarily a reason for exclusion. The safer response in a structured review is often to keep an uncertain record in consideration until enough information is available. A paper can be assessed more carefully once the full text is retrieved.

Do Not Confuse a Paper With a Study

There is another complication that becomes important once you begin screening seriously: papers and studies are not always one-to-one.

A single research project may generate a protocol, conference abstract, primary results paper, follow-up analysis, qualitative companion paper, and secondary analysis. Conversely, apparently different publications may report overlapping participants or the same dataset.

That means the question “Should I include this paper?” may eventually need to become “What study does this report represent, and what information does this particular report contribute?” When several reports originate from the same underlying research, you need to recognize and handle multiple papers from the same study or dataset rather than automatically counting each publication as independent evidence.

Different Review Types Require Different Degrees of Formality

Not every literature review is a systematic review. This matters because study-selection procedures should be proportionate to the claims the review intends to make.

A systematic review is expected to use explicit eligibility criteria and transparent methods for selecting evidence. PRISMA 2020 provides reporting guidance for systematic reviews, including reporting how studies were selected and representing the flow of records through the review. Scoping reviews likewise use explicit eligibility and selection procedures, although their objectives and the breadth of evidence considered may differ.

A narrative literature review can involve more interpretive selection. Authors may intentionally include seminal theoretical works, contrasting perspectives, methodological papers, or particularly influential studies because these sources help construct an argument or explain the development of a field.

That flexibility is not a license for arbitrary selection. If a review makes broad claims about “the literature” while quietly choosing only studies that support the author's position, the problem is not solved by calling the review narrative. Readers should still be able to understand the basis on which the literature was assembled and why particular evidence was emphasized.

From 640 Search Results to a Defensible Set of Studies

Notice what the researcher is not doing. The decision is not based on whether the paper sounds interesting, whether its findings are positive, whether the journal is prestigious, or whether the researcher agrees with the conclusions. Each decision returns to the review question and the criteria derived from it.

Some excluded papers may still be useful. The teacher-attitude study could help explain the broader educational context in the introduction or discussion. That does not mean it has to become part of the evidence being synthesized. A source can be worth reading without being eligible for inclusion in the review's analytic corpus.

Common Ways Study Selection Drifts Off Course

Use a Defensible Rule for Every Include-or-Exclude Decision

Before screening, make the connection between your research question and your selection criteria explicit. You should be able to complete this sentence:

“A source belongs in this review if...”

What follows should describe the evidence needed to answer your question, not merely characteristics of papers you happen to prefer.

For structured evidence syntheses, consistency is especially important. Applying criteria more generously to familiar authors, prestigious journals, expected findings, or studies you personally find compelling defeats much of the purpose of defining criteria in advance. The criteria should govern the papers, rather than the papers quietly governing the criteria.

You should also keep an appropriate record of why studies were excluded, particularly at the full-text stage of systematic reviews. That record supports transparency, helps resolve disagreements, and makes it much easier to explain how the final evidence base emerged from the original search.

Before You Decide That a Paper Belongs in Your Review

Questions About Choosing Papers for a Literature Review

The Right Papers Are the Ones That Fit the Question

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.