03 · What You Need to Know
Two Reviewers Are Used to Protect the Selection Decision, Not Merely to Divide the Work
Study Selection Is a Judgment Task
Eligibility criteria may look objective on paper: correct population, correct intervention, eligible study design, appropriate context. Applying those criteria to real publications is often less straightforward.
An abstract may describe the population vaguely. A study design may be difficult to classify. Two reviewers may interpret “higher education” differently. One person may overlook a sentence that establishes ineligibility. Another may exclude a study because a detail appears absent from the abstract even though the detail is reported in the full text.
Cochrane describes inclusion decisions as among the most influential decisions made during a review and emphasizes that they involve judgment. Its rationale for duplicate eligibility assessment is that using more than one reviewer can reduce both mistakes and the possibility that study selection is influenced by a single person's biases.
This matters because screening determines which evidence is allowed to contribute to the review. A statistical analysis cannot recover a relevant study that was incorrectly removed before the analysis began.
“Two Reviewers” Usually Means Independent Decisions
Simply putting two researchers in front of the same screen is not what methodological guidance generally means by duplicate independent screening.
Independent screening
Each reviewer assesses the record or report without knowing or being influenced by the other reviewer's decision before making their own judgment.
Consensus screening
Two people discuss each paper together and produce one shared decision from the beginning.
Consensus screening can involve two people, but it does not provide the same check on individual judgment. If Reviewer A immediately says, “This is obviously irrelevant,” Reviewer B's assessment may be influenced before an independent decision has been formed.
In a conventional dual-screening workflow, each reviewer records an include, exclude, or other permitted decision independently. Their decisions are then compared. Agreement proceeds automatically, while disagreements enter a predefined resolution process.
Title and Abstract Screening and Full-Text Screening Are Not Necessarily Governed by the Same Rule
This distinction is crucial.
For Cochrane intervention reviews, at least two people working independently are mandatory for determining whether potentially eligible studies finally meet the eligibility criteria. Duplicate title and abstract screening is described as desirable but not mandatory, meaning one person may conduct that initial screening under the Cochrane intervention-review standard.
JBI guidance is stricter for many of its systematic review methodologies. Its Manual for Evidence Synthesis states that selection at both title-and-abstract and full-text stages should be performed independently by two or more reviewers, with disagreements resolved through consensus or another reviewer.
| Screening stage |
Cochrane intervention reviews |
JBI systematic review approach |
| Title and abstract |
Two independent reviewers desirable, but one reviewer acceptable under the stated standard |
Two or more independent reviewers expected |
| Full-text eligibility |
At least two people working independently required |
Two or more independent reviewers expected |
| Disagreements |
Resolution process defined in advance |
Consensus or another reviewer |
There is therefore no responsible universal answer such as “every literature review always needs exactly two screeners.” The applicable methodology matters.
PRISMA Does Not Itself Tell You That Exactly Two Reviewers Must Screen
This is another distinction researchers sometimes miss.
PRISMA 2020 is primarily a reporting guideline. It asks systematic reviewers to report how studies were selected, including how many reviewers screened each record and each retrieved report, whether reviewers worked independently, and how disagreements were resolved.
PRISMA therefore makes the selection process transparent. The methodological requirement for one, two, or more reviewers comes from the review methodology you are following, such as Cochrane, JBI, or another evidence-synthesis framework.
Conduct guidance
Specifies how the review should be performed, including requirements or recommendations concerning independent reviewers.
Reporting guidance
Specifies what you should tell readers about how the review was actually performed.
This distinction prevents a common mistake: citing PRISMA as though it were the methodological authority requiring every screening decision to be duplicated.
Why Does Independent Screening Reduce Errors?
Screening errors can occur for ordinary reasons.
A reviewer may:
- misread an abstract;
- overlook an eligible subgroup;
- interpret an eligibility criterion incorrectly;
- click the wrong screening option;
- assume that missing information establishes ineligibility;
- misclassify a study design; or
- become less careful after screening hundreds of records.
Independent duplicate screening creates redundancy. If one reviewer makes an error and the other does not, the disagreement brings the record back for examination.
Cochrane cites empirical evidence suggesting that using at least two reviewers may reduce the possibility that relevant reports are discarded, while also acknowledging research exploring more efficient single-screening approaches.
The second reviewer is therefore not assumed to be infallible. The protection comes from independent judgments being compared.
A Second Reviewer Also Helps Detect Ambiguous Criteria
Disagreement is not always evidence that one reviewer made a mistake.
Suppose your inclusion criteria specify “postsecondary students.” Reviewer A includes students enrolled in vocational colleges. Reviewer B interprets postsecondary education as degree-granting universities only.
The disagreement has revealed something important: the criterion does not operationally specify what the team means by postsecondary education.
That discovery can improve the review if it occurs during pilot screening. The team can clarify the criterion and then apply the resulting interpretation consistently across subsequent records.
Without independent decisions, such ambiguity may remain invisible because one person's interpretation quietly becomes the review's interpretation.
Do the Two Reviewers Need to Have the Same Expertise?
Not necessarily.
Cochrane notes that content experts can have pre-existing views that influence assessments of relevance and validity. It suggests that although at least one review author should know the subject area, there may be advantages to having another reviewer who is not a content expert.
That does not mean content expertise is undesirable. Screening often requires enough subject knowledge to recognize terminology, populations, interventions, and study designs correctly.
A complementary team can be useful. One reviewer may bring deep domain knowledge, while another contributes methodological expertise and a less assumption-laden reading of the literature.
The essential requirement is that both understand the eligibility criteria well enough to apply them reliably.
Pilot Screening Before the Main Screening Can Prevent Hundreds of Disagreements
Two reviewers who interpret the criteria differently will not solve the problem simply by independently repeating that disagreement across 5,000 records.
Before full screening begins, test the criteria on a sample of records.
Choose a pilot sample Include clearly eligible, clearly ineligible, and genuinely ambiguous records where possible.
Screen independently Each reviewer applies the current criteria without seeing the other's decisions.
Compare decisions Identify disagreements and determine what caused them.
Clarify the criteria Resolve ambiguous terminology and establish operational rules for recurring situations.
Begin formal screening Use the agreed criteria and the predefined disagreement-resolution procedure.
This calibration is especially valuable when several screening pairs are working on the same review. Otherwise, different pairs can gradually develop different interpretations of the same eligibility criteria.
What Happens When the Two Reviewers Disagree?
Disagreement is expected. The review protocol should specify how it will be resolved.
Cochrane states that disagreements can often be resolved through discussion. If disagreement reflects different interpretations rather than a simple oversight, another person may need to arbitrate. When necessary information is unavailable, a study may remain awaiting assessment until additional information is obtained.
JBI similarly specifies consensus or a third reviewer for resolving disagreements.
A typical disagreement process
If the disagreement results from an obvious oversight
Review the relevant information and agree on the corrected decision.
If reviewers interpret a criterion differently
Discuss the intended operational meaning and apply the resolved interpretation consistently to comparable records.
If consensus cannot be reached
Use the predefined third-reviewer or arbitration procedure.
If the necessary information is unavailable
Seek additional information or use the unresolved-status procedure specified by the review methodology rather than forcing an unsupported decision.
The third reviewer should not merely vote on personal preference. The purpose is to resolve the decision against the review's criteria and available evidence.
Do You Need a Third Reviewer From the Beginning?
Not necessarily as a routine screener.
A common design uses two independent reviewers and a third person who becomes involved only when disagreements cannot be resolved through discussion. This person may also help clarify difficult eligibility questions during pilot screening.
The important point is to define the procedure before disagreements occur. Deciding who will arbitrate only after a contentious paper appears can make the process less predictable.
Should Reviewers Be Blinded to Authors, Journals, and Institutions?
In theory, blinding might reduce influence from author reputation, journal prestige, institution, or prior knowledge. In practice, meaningful blinding during screening can be difficult because titles, abstracts, terminology, interventions, and citations can reveal a study's identity.
Major review guidance generally focuses on independent eligibility assessment rather than requiring reviewers to conceal authorship or journal information.
The more important safeguard is to apply explicit eligibility criteria consistently and independently. A famous author should not receive more generous eligibility treatment than an unfamiliar research group.
What If You Are the Only Researcher Available?
This is where review type matters considerably.
If you are conducting a review under a methodology that requires independent duplicate eligibility assessment, being the only available researcher does not make the requirement disappear. You may need to involve a co-reviewer, supervisor, collaborator, or other appropriately trained person for the required stage.
Cochrane, for example, will not publish a review proposed to be undertaken by a single person and emphasizes team-based independent work for key tasks such as study selection.
For a review methodology in which duplicate title and abstract screening is desirable rather than mandatory, a single screener may be methodologically permissible at that stage. You should still report what you actually did rather than describing the process as duplicate screening.
If you are conducting a student literature review, narrative review, or another review not governed by a systematic-review methodology requiring independent duplicate selection, a second screener may not be necessary at all. Do not import procedural requirements merely to make the review sound systematic.
Can One Reviewer Screen and Another Person Check Only the Exclusions?
This is sometimes proposed as a compromise.
One reviewer screens every record. A second reviewer then checks only those marked for exclusion, or checks a sample of decisions.
This is not equivalent to conventional independent duplicate screening because the second person's role and exposure to the first decision differ. However, alternative screening approaches are an active methodological area, and some review contexts use modified procedures to reduce workload.
If you adopt such an approach, do not call it “two independent reviewers screened all records” unless that is actually what happened. Describe the procedure accurately, explain its methodological basis where needed, and ensure that it complies with the review framework you are following.
What About Automation and AI-Assisted Screening?
Automation tools can prioritize records, classify likely study designs, remove clearly ineligible records under specified procedures, or otherwise support screening. Their use does not eliminate the need to define who or what made each eligibility decision.
PRISMA 2020 specifically asks authors to report automation tools used during study selection, how they were integrated into the process, and, where applicable, how classifiers were trained or validated.
The methodological acceptability of replacing a human reviewer with automation depends on the review framework, tool, workflow, and evidence supporting that use. Do not assume that “AI-assisted” automatically satisfies a requirement for two independent human reviewers.
Watch Out
Do not write that studies were screened by “two independent reviewers” if one person made the decisions and another merely approved them afterward, if the reviewers discussed each record before deciding, or if an automated tool acted as the second screener without your methodology supporting that substitution. Describe the selection process you actually used.
Two Reviewers Do Not Fix Poor Eligibility Criteria
Duplicate screening can detect inconsistent application. It cannot rescue criteria that are fundamentally vague or poorly aligned with the review question.
If both reviewers consistently apply a bad criterion, the decisions can be perfectly reproducible and still methodologically weak.
The sequence matters:
Define defensible criteria → operationalize them clearly → pilot them → screen independently → resolve disagreements consistently.
This is why duplicate screening should be viewed as one safeguard within study selection rather than a certificate of rigor by itself.
Different Review Types Need Different Screening Procedures
The phrase “literature review” covers methods with very different expectations.
A Cochrane intervention review has explicit requirements for independent final eligibility assessment. JBI systematic reviews generally use independent duplicate screening at both major selection stages.
A narrative literature review, by contrast, may be conducted by one scholar selecting literature through an interpretive process. It would be misleading to suggest that such a review is automatically invalid because every citation was not independently screened twice.
The appropriate question is therefore:
What selection procedure does the review methodology I am claiming to use require?
Once that is established, follow it and report it accurately.