03 · What You Need to Know
What Separates Background Reviewing From Review-Based Research?
Most research uses literature without making the literature the research data
Suppose you are conducting interviews about doctoral students' experiences of supervision. Before collecting data, you review previous research to understand relevant concepts, identify what is already known, refine your research question, justify the study, and interpret your eventual findings.
That literature review performs important scholarly work. It does not necessarily constitute a separate research study.
The primary investigation remains the interview study. The literature helps establish and interpret that investigation.
This distinction matters because merely reading extensively and summarizing previous scholarship does not automatically satisfy the broader requirements of systematic research.
In review-based research, existing studies become the evidence being investigated
Now imagine a different project. Researchers ask a specific question about the effects of an educational intervention. Instead of recruiting new participants, they systematically identify all eligible studies addressing that question, apply explicit inclusion criteria, extract relevant information, evaluate methodological limitations, and synthesize the evidence.
The existing studies are now the primary evidential material of the new investigation.
Cochrane describes a systematic review as attempting to collate all empirical evidence meeting prespecified eligibility criteria to answer a specific research question, using explicit systematic methods intended to minimize bias. Its Handbook treats question formulation, eligibility criteria, searching, study selection, data collection, risk-of-bias assessment, synthesis, and interpretation as components of review methodology.
This is research without new participant-level data collection, consistent with the broader principle that research can use existing evidence.
A research question is central
A standalone evidence synthesis should normally be organized around a question or clearly specified objective rather than the vague intention to "review studies about X."
Cochrane emphasizes that defining the focus and framing the review question are foundational decisions because the question guides eligibility criteria, searching, data collection, synthesis, and presentation.
The form of the question depends on the review. An intervention review may ask about effects. A scoping review may map how a concept has been studied. A qualitative evidence synthesis may investigate experiences or perspectives. A methodological review may examine how particular methods have been used.
The review methodology should follow from the question rather than the other way around.
Systematic searching is more than typing keywords into a database
Search methodology is one of the clearest differences between casual reviewing and systematic evidence synthesis.
A researcher preparing a background section may reasonably identify influential and relevant literature without attempting exhaustive retrieval. A systematic review usually requires a much more explicit search strategy designed to identify the evidence meeting predefined criteria.
Cochrane guidance emphasizes sensitive searching, appropriate combinations of concepts and terms, and systematic study selection. Search decisions should be documented because incomplete or selective retrieval can distort the resulting synthesis.
This does not mean every research review must search every database in existence. The appropriate search depends on the review question, discipline, review type, available sources, and methodological standards. What matters is that the search strategy is defensible relative to the claims the review intends to make.
Explicit eligibility criteria reduce arbitrary selection
If reviewers choose studies because they agree with a preferred conclusion, the resulting review can provide a badly distorted account of the evidence.
Research reviews therefore establish criteria defining which studies or sources are eligible. These criteria may concern population, phenomenon, intervention, comparison, outcome, methodology, publication period, language, setting, document type, or other features relevant to the question.
Criteria should be justified rather than selected merely because they produce a convenient set of papers.
Prespecification can be particularly valuable in systematic reviews because reviewers usually know at least some of the literature before the project begins. A protocol makes consequential methodological decisions more visible and reduces opportunities to redefine the review after seeing which studies produce desirable findings.
Review research requires systematic data extraction or evidence handling
Finding eligible studies is only the beginning. Reviewers need a structured process for extracting and organizing the information required to answer the question.
Cochrane guidance stresses that data collection decisions should be transparent and designed to minimize bias and human error. It also emphasizes accurate, complete, and accessible review data.
Depending on the review, researchers might extract participant characteristics, contexts, interventions, outcomes, effect estimates, methodological features, theoretical concepts, qualitative findings, or other relevant information.
The extraction process effectively constructs the dataset or corpus that the review will analyze.
Critical appraisal can be essential because studies are not equally trustworthy
A review that treats every included study as equally credible can produce misleading conclusions.
Systematic reviews may therefore evaluate risk of bias, methodological limitations, certainty of evidence, or study quality using approaches appropriate to the research designs included.
The purpose is not to assign decorative scores. Appraisal should influence how evidence is interpreted.
A large number of studies does not automatically create strong evidence if those studies share serious methodological weaknesses.
Synthesis is more than listing one study after another
A weak literature review can become an annotated bibliography in paragraph form: Study A found this. Study B found that. Study C found something else.
Research synthesis asks what can be learned from the evidence considered together.
Researchers may compare findings, examine heterogeneity, identify recurring concepts, evaluate consistency, investigate methodological patterns, synthesize qualitative findings, estimate pooled effects, develop conceptual explanations, or identify conditions under which findings differ.
The analytical method should fit the question and evidence.
A systematic review does not have to contain a meta-analysis
Systematic review and meta-analysis are often treated as synonyms. They are not.
A systematic review refers to the broader process of systematically identifying, selecting, appraising, and synthesizing evidence to answer a question. Meta-analysis is a statistical method for quantitatively combining compatible results.
Cochrane's Handbook includes both meta-analytic methods and synthesis approaches used when meta-analysis is not appropriate.
Systematic review
A review using explicit systematic methods to identify, evaluate, and synthesize evidence addressing a defined question.
Meta-analysis
A statistical synthesis that quantitatively combines results from multiple studies when such combination is methodologically appropriate.
You can therefore conduct a systematic review without meta-analysis, and a numerical pooling exercise should not be treated as trustworthy merely because it contains a meta-analysis.
Different review questions require different review methodologies
Systematic reviews of intervention effects are only one form of evidence synthesis. Researchers may conduct reviews of diagnostic accuracy, prognosis, qualitative evidence, methodology, prevalence, etiology, or other questions.
Scoping reviews are commonly used to map a broad body of literature, clarify concepts, examine how research has been conducted, or identify characteristics and gaps in the evidence base. Qualitative evidence syntheses integrate qualitative findings. Other review traditions serve different purposes.
The existence of these approaches is important because "systematic" should not mean forcing every literature-based project into an intervention-review template.
Use a recognized methodology appropriate to what you are trying to learn.
Narrative review is a broad label, not automatically a synonym for “not research”
The term narrative review is used inconsistently. It can refer to a relatively informal overview written to introduce a field, but it can also describe substantial scholarly syntheses that critically integrate theories, concepts, and evidence.
A narrative form of presentation does not by itself determine whether the work is research. The more useful questions concern whether the review has a defined knowledge objective, how evidence was selected, how analysis was conducted, what methodological safeguards were used, and what kind of contribution is claimed.
Researchers should therefore identify their review methodology more precisely than simply calling everything that lacks meta-analysis a narrative review.
A review can produce new knowledge from existing knowledge
This initially sounds paradoxical. If every included study already exists, what is new?
The contribution can emerge from synthesis.
A systematic review may show that an apparently consistent literature becomes uncertain once risk of bias is considered. A meta-analysis may estimate an overall effect more precisely than individual studies. A qualitative synthesis may identify a conceptual pattern distributed across separate investigations. A scoping review may reveal that a supposedly well-studied topic has been examined only in narrow populations.
Nothing about this requires the underlying studies to be newly created.
It follows from the broader principle that new knowledge can arise through synthesis, reinterpretation, and stronger evaluation of existing evidence.
A review that finds no studies can still produce an informative result
Suppose researchers formulate an important question, conduct a defensible comprehensive search, and discover that no eligible studies exist.
The review has not necessarily failed.
A well-conducted empty review can establish that a specified evidence base was absent under the review's eligibility criteria at the time of searching. That finding can prevent decision-makers from assuming evidence exists and can identify a genuine research need.
The conclusion should remain precise. "We found no eligible studies" is defensible. "Nobody has ever researched anything related to this topic" may not be.
Finding a gap is not enough if the review methodology cannot support the claim
Researchers frequently conclude that "there is a lack of studies" after a limited database search. Such claims can be stronger than the search warrants.
If you intend to make systematic claims about what evidence exists or does not exist, the retrieval method needs to support those claims.
A literature search performed only to prepare a thesis introduction may reveal an apparent gap worth investigating, but it should not automatically be presented as proof that no relevant research exists anywhere.
Systematic does not mean unbiased by default
A review can have a detailed flow diagram, protocol, dozens of search terms, and several databases yet still contain consequential bias.
Eligibility criteria can be inappropriate. Searches can miss relevant evidence. Study selection can be inconsistent. Data can be extracted incorrectly. Risk-of-bias judgments can be weak. Incompatible results can be pooled. Conclusions can exceed the certainty of the evidence.
Cochrane explicitly frames systematic review methods as attempts to minimize bias, not mechanisms that make bias impossible.
This is another application of the distinction between being systematic and being genuinely rigorous.
Transparent reporting makes the review scrutinizable
Readers should be able to understand what reviewers searched, which studies were eligible, how information was extracted, how evidence was evaluated, which synthesis methods were used, and what post hoc decisions occurred.
Cochrane's current reporting guidance emphasizes that clear reporting enables readers to assess and potentially reproduce review methods and better evaluate bias in review findings.
Reporting guidance such as PRISMA can support transparency for systematic reviews, but reporting a review well is not the same thing as conducting it well. A checklist cannot retroactively repair a flawed review methodology.
Review-based research does not eliminate the need to understand primary studies
A synthesis is only as interpretable as the evidence being synthesized. Reviewers need enough substantive and methodological understanding to recognize whether studies actually address the same question, whether measures are comparable, and whether apparent differences reflect substantive effects or methodological variation.
This is why Cochrane recommends review teams with relevant methodological and subject expertise rather than treating evidence synthesis as an automated literature-management exercise.
Software can accelerate searching, screening, extraction, and analysis. It cannot decide what the evidence means without defensible methodological reasoning.
