03 · What You Need to Know
Different Reviewers Look for Different Problems
Start with a structured review by the research team
The investigators themselves should perform the first systematic review. This is more than proofreading. The team should challenge whether the protocol describes the study they actually intend to conduct and whether the sections agree with one another.
Each primary question should have a corresponding source of evidence and analytical pathway. Eligibility criteria should agree with recruitment procedures. Instruments named in the methods should match the variables described in the analysis. Timelines should be feasible. Consent materials and data collection procedures should describe the same participant experience.
This internal review is also the point at which researchers should confirm that the protocol remains aligned with the research questions, objectives, and hypotheses.
Familiarity, however, creates a limitation. The same people who developed an assumption may be the least likely to notice that it has never actually been written down. Independent or specialist review can therefore add considerable value.
Subject-matter experts can test the scientific logic
A reviewer who understands the substantive field can ask whether the research problem is meaningful, whether the rationale reflects current knowledge, whether the questions are appropriately framed, and whether important domain-specific considerations have been overlooked.
Subject expertise also matters when seemingly technical methodological choices have disciplinary implications. A measurement may be common in one population but unsuitable in another. A follow-up interval may be too short to observe the phenomenon of interest. An intervention described as realistic by the research team may be implausible in actual practice.
The appropriate expert is therefore not simply someone with an advanced degree. The reviewer should understand the phenomenon, population, intervention, setting, or evidence base relevant to the study.
A methodologist can examine whether the design can answer the question
Methodological review focuses on the architecture of the study. Does the design match the type of inference the researchers want to make? Are comparison groups appropriate? Is the sampling strategy defensible? Are measurements compatible with the constructs? Does the sequence of procedures create avoidable bias?
This review should look across the protocol rather than treating each section as independent. A reasonable sampling strategy can still be incompatible with the analysis, and a strong instrument can still measure the wrong construct for the research question.
A useful methodological review therefore examines whether sampling, measurement, data collection, and analysis fit together as one evidence-generating system.
Quantitative studies may benefit from statistical review before data are collected
Statistical expertise is particularly valuable when the study relies on sample-size calculations, complex sampling, randomization, repeated measurements, clustered observations, multivariable models, interim analyses, missing-data methods, or other consequential quantitative decisions.
Importantly, statistical review should not be postponed until the dataset is ready for analysis. Guidance developed by biostatisticians for clinical and translational research emphasizes that protocol review before data collection should consider the research strategy broadly, not merely the sample-size calculation and statistical-analysis section.
A statistician may identify that the proposed outcome cannot be captured reliably, the planned sample will not support the intended analysis, the data structure requires a different model, or important variables needed for analysis are absent from the collection plan. Those are design problems, not problems to hand to a statistician after data collection.
Qualitative studies may require methodological expertise specific to the approach
A qualitative study should not be reviewed as though its quality depends on satisfying quantitative conventions. The relevant reviewer may need expertise in the particular methodological tradition being used and in the relationship among the research question, sampling logic, data-generation method, researcher role, analytical approach, and claims the study intends to make.
For example, a reviewer might examine whether semistructured interviews can generate evidence appropriate to the question, whether purposive or iterative sampling is justified, whether the planned analysis is described with enough specificity, and whether legitimate adaptation has been distinguished from incomplete planning.
The broader lesson is that methodological review should be methodology-appropriate. Expertise in research methods is not completely interchangeable across designs.
People responsible for implementation can identify feasibility problems
A protocol can be scientifically coherent and still be impossible to execute as written. Data collectors, research assistants, laboratory personnel, site coordinators, information-technology staff, clinicians, teachers, fieldworkers, or other people who will implement the study may notice practical problems that are invisible from a desk.
Can participants realistically complete the assessment in the proposed time? Can the required records actually be accessed? Does the data system capture the identifiers needed for follow-up? Can the intervention be delivered at the planned frequency? Are recruitment assumptions plausible?
SPIRIT 2025 describes trial protocols as a basis for review of scientific, ethical, safety, and operational issues by multiple stakeholders. Although its recommendations apply specifically to randomized trials, the distinction is useful more broadly: operational feasibility deserves explicit review rather than being assumed from methodological elegance.
People with relevant lived experience can identify problems researchers may miss
Some studies may benefit from input from patients, participants, community representatives, students, practitioners, or other people with direct experience of the population or context being studied.
Their contribution is not a substitute for scientific or ethical review. They may instead identify whether recruitment materials are understandable, procedures are burdensome, outcomes matter to the people affected, terminology is inappropriate, or implementation assumptions are unrealistic.
The relevance of such involvement varies across studies. Where meaningful stakeholder or public involvement is appropriate, it should be integrated because it can improve the study, not merely added as a ceremonial endorsement.
Ethics review addresses a distinct responsibility
Research ethics committees or institutional review boards evaluate research involving human participants according to applicable ethical and institutional requirements. Their responsibilities can include examining the balance of risks and potential benefits, participant selection, informed consent, privacy, confidentiality, safeguards for participants, and other ethical aspects of the protocol.
Formal requirements vary across jurisdictions and institutions. For randomized trials, SPIRIT 2025 states that the full protocol must be submitted to a research ethics committee or institutional review board for approval before participants are enrolled.
Other research may be exempt from formal review, eligible for a different review pathway, or governed by other procedures. Researchers should verify the requirements that apply to their institution and study rather than assuming that one process applies universally.
Ethics approval should not be treated as a certificate of methodological perfection
Ethical and scientific considerations often overlap because exposing participants to a study incapable of answering its question can itself raise ethical concerns. Nevertheless, an ethics committee is not automatically a substitute for every form of methodological or statistical review.
A protocol may receive ethics approval while still benefiting from specialist advice on measurement, sampling, qualitative methodology, statistical modeling, intervention delivery, or data management.
Methodological review
Asks whether the design and methods can generate evidence capable of answering the research question.
Ethics review
Evaluates the ethical acceptability of the proposed research under the applicable review framework, including participant protections where relevant.
The functions can overlap, but they should not be casually treated as identical.
Data-management or information-security review may be necessary
Studies involving sensitive information, linked records, large datasets, identifiable data, complex databases, or specialized technical systems may need review from people with expertise in data governance, security, privacy, or research data management.
Questions can include who will have access to identifiable information, how identifiers will be separated from analytical data, how records will be transferred, where data will be stored, how access will be controlled, and whether the proposed retention or sharing arrangements comply with applicable requirements.
These issues are particularly difficult to repair after sensitive data have already been collected incorrectly.
Formal scientific, sponsor, funder, or regulatory review may also apply
Some research environments impose additional review. A funder may conduct scientific peer review. A sponsor may require protocol approval. An institution may use a scientific review committee. Regulated studies can be subject to additional authorities and documentation requirements.
These processes should be identified during planning because they can affect what must be included in the protocol and when research may begin.
No single reviewer needs to know everything
A useful protocol review is often distributed across people with complementary expertise. The principal investigator understands the overall study. A domain expert may challenge the scientific rationale. A methodologist may examine design coherence. A statistician may scrutinize quantitative assumptions. An ethics committee may evaluate participant protection. Operational staff may test feasibility.
This does not mean every study needs a committee of specialists large enough to require its own conference catering budget. Review should be proportional to the complexity, risks, and methodological demands of the project.
| Reviewer or perspective |
Questions they may be especially suited to examine |
| Research team |
Internal consistency, completeness, roles, feasibility, and whether the written protocol matches the intended study |
| Subject-matter expert |
Scientific rationale, relevance, domain assumptions, population, intervention, outcomes, and interpretation |
| Methodologist |
Design, sampling, measurement, bias, methodological coherence, and inferential limits |
| Statistician or quantitative methodologist |
Sample size, quantitative design, data structure, statistical methods, missing data, and analytical feasibility |
| Qualitative methodologist |
Methodological fit, sampling logic, data generation, reflexivity, analysis, and appropriate flexibility |
| Operational or implementation personnel |
Recruitment, workflow, staffing, timing, equipment, data capture, and practical feasibility |
| Research ethics committee or institutional review board |
Ethical acceptability and participant protections under the applicable review framework |
| Data governance or security specialist |
Privacy, access, storage, transfer, retention, security, and data-management arrangements |
Review should happen while meaningful changes are still possible
A methodological review conducted after recruitment begins has already lost part of its preventive value. Review should occur early enough that problems in the design, measurements, procedures, or analysis can be corrected before they become embedded in the data.
This is one reason a protocol should reach an appropriate level of detail before specialist review. A reviewer cannot evaluate a sampling rule, outcome definition, or analysis that remains unspecified.
Watch Out
Do not wait until data collection is complete to ask a methodologist or statistician whether the design and analysis fit together. Some problems can be addressed analytically later, but missing measurements, unsuitable sampling, absent comparison groups, and fundamentally misaligned designs often cannot be repaired after the evidence has already been generated.
