03 · What You Need to Know
How to Evaluate Reviewer Requests for a More Comprehensive Research Design
Why Might Reviewers Recommend Mixed Methods?
Reviewers may recommend mixed methods for several legitimate reasons. They may believe that the research question requires more than one form of evidence, that the current findings are difficult to interpret, or that an important aspect of the phenomenon has not been investigated.
For example, a quantitative evaluation may establish an intervention effect but provide little information about implementation. A qualitative follow-up could help investigate whether participants received the intended intervention and how they experienced its activities.
In another study, interviews may identify concepts that require subsequent numerical measurement before the researcher can make population-level claims.
Such recommendations can improve methodological alignment when they address evidence genuinely required by the intended conclusion.
Other requests may reflect disciplinary conventions, journal expectations, or a reviewer's preference for a particular approach. These considerations can influence publication decisions, but they do not independently establish methodological necessity.
The researcher should therefore examine the reasoning behind the recommendation rather than accepting or rejecting it simply because it comes from a reviewer.
What Does a More Comprehensive Study Actually Mean?
Comprehensiveness can refer to several different characteristics of research.
A study may be comprehensive in its coverage of a population, its examination of relevant variables, its attention to competing explanations, or its investigation of multiple dimensions of a phenomenon.
Methodological variety is only one possible contributor.
For example, a carefully designed longitudinal quantitative study may provide extensive evidence about changes in student achievement across several semesters. Adding a small number of superficial interviews would not necessarily make its answer more convincing.
Similarly, an intensive qualitative investigation of how teachers negotiate professional identity during AI adoption may provide a rich interpretive account without requiring a survey.
Comprehensiveness should be assessed relative to the question, not treated as an obligation to investigate every adjacent issue.
A study can be appropriately bounded without being incomplete.
How Do You Distinguish a Genuine Evidence Gap From a Preference for More Methods?
The central test is whether the requested method would address something essential that the current evidence cannot establish.
| Reviewer recommendation |
Potential methodological value |
Question to ask |
| Add interviews to explain unexpected group differences. |
May clarify experiences, practices, or contextual conditions associated with the results. |
Is explaining the difference necessary to the intended conclusion? |
| Add a survey to a qualitative study. |
May estimate the distribution of concepts identified qualitatively. |
Does the study actually make population-level claims? |
| Add interviews to demonstrate intervention effectiveness. |
May illuminate experiences or implementation but ordinarily cannot replace causal-effect evidence. |
What inferential limitation would interviews resolve? |
| Add qualitative evidence to examine implementation. |
May document adaptations, delivery processes, and barriers. |
Does interpreting the outcome require evidence about actual delivery? |
| Add mixed methods because the study is too simple. |
Unclear without a specified evidentiary gap. |
Which research claim is currently unsupported? |
A reviewer request becomes more persuasive when it identifies a specific inference that the original study cannot defend.
For example, if a manuscript claims to explain why an intervention worked but measures only outcomes, the reviewer may reasonably question the absence of mechanism evidence.
The researcher could respond by collecting suitable explanatory evidence or narrowing the claim to effectiveness.
Neither response automatically requires mixed methods. The appropriate remedy depends on the intended contribution.
Does a Quantitative Study Need Interviews to Be Considered Rigorous?
No. Quantitative rigor depends on the adequacy of measurement, sampling, design, analysis, and assumptions relevant to the question.
A randomized experiment may estimate an intervention's causal effect without collecting qualitative interviews. A probability-based survey may estimate population prevalence without investigating every respondent's personal experience.
Interviews could address additional questions, but their absence does not necessarily invalidate the original inference.
Conversely, adding interviews does not automatically correct confounding, biased sampling, missing outcomes, or an invalid measurement instrument.
The researcher should identify what the quantitative approach can establish and whether the reviewer has identified a limitation relevant to that purpose.
Does a Qualitative Study Need a Survey to Strengthen Its Findings?
Not necessarily. Qualitative studies can support interpretive, conceptual, and process-oriented contributions through appropriate evidence and analysis.
For example, a study investigating how doctoral students experience research-related uncertainty may not need to estimate the prevalence of each identified experience.
A reviewer might reasonably request greater transparency about participant selection, analytical procedures, contradictory accounts, or the relationship between evidence and interpretation.
These concerns may be addressed by strengthening the qualitative study rather than adding a quantitative component.
However, if the manuscript claims that an experience is widespread across a population, a reviewer may reasonably request numerical evidence supporting that claim.
The researcher could obtain appropriate quantitative evidence or remove the unsupported prevalence claim.
Understanding the inferential contribution of qualitative research helps distinguish methodological limitations from inappropriate expectations.
Can Reviewer Requests Reveal That the Research Question Is Too Broad?
Yes. Sometimes a reviewer recommends additional methods because the manuscript promises more than its design can establish.
Consider a study titled "The Effectiveness and Implementation of AI-Supported Assessment in Higher Education."
If the study contains only student achievement outcomes, the implementation component of the title may be insufficiently supported.
A reviewer requesting implementation interviews or observations may be identifying a genuine mismatch between the study's claims and its evidence.
However, the researcher could also narrow the title and objectives to the effectiveness question actually investigated.
This distinction matters because adding methods is not the only way to resolve a mismatch. Sometimes the more defensible solution is to reduce the scope of the claim.
When Would Adding Mixed Methods Improve the Study?
Mixed methods may improve the study when the additional component addresses a clearly identified uncertainty that matters to the research question.
Suppose a randomized intervention evaluation finds a small average effect but substantial variation in implementation across classrooms.
Qualitative observations or interviews could help investigate how instructors delivered the intervention, why adaptations occurred, and how students experienced the activities.
The quantitative component would estimate the effect of assignment, while the qualitative component would examine implementation processes.
If these findings are meaningfully integrated, the study may provide a more informative evaluation.
However, the qualitative component must be adequately designed. Interviewing a few conveniently available students solely to obtain positive quotations would not establish a credible explanation.
The decision should follow the actual evidentiary rationale for mixed methods.
When Could Adding Mixed Methods Make the Study Weaker?
An additional component can weaken a study when it introduces claims that cannot be adequately supported.
For example, a quantitative researcher may add three short interviews after receiving reviewer feedback and describe the resulting comments as explaining the causal mechanism behind an intervention effect.
The interviews may not provide sufficient evidence about the proposed mechanism, and the participants may have been selected without a clear explanatory rationale.
The manuscript now contains an additional inferential claim that is weaker than its original effectiveness estimate.
Other problems may arise when the two components involve incompatible populations, different observation periods, or constructs that cannot be meaningfully related.
Mixed-methods integration also requires analytical work. Presenting two sets of findings separately without explaining their relationship may add length without improving the answer.
The additional method should be evaluated according to its contribution and adequacy, not merely its presence.
How Should You Evaluate the Feasibility of a Reviewer Request?
Reviewer recommendations sometimes involve new data collection, recruitment, ethical approval, or access to participants who may no longer be available.
These constraints matter because an inadequately conducted supplementary component may introduce more uncertainty than it resolves.
Before agreeing to a substantial revision, determine whether the proposed method can be conducted with appropriate sampling, instruments, data collection procedures, analysis, and integration.
If the necessary evidence cannot be obtained, explain the limitation and consider whether the manuscript's claims can be narrowed.
Researchers should also avoid implying that interviews or observations were part of the original design when they were introduced retrospectively in response to peer review.
Transparency about when and why a component was added is important for interpreting the findings.
What If the Journal Strongly Prefers Mixed-Methods Research?
Journal scope and editorial expectations are legitimate considerations when selecting a publication venue.
Some journals prioritize integrated methodological contributions, while others welcome focused quantitative or qualitative studies. Such preferences should be established from the journal's current aims, scope, and author guidance rather than assumed from isolated reviewer comments.
If a journal's expectations do not align with the study's legitimate methodological purpose, submitting to a more suitable venue may be preferable to substantially redesigning the research for presentation alone.
However, a manuscript submitted to a journal explicitly devoted to mixed-methods methodology may reasonably be expected to demonstrate substantive integration.
Methodological appropriateness and publication fit are related considerations, but neither should be confused with the other.
How Can You Respond When a Reviewer Requests an Unnecessary Method?
A defensible response should acknowledge the recommendation, identify the intended inference, and explain why the existing design is adequate for that purpose.
For example, a researcher might explain that the study was designed to estimate the effect of assignment to an intervention on a prespecified outcome, rather than investigate participants' experiences or implementation mechanisms.
The response could clarify that qualitative interviews would address a valuable but distinct research question and that the manuscript has been revised to avoid implying that those mechanisms were established.
This is stronger than simply stating that interviews are unnecessary or that the reviewer misunderstands the methodology.
The researcher should also consider whether the reviewer's concern can be addressed through clearer reporting, stronger limitations, additional analyses of existing data, or more precise wording.
Respectful methodological disagreement is appropriate when it is supported by a coherent explanation.
When Should You Accept the Recommendation?
Accepting a reviewer recommendation is appropriate when it identifies an important gap that can be addressed through a feasible and rigorous additional component.
For example, a manuscript may claim to explain why students with similar AI literacy scores exhibit different evaluation practices, yet contain only numerical scores and no evidence about actual practices.
Qualitative observation or task-based interviews may be necessary to support that interpretive claim.
Alternatively, the researcher could narrow the manuscript to the statistical relationship if the explanatory investigation is beyond the study's scope.
The choice should be made according to the contribution the researcher intends to preserve.
What Role Should Reporting Standards Play?
Reporting standards can help researchers communicate methodological decisions clearly, but they should not be used to require inappropriate designs.
The Good Reporting of A Mixed Methods Study (GRAMMS) framework emphasizes explaining the justification for mixed methods, the design, the contribution of each component, and how integration occurred.
These expectations are relevant when a study genuinely uses mixed methods.
They do not mean that every quantitative or qualitative manuscript must adopt mixed methods to satisfy a reporting standard.
Similarly, reporting guidance for randomized trials or qualitative studies should be applied according to the actual research design.
Watch Out
Do not treat a reviewer's request for comprehensiveness as automatic evidence that the research design is inadequate. Equally, do not dismiss the recommendation without examining whether the manuscript makes claims that its existing evidence cannot support.
