03 · What You Need to Know
How Qualitative Evidence Can Clarify, Challenge, and Extend Quantitative Findings
What Does It Mean to Explain a Quantitative Finding Qualitatively?
Explaining a quantitative finding qualitatively means investigating the experiences, processes, meanings, or conditions that may help account for a numerical result.
For example, a survey might show that students with higher AI literacy scores report greater confidence in evaluating AI-generated information.
Qualitative interviews or observations could investigate how these students judge credibility, which verification practices they use, and whether their reported confidence corresponds with their actual behavior.
The qualitative findings may reveal that confidence reflects different practices among participants. Some may verify information against external sources, while others may rely on familiarity with the technology.
Such evidence can clarify what lies behind the numerical relationship without necessarily establishing that AI literacy causes confidence.
Researchers should therefore distinguish explaining the meaning of a statistical pattern from identifying its causal origin.
What Kinds of Quantitative Findings Benefit From Qualitative Follow-Up?
Qualitative follow-up is particularly useful when the quantitative results raise an important interpretive question that the existing measurements cannot adequately resolve.
| Quantitative finding |
Unresolved question |
Potential qualitative contribution |
| Unexpected group difference |
Why do participants in different settings report different outcomes? |
Investigates contextual conditions, practices, and experiences associated with the difference. |
| Unexpectedly weak intervention effect |
How did participants actually encounter and use the intervention? |
Examines delivery, participation, adaptation, and perceived barriers. |
| Contradictory survey responses |
How do participants interpret apparently inconsistent items? |
Clarifies response meanings and possible measurement limitations. |
| Unanticipated statistical association |
What practices or circumstances might account for the relationship? |
Develops or evaluates plausible explanations without assuming causation. |
| Variation in intervention response |
How do participants with contrasting outcomes experience the intervention? |
Investigates processes and conditions associated with different responses. |
| Unclear meaning of a numerical score |
What does the measured construct represent in participants' experiences? |
Examines how participants understand and enact the phenomenon being measured. |
These purposes are not interchangeable. Investigating measurement interpretation requires different evidence from reconstructing an intervention's implementation or evaluating a proposed causal mechanism.
The qualitative component should be designed around the particular uncertainty raised by the quantitative findings.
What Is an Explanatory Sequential Mixed-Methods Design?
An explanatory sequential mixed-methods design generally begins with quantitative data collection and analysis, followed by a qualitative phase intended to explain or elaborate selected quantitative results.
The sequence is commonly represented as quantitative investigation followed by qualitative investigation and integrated interpretation.
The defining feature is not simply that interviews occur after a survey. The qualitative phase must be informed by the quantitative results.
For example, a researcher might identify students with unexpectedly high or low engagement scores and select participants from those groups for interviews.
The interview questions would then investigate the experiences and practices relevant to the observed differences.
The final analysis would relate the qualitative findings back to the quantitative patterns.
This is one way of implementing a mixed-methods approach justified by a need for integration.
Must the Qualitative Phase Always Follow the Quantitative Phase?
No. Although explanatory sequential designs use that ordering, qualitative evidence can contribute to interpreting quantitative findings through other mixed-methods arrangements.
In a convergent design, researchers may collect both forms of evidence during a similar period and compare the findings during integration.
In an embedded design, qualitative inquiry may investigate implementation or participant experiences within a larger quantitative evaluation.
Qualitative observations may also be collected prospectively during an intervention and analyzed in relation to quantitative outcomes once those outcomes become available.
The appropriate sequencing depends on whether qualitative sampling and questions need to be determined by the quantitative results or whether both components can be planned together.
When the purpose is specifically to investigate unexpected statistical findings, a sequential design may provide a particularly clear connection between phases.
How Should You Select Participants for Qualitative Follow-Up?
Participant selection should follow the explanatory purpose rather than convenience alone.
Researchers may select participants representing contrasting quantitative patterns, such as high and low scores, different trajectories, or unexpected combinations of characteristics.
For example, a researcher might interview students who report high confidence but demonstrate low performance on an AI literacy assessment, alongside students whose confidence and performance are more closely aligned.
This selection could help investigate how participants interpret confidence and how their evaluation practices differ.
Other strategies include maximum variation sampling, criterion-based selection, and purposive selection of cases that are especially informative for the proposed explanation.
However, selecting only extreme cases can create interpretive difficulties. Extreme observed scores may partly reflect measurement error or random variation, and participants selected because of an outcome may not represent typical experiences.
Researchers should explain why the selected cases are informative and avoid generalizing their qualitative accounts beyond the evidence.
Should You Interview Only Participants With Unexpected Results?
Not necessarily.
Unexpected cases can reveal important limitations in existing explanations, but contrasting or typical cases may be equally useful.
Suppose a quantitative study finds that some students with low AI literacy scores nevertheless demonstrate strong academic performance.
Interviewing only those students may reveal interesting strategies, but the researcher may have difficulty determining whether those strategies are distinctive.
Including students with other combinations of literacy and performance may provide a stronger comparative basis.
The appropriate selection depends on whether the qualitative phase aims to understand exceptional cases, compare processes, examine measurement meaning, or develop an explanation of a broader pattern.
Case selection should be justified as part of the analytical strategy, not merely described as purposive sampling.
What Should the Qualitative Questions Actually Ask?
Qualitative questions should investigate the phenomenon rather than invite participants to confirm the researcher's preferred explanation.
Suppose a survey finds that students receiving frequent AI feedback report greater writing confidence.
A leading question would be: "How did AI feedback improve your confidence?"
This wording assumes the very relationship the researcher is trying to understand.
A more open question might ask students to describe their experiences with feedback and explain what influenced their confidence when revising academic work.
Researchers could then examine whether AI feedback, instructor guidance, prior writing experiences, or other circumstances appear relevant.
Where possible, questions should be connected to specific events, activities, or examples rather than relying entirely on generalized retrospective explanations.
The objective is to generate evidence capable of informing the interpretation, including evidence that challenges the initial hypothesis.
Can Interviews Establish Why a Statistical Relationship Exists?
Interviews can contribute important explanatory evidence, but they do not automatically identify the causes of a statistical relationship.
Participants may describe why they believe they behaved in particular ways or what circumstances influenced their decisions.
These accounts are valuable evidence about interpretations and reported experiences. However, participants may not be aware of every influence on their behavior, and retrospective accounts may be affected by memory or social expectations.
For example, students may attribute improved performance to an AI tutoring platform because they enjoyed using it. Their attribution does not independently establish that the platform caused the improvement.
Researchers may strengthen the explanation by combining interviews with observations, documents, activity records, or other evidence that bears directly on the proposed process.
When the intended claim concerns causation, the qualitative component should be evaluated according to its ability to address alternative explanations and relevant causal assumptions.
How Can Qualitative Research Investigate Mechanisms Behind Quantitative Results?
Mechanism-focused follow-up investigates the processes through which an intervention or condition may produce an outcome.
Suppose a randomized evaluation establishes that an AI-supported feedback system improves students' writing performance.
Qualitative observations might investigate whether students interpret feedback, reconsider their arguments, and revise their drafts in ways consistent with a proposed learning mechanism.
Successive writing drafts, feedback records, and observed revision sessions may provide evidence about the process.
However, observing a proposed process among successful participants does not necessarily establish that it caused the improvement.
Researchers should consider whether the process also occurred among participants who did not improve and whether other factors could account for the outcome.
Qualitative inquiry can contribute to the investigation of causal mechanisms, but the strength of the causal claim depends on the complete inferential design.
Can Qualitative Follow-Up Reveal Problems With Quantitative Measurement?
Yes. Qualitative evidence may reveal that respondents interpret survey questions or numerical scales differently from what the researcher intended.
For example, students may report high confidence in using AI because they can generate answers quickly, while the assessment is intended to measure confidence in critically evaluating AI-generated information.
Interviews or task-based observations may reveal that the same survey response reflects different understandings of the construct.
Such findings can identify possible response-process or construct-validity problems.
However, a few interviews do not automatically establish the extent of measurement bias across the full sample.
Researchers may need additional instrument development, cognitive interviewing, or quantitative validation to evaluate the problem systematically.
Qualitative findings can therefore challenge the interpretation of a quantitative result rather than merely explain it.
How Should Quantitative and Qualitative Findings Be Integrated?
Integration should occur through an explicit analytical process that relates the two components.
One approach is to organize qualitative findings around the quantitative patterns that motivated follow-up.
Another is to construct a joint display showing quantitative results alongside qualitative observations and the resulting integrated interpretation.
For example, a joint display could compare students with high and low engagement scores, summarize relevant qualitative practices, and identify which interpretations are supported or remain uncertain.
Integration may also involve comparing explanations across cases, examining contradictions, or revising the conceptual interpretation of quantitative measures.
The goal is to develop a meta-inference, meaning an interpretation informed by the relationship between the quantitative and qualitative findings.
Such an inference should not exceed what the components can establish individually and together.
What Should You Do When the Qualitative Findings Contradict the Quantitative Results?
Contradiction can be analytically valuable.
Suppose survey findings indicate that students are highly satisfied with an AI-supported advising system, while interviews reveal considerable uncertainty about the reliability of its recommendations.
These findings may not be genuinely contradictory. Students might appreciate convenience while distrusting some recommendations.
Alternatively, the survey and interviews may involve different participants, time periods, or interpretations of satisfaction.
Researchers should investigate whether divergence reflects measurement differences, contextual variation, sampling, or genuinely conflicting evidence.
Forcing agreement can obscure important findings.
A credible integrated interpretation may preserve uncertainty or identify a more complex relationship than either component initially suggested.
Does Qualitative Follow-Up Improve the Validity of Quantitative Results?
It can contribute to evaluating the interpretation of quantitative findings, but it does not automatically correct weaknesses in the quantitative design.
For example, interviews may reveal how participants understood survey items, potentially informing construct validity.
Observations may help establish whether an intervention was implemented as intended.
However, qualitative follow-up does not automatically correct nonrepresentative sampling, unmeasured confounding, or inappropriate statistical analysis.
A study that lacks a credible counterfactual cannot simply acquire causal validity by adding participant explanations.
The qualitative component should be valued for the evidence it provides rather than treated as a universal remedy for quantitative limitations.
How Do You Avoid Constructing Explanations After Seeing the Results?
Explanatory sequential research necessarily uses quantitative findings to inform later qualitative investigation. This is a legitimate design feature, not inherently a methodological problem.
The risk arises when researchers select only evidence that supports a preferred interpretation or present post hoc explanations as though they had been independently confirmed.
Researchers should document why particular findings were selected for follow-up, how participants were sampled, and how interview or observation procedures were developed.
They should actively examine alternative explanations and cases that challenge emerging interpretations.
Where explanations arise after quantitative analysis, their exploratory or confirmatory status should be communicated accurately.
A plausible explanation generated through follow-up may provide a strong basis for future research without being established as a causal fact.
When Is Qualitative Follow-Up Unnecessary?
Qualitative follow-up is not necessary simply because quantitative analysis has been completed.
If the research question requires only a credible estimate of prevalence, association, prediction, or causal effect, the quantitative component may provide a sufficient answer.
For example, a well-designed randomized study may estimate the effect of an instructional intervention on examination performance without needing interviews to establish the primary causal contrast.
Qualitative inquiry becomes useful when the intended research contribution also requires understanding experiences, implementation, mechanisms, or contextual conditions.
Researchers should not add interviews merely to make the findings appear more comprehensive.
When one approach already provides the necessary evidence, a focused study may be more defensible than an underdeveloped mixed-methods design.
What Are the Main Limitations of Qualitative Explanations of Quantitative Findings?
Several limitations deserve particular attention.
Participants selected for follow-up may not represent the broader quantitative sample. Retrospective accounts may be incomplete or influenced by participants' awareness of their outcomes.
Qualitative evidence may identify plausible processes without establishing their causal contribution. Differences in the timing of data collection may also complicate interpretation.
Integration can become misleading when researchers combine findings that concern different constructs or units of analysis without acknowledging those differences.
These limitations do not invalidate qualitative follow-up. They define the conditions under which its explanations should be interpreted.
The appropriate goal is an integrated account whose claims remain proportionate to the evidence.
Watch Out
Do not select a few favorable interview quotations after obtaining significant quantitative results and present them as proof of why the relationship occurred. Qualitative explanation requires systematic sampling, appropriate analysis, integration, and consideration of alternative interpretations.
