03 · What You Need to Know
How Numerical and Experiential Evidence Answer Different Research Questions
What Counts as Numerical Evidence?
Numerical evidence consists of measurements, counts, scores, estimates, or other quantitative representations that are relevant to a research claim.
Examples in educational research include examination scores, attendance rates, completion times, frequency of learning-platform use, survey-scale responses, and the number of students meeting a specified criterion.
Numerical evidence is especially useful when a question asks:
- How many people or cases have a particular characteristic?
- How frequently does an event occur?
- How much do groups or individuals differ?
- How strongly are measured variables associated?
- How accurately can an outcome be predicted?
- What is the estimated magnitude of an intervention's effect?
However, numerical representation is not equivalent to measurement validity. A score is useful only to the extent that its interpretation is supported by appropriate evidence.
For example, counting the number of times a student opens a learning platform provides a numerical measure of recorded activity. It does not automatically measure cognitive engagement, motivation, or learning quality.
Researchers must establish how the numerical indicator relates to the construct they intend to investigate.
What Counts as Experiential Evidence?
Experiential evidence concerns how people encounter, understand, interpret, respond to, or make meaning from events and circumstances.
It may be obtained through interviews, participant observations, diaries, reflective accounts, open-ended responses, or other materials that provide access to experiences and their interpretation.
Such evidence is particularly relevant when the question asks:
- How do participants experience a particular practice or intervention?
- What meanings do they attach to an event?
- How do they interpret institutional expectations?
- What tensions or dilemmas shape their decisions?
- How do their experiences develop within particular contexts?
Consider a teacher who reports that an AI assessment tool is useful but refuses to incorporate it into classroom practice. A numerical usefulness score records the evaluation, but it may not reveal the teacher's concerns about accountability, professional judgment, or the reliability of automated recommendations.
Experiential evidence can illuminate these considerations, although participants' accounts should not automatically be treated as complete or independently verified explanations of behavior.
People may misremember events, interpret experiences retrospectively, or provide accounts shaped by social expectations. Qualitative researchers therefore examine how accounts are produced and interpreted, rather than assuming that every statement transparently represents an underlying reality.
Experiential Evidence Is Not the Same as All Qualitative Evidence
One important distinction is frequently overlooked: qualitative research encompasses more than the investigation of lived experience.
Qualitative researchers may examine institutional documents, organizational routines, classroom interactions, policy discourse, cultural practices, or historical processes without making individual experience their principal object of inquiry.
An ethnographic study of classroom technology practices might investigate observable routines and social relationships. A discourse analysis might examine how university policies construct acceptable AI use. Neither necessarily depends on asking participants to describe their personal experiences.
Consequently, the choice between numerical and experiential evidence is narrower than the broader choice between quantitative and qualitative research approaches.
When deciding whether individual experiences are central to your question, first determine whether experience itself is the phenomenon of interest or merely one possible source of information about another phenomenon.
What Can Numerical Evidence Establish That Experiential Evidence Usually Cannot?
Numerical evidence is generally better suited to estimating quantities and their uncertainty when the measurement and sampling procedures are appropriate.
Suppose a university wants to estimate the percentage of students who have used generative AI for coursework during the current semester.
A suitably designed survey could produce an estimate with an accompanying measure of sampling uncertainty, subject to assumptions and potential nonsampling errors. Interviews with a small purposively selected group might reveal different forms of AI use, but they would not ordinarily justify a population prevalence estimate.
Numerical evidence can also support comparisons across groups or periods, formal analyses of associations, and evaluations of predictive performance.
These capabilities depend on the design. Numerical data collected through convenience sampling, poorly validated measures, or incomplete records may not support the intended population-level inference.
Nor does numerical evidence automatically establish causation. A numerical association between AI use and academic achievement can remain vulnerable to confounding and reverse causation.
What Can Experiential Evidence Reveal That Numerical Evidence May Miss?
Experiential evidence can provide access to distinctions that predefined measures may not capture.
Suppose students assign an AI feedback system an average usefulness score of four out of five. The score summarizes their evaluations, but it does not establish what usefulness means to them.
Some students may value immediate responses. Others may appreciate the opportunity to revise work privately. A few may regard the feedback as useful precisely because they can challenge it against other sources.
Qualitative inquiry can investigate these interpretations and examine how they relate to participants' circumstances.
It can also reveal unanticipated concerns, contradictions, and processes that a fixed-response instrument may overlook. These findings may inform theory development, measurement refinement, or subsequent quantitative research.
Nevertheless, experiential accounts do not automatically establish how common a particular experience is within a wider population. Statements such as "most students felt..." require an appropriate evidential basis beyond the existence of several similar interview responses.
Can the Same Phenomenon Be Studied Using Either Form of Evidence?
Yes. Many phenomena can be investigated through different evidential approaches, provided the research questions are formulated accordingly.
Take student engagement in online learning.
A quantitative study might estimate changes in engagement scores over a semester. A qualitative study might investigate how students experience participation, isolation, and interaction in virtual classrooms.
Both may contribute to understanding engagement, but their findings operate at different levels.
The quantitative study might establish a pattern in measured engagement. The qualitative study might illuminate how students understand and experience engagement in context.
Neither should be described as inherently more complete without considering the research purpose.
When Does a Research Question Genuinely Require Both?
Both forms of evidence may be necessary when the question contains complementary evidential demands that cannot be adequately addressed by one form alone.
For example, a researcher may want to establish whether students' engagement changes after introducing an AI-supported learning system and understand how students interpret the system's role in their learning.
The first component requires evidence about measured engagement across relevant conditions or periods. The second requires access to students' experiences and interpretations.
A mixed-methods design may be appropriate if the researcher intends to integrate the findings to address a coherent overarching question.
Integration can occur at different stages, including sampling, data collection, analysis, and interpretation. For instance, quantitative results may inform the selection of interview participants, or qualitative findings may help explain unexpected patterns in numerical results.
However, using both forms of data is not sufficient to establish that a study is meaningfully mixed methods. The researcher must explain what is gained through integration and how the resulting conclusions are supported.
| Research purpose |
Potentially appropriate evidence |
Important limitation |
| Estimate how many students use AI tools |
Numerical evidence from a suitable survey or usage records |
Representativeness and measurement accuracy must be evaluated. |
| Understand students' experiences of AI feedback |
Experiential accounts and relevant observations |
Findings do not automatically estimate population prevalence. |
| Compare achievement across instructional conditions |
Comparable numerical outcome measurements |
Observed differences do not automatically establish causality. |
| Understand how teachers negotiate AI-related professional dilemmas |
Interviews, observations, or other qualitative materials |
Reported reasons may not independently establish causal determinants. |
| Examine achievement patterns and students' interpretations of feedback |
Numerical and experiential evidence, with justified integration |
Both components must contribute to the research inference. |
Why Adding Interviews to a Survey Does Not Automatically Improve a Study
Researchers sometimes assume that adding interviews makes quantitative findings more credible. This may be true when the interviews address a genuine evidential gap, but it is not a general rule.
If a study asks only for the prevalence of a clearly defined behavior, well-designed numerical evidence may be sufficient. Interviews might contribute contextual insight, but they do not necessarily improve the accuracy of the prevalence estimate.
Likewise, adding a small questionnaire to an interpretive qualitative study may contribute little if its items are poorly aligned with the phenomenon being investigated.
The relevant question is whether the additional component changes what can reasonably be concluded.
A second method introduces additional sampling, analytical, integration, and reporting demands. When these cannot be met adequately, a focused single-method study may be more defensible.
Do Numerical and Experiential Evidence Have Different Standards of Quality?
Yes, although both require methodological transparency and a credible relationship between evidence and inference.
For numerical evidence, researchers commonly examine measurement validity, reliability or precision, sampling procedures, missing data, statistical assumptions, bias, and uncertainty.
For experiential evidence, relevant considerations may include the appropriateness of participant or case selection, the richness of the material, reflexivity, analytic coherence, contextual grounding, and the transparency of interpretation.
These considerations are not interchangeable checklists. Different qualitative traditions may adopt different epistemological assumptions and standards of quality.
For example, a phenomenological investigation may prioritize careful interpretation of lived experience, while a constructivist grounded theory study may emphasize the development of an explanatory conceptual account through iterative data collection and analysis.
In mixed-methods research, the credibility of the integrated inference also matters. Strong quantitative and qualitative components do not automatically produce a coherent conclusion if their relationship is poorly specified.
What Happens When Numerical and Experiential Findings Disagree?
Disagreement should not automatically be interpreted as evidence that one source is incorrect.
Suppose survey respondents report high satisfaction with an AI tutoring system, while interviews reveal persistent distrust of its recommendations.
Several interpretations are possible. Students may value the convenience of the system while doubting its accuracy. The survey may measure general satisfaction, whereas the interviews examine trust. Alternatively, sampling differences or limitations in the instruments may contribute to the apparent discrepancy.
The researcher should examine whether the sources address the same constructs, populations, and time periods before attempting to reconcile them.
Mixed-methods integration can make such tensions analytically productive, but it should not force contradictory findings into an artificially consistent narrative.
How Should Feasibility Influence the Choice?
Feasibility matters, but it should not silently redefine the evidential requirements of the question.
If population-level estimation is essential, interviewing a handful of accessible participants is not an adequate substitute simply because a survey is difficult to administer.
Similarly, if the research concerns the meanings participants attach to a sensitive experience, collecting only numerical ratings may leave the central question unanswered.
When the necessary evidence cannot realistically be obtained, researchers should consider alternative sources, a narrower research question, or a revised inferential objective.
This is consistent with the broader principle that evidence must support the intended answer, not merely be convenient to collect.
