03 · What You Need to Know
Knowledge Production Is a Chain of Reasoning, Not a Data-Collection Event
Research Usually Begins With Something We Do Not Yet Know Well Enough
Research begins with uncertainty. Perhaps an expected relationship has not been adequately tested, a phenomenon has not been described, competing explanations remain plausible, or existing studies leave an important question unresolved.
A researcher turns that uncertainty into a question that can be investigated. This matters because the question determines what kinds of observations would be informative and what claims the study might eventually support.
A vague curiosity such as “Does technology improve learning?” does not yet specify what must be observed. Which technology? What kind of learning? Compared with what? Among whom? Under what conditions? A researchable question narrows that uncertainty enough to make systematic investigation possible.
Researchers Connect Abstract Ideas to Observable Information
Many research questions concern concepts that cannot simply be observed in their entirety. Motivation, inequality, learning, organizational culture, trust, health, and social influence are concepts that researchers must somehow connect to observable information.
That connection may involve measurements, interviews, documents, experiments, field observations, administrative records, images, artifacts, simulations, or other forms of data appropriate to the question and discipline.
This is an important part of knowledge production because the resulting evidence is never entirely independent of how the study was designed. A survey item may capture one aspect of motivation. An examination score may represent one dimension of learning. An interview may reveal how a participant interprets an experience. Different approaches can illuminate different aspects of the same underlying phenomenon.
Data Are Not Yet the Conclusion
Once observations have been recorded, the researcher has data. Those data may be quantitative, qualitative, textual, visual, computational, or some combination of these forms.
But possessing data does not automatically answer the research question. Researchers must determine what patterns, relationships, differences, meanings, mechanisms, or other features can defensibly be inferred from them.
Data
Recorded observations or information generated, collected, or assembled for analysis.
Finding
A result that emerges from analyzing or interpreting those data.
Conclusion
An interpretation of what the findings imply in relation to the research question.
These distinctions help explain why researchers can work with the same broad phenomenon yet reach different levels of understanding. The path from observation to explanation involves analytical and inferential decisions rather than a mechanical conversion of data into truth.
Methods Make the Reasoning Systematic and Scrutinizable
Research methods provide procedures for connecting a question to evidence and evidence to a conclusion. Their purpose is not simply to make research look formal. They help researchers make those connections in ways that other people can inspect.
Consider a researcher asking whether a new teaching strategy improves student performance. Simply observing that students obtained high scores after using the strategy would leave many alternative explanations open. Perhaps those students were already high-performing. Perhaps the assessment was easier. Perhaps another change occurred at the same time.
A stronger design attempts to distinguish the explanation of interest from plausible alternatives. Depending on the question, this could involve comparison groups, repeated observations, randomization, statistical adjustment, carefully selected cases, triangulation, systematic coding, sensitivity analyses, or other methodological strategies.
The relevant standards vary considerably across research traditions. What counts as a defensible inference in an ethnographic study will not be identical to what is required in a randomized experiment. The underlying principle, however, is similar: the conclusion should follow from an explicit and defensible relationship among the question, evidence, method, and reasoning.
Analysis Turns Observations Into Findings
Analysis is the stage at which researchers examine the data for information relevant to the question. The form of analysis depends on what is being investigated.
A quantitative study might estimate an association, compare groups, quantify uncertainty, or evaluate how well a model fits observed data. A qualitative study might identify patterns of meaning, examine processes, develop interpretations across cases, or construct an explanation grounded in participants' accounts and contextual information. Computational and mixed-methods research may use still other analytical strategies.
The output of analysis is therefore not “knowledge” in some undifferentiated sense. It produces findings whose meaning depends on the data and the analytical procedures used to generate them.
Researchers Then Make an Inference
The critical intellectual step occurs when researchers ask: What does this finding allow us to conclude?
Suppose a study finds that students who use a particular learning platform obtain higher examination scores. That finding does not automatically establish that the platform caused the improvement. The conclusion depends on the research design, how participants were selected, what was measured, what alternative explanations were addressed, and how much uncertainty remains.
This is why what counts as valid evidence depends partly on the claim being made. Evidence adequate for describing an observed pattern may be inadequate for establishing a causal explanation.
Watch Out
A conclusion can go beyond the evidence even when the data themselves are accurate. Strong research requires researchers to distinguish what they observed from what they can reasonably infer from those observations.
Research Usually Changes What We Know by Changing Our Uncertainty
Research questions are rarely resolved by moving directly from “unknown” to “known with certainty.” More often, a study changes how plausible different explanations appear, narrows a range of possibilities, identifies conditions under which something occurs, reveals an unexpected pattern, or exposes weaknesses in an existing explanation.
Scientific results therefore carry uncertainty. The National Academies of Sciences, Engineering, and Medicine emphasizes the importance of estimating, characterizing, and reporting uncertainty, noting that scientific inquiry generally does not deliver absolute certainty. Instead, confidence in claims can increase or decrease as further evidence becomes available.
This is why understanding the role of uncertainty in scientific knowledge is central to understanding research itself. Uncertainty is not necessarily evidence that research has failed. Often, identifying the boundaries of what can and cannot currently be concluded is itself an important contribution to knowledge.
A Study Contributes to a Larger Body of Knowledge
Even a carefully conducted study remains one investigation conducted with particular data, methods, assumptions, participants, settings, or conditions. Its contribution must therefore be interpreted in relation to what was already known and what later investigations find.
This makes research fundamentally cumulative. Researchers compare new findings with previous work. Other investigators may test similar questions using new data, different populations, alternative measurements, or different analytical methods. Reviews and research syntheses can then examine patterns across multiple studies.
The National Academies describes scientific confidence as emerging through multiple lines of evidence and inquiry rather than through replication between only two individual studies. Research synthesis and meta-analysis, where appropriate, provide additional ways of evaluating a body of research.
Consequently, one research study is rarely enough to provide a definitive answer. A study may make an important contribution without settling the question permanently.
Repeated Investigation Can Strengthen, Qualify, or Challenge a Claim
When independent studies addressing a similar question obtain compatible results, confidence in the underlying claim may increase. When results differ, researchers have more work to do.
A difference does not automatically mean that one study was badly conducted. Studies can differ because of sampling variation, populations, contexts, measurement choices, implementation, analytical decisions, or genuine differences in how a phenomenon behaves under different conditions.
This is part of how research builds knowledge across multiple studies. Accumulation is not simply a matter of counting how many papers agree. Researchers consider the quality, relevance, consistency, independence, limitations, and collective implications of the available evidence.
Knowledge Remains Open to Revision
A research conclusion can be well supported without being permanently immune to revision. New evidence may reveal that an effect is smaller than originally estimated, applies only under certain conditions, has an alternative explanation, or does not generalize to populations that were never adequately studied.
Sometimes later research substantially overturns an earlier claim. In other cases, it refines rather than rejects it.
This capacity for revision is one reason scientific knowledge can change when new evidence appears. Knowledge claims remain answerable to evidence rather than becoming untouchable once they have been published.