03 · What You Need to Know
Scientific Knowledge Includes Both What We Know and How Well We Know It
Uncertainty Is Not the Same as Ignorance
Imagine that researchers estimate an intervention improves an outcome by approximately 10%, but the available evidence is compatible with somewhat smaller or larger benefits.
They do not know the exact effect with certainty. Yet they may know considerably more than they did before the research was conducted.
This is why scientific knowledge is better represented by degrees of confidence than by a simple division between “known” and “unknown.” The National Academies describes uncertainty as inherent in scientific knowledge and emphasizes that science aims for refined degrees of confidence rather than complete certainty.
Research can therefore reduce uncertainty without eliminating it, which is central to understanding how research can advance knowledge by narrowing what remains unresolved.
Different Parts of a Claim Can Have Different Uncertainties
Researchers may be highly confident that a phenomenon exists while remaining less certain about its exact magnitude, mechanism, duration, or applicability.
For example, evidence might strongly support the conclusion that an intervention produces some benefit while leaving substantial uncertainty about whether the average improvement is 5% or 10%. Researchers might also remain uncertain about which participants benefit most.
It is therefore rarely useful to ask only, “Is this uncertain?”
A better question is, “What exactly is uncertain, and how much does that uncertainty matter for the conclusion?”
Sampling Creates Uncertainty
Many studies observe a sample rather than every member of the population researchers want to understand.
Another appropriate sample would contain different individuals or observations and would generally produce a somewhat different estimate. This sampling variability creates uncertainty about how closely the observed result represents the broader population or process.
Statistical methods can characterize some of this uncertainty. For example, confidence intervals can indicate the precision associated with an estimated effect under the assumptions of the analysis.
A narrow interval generally indicates greater statistical precision than a wide one. It does not, however, address every other source of uncertainty in the research.
Measurement Introduces Its Own Uncertainty
Researchers also need to consider how accurately observations represent the quantity or construct of interest.
NIST describes measurement uncertainty as characterizing the dispersion of values that could reasonably be attributed to what is being measured based on the information available. In measurement science, reporting a measured value without understanding its associated uncertainty can give an incomplete account of the result.
Research outside metrology encounters analogous issues. Test scores imperfectly represent learning. Survey responses may imperfectly capture attitudes. Sensors have limits. Coding decisions can involve ambiguity. Administrative records may contain incomplete or inaccurate entries.
A large sample cannot automatically compensate for a measurement that systematically represents the wrong thing.
Models Create Uncertainty Because They Simplify Reality
Researchers frequently use statistical, mathematical, computational, conceptual, or theoretical models to understand complex systems.
Models necessarily emphasize some features while simplifying others. Their conclusions may depend on assumptions about relationships, distributions, mechanisms, boundary conditions, or inputs.
A model can be extremely useful without being a literal replica of reality. Researchers therefore need to consider whether conclusions are robust to reasonable changes in assumptions and whether important aspects of the phenomenon have been omitted.
Inference Creates Uncertainty Beyond Direct Observation
Researchers often want to know more than what occurred in the collected data.
They may infer from a sample to a population, from an association to a possible mechanism, from observed evidence to an unobserved process, or from past observations to future outcomes.
Each inferential step introduces questions about whether the evidence genuinely supports the broader proposition.
This is why findings, evidence, and conclusions should be distinguished. Uncertainty associated with a finding does not vanish when researchers turn it into a conclusion, and additional inferential uncertainty may be introduced along the way.
Uncertainty Can Come From Bias, Not Only Random Variation
Researchers sometimes discuss uncertainty primarily in terms of statistical precision. That is too narrow.
A precisely estimated result can still be misleading if the study contains systematic bias. Selection processes, confounding, missing data, measurement problems, analytical decisions, selective reporting, or other factors can shift results away from the quantity researchers intend to estimate.
Cochrane's approach to certainty of evidence illustrates this broader perspective. Its assessments consider not only imprecision but also concerns such as risk of bias, inconsistency, indirectness, and publication bias.
Watch Out
A narrow confidence interval does not mean that every important uncertainty has been resolved. Precision addresses only part of the credibility of a research conclusion.
Uncertainty Can Arise Because Studies Disagree
When multiple studies address a similar question but produce different results, researchers need to determine what that variation means.
Some differences are expected from sampling variability. Others may reflect differences in populations, settings, measurement, implementation, methods, or genuine variation in the underlying phenomenon.
Unexplained inconsistency can reduce confidence in a simple universal conclusion.
At the same time, disagreement can reveal scientifically useful information. Understanding why well-conducted studies reach different conclusions can expose conditions under which a phenomenon changes.
Indirect Evidence Creates Uncertainty About Applicability
Evidence may be rigorous yet only indirectly relevant to the question you need to answer.
Suppose an intervention has been studied extensively among adults but you want to know whether it works among children. Or research measures an intermediate outcome while your question concerns a longer-term consequence.
The evidence may still be informative, but additional uncertainty arises when moving from what was directly studied to the population, intervention, comparison, outcome, or setting of interest.
Cochrane and GRADE refer to this problem as indirectness when assessing certainty in bodies of evidence.
Some Uncertainty Can Be Reduced Through More Research
Additional evidence can reduce uncertainty in several ways.
Larger appropriate samples may improve precision. Better measurements can represent phenomena more accurately. Stronger research designs can address alternative explanations. Replication can show whether findings recur with new data. Studies in new populations can clarify generalizability.
This is one reason replication and repeated evidence can strengthen what we know.
Research is particularly valuable when it identifies which uncertainty matters and generates evidence capable of reducing it.
Some Uncertainty May Persist Even With Extensive Research
Not every uncertainty disappears simply by collecting more data.
Natural systems vary. Human behavior changes. Future conditions are not perfectly predictable. Some constructs are difficult to measure directly. Models necessarily simplify. Contexts evolve.
Research may therefore improve knowledge by characterizing the range and sources of uncertainty rather than eliminating them.
Knowing that an outcome is highly variable under specified conditions can itself be important scientific knowledge.
Uncertainty About Evidence Is Different From Variability in the Phenomenon
Researchers should distinguish uncertainty caused by incomplete knowledge from variation that is genuinely part of the phenomenon being studied.
Uncertainty from limited knowledge
What researchers do not yet know because evidence, measurement, models, or understanding remain incomplete.
Variability in the phenomenon
Real differences among individuals, situations, events, environments, or outcomes that remain even when the system is studied carefully.
The distinction is useful because the response differs. Additional research may reduce uncertainty caused by insufficient information, while genuine variability may need to be described and modeled rather than expected to disappear.
Uncertainty Can Increase When Research Improves
More research does not always make scientists sound more certain.
A new study may reveal that an apparently simple phenomenon varies across contexts. Better measurement may uncover differences that older instruments concealed. A larger evidence base may expose disagreement that was invisible when only one study existed.
Scientific knowledge can therefore improve while acknowledged uncertainty increases.
This is not contradictory. Recognizing that an earlier claim was too confident can be an advance in knowledge.
Scientific Confidence Should Reflect the Full Evidence Base
Confidence in a conclusion depends on more than the uncertainty reported in one study.
Researchers may need to consider study quality, consistency across investigations, relevance to the question, precision, possible publication bias, replication, and whether different forms of evidence converge.
Frameworks such as GRADE formalize this process for particular kinds of evidence synthesis. Other disciplines use different standards and terminology.
The broader principle is transferable: confidence in a conclusion should reflect the strengths and uncertainties of the evidence supporting it, not merely the rhetorical confidence of the researcher.
Communicating Uncertainty Is Part of Scientific Accuracy
The National Academies identifies reporting uncertainty as a central feature of the scientific process and emphasizes that researchers should convey the appropriate degree of uncertainty accompanying their claims.
That does not mean adding “maybe” to every sentence.
Good uncertainty communication identifies what is well supported, what remains uncertain, how consequential the uncertainty is, and whether additional evidence could reasonably change the conclusion.
Too little qualification exaggerates what research establishes. Too much qualification can make strong evidence sound indistinguishable from speculation.
