What Counts as Valid Evidence in Research? Understanding the Different Types and Sources of Evidence

What Makes Something Good Enough to Count as Research Evidence?

Researchers work with remarkably different materials. One study may use laboratory measurements, another survey responses, another interviews, and another historical documents. A systematic review may not collect new participant data at all, yet it can still provide important evidence.

So what actually counts as valid evidence?

There is no single form of evidence that is inherently valid for every research question. The more useful question is whether the evidence is appropriate for the claim you want to evaluate, has been obtained and analyzed through defensible procedures, and supports the inference you intend to make.

Valid Evidence Must Fit the Question and the Claim

Evidence Is Valid for a Purpose, Not in the Abstract

Start With the Claim, Not the Data Type

Researchers sometimes ask whether interviews, surveys, experiments, observations, or secondary data are “valid evidence.” That question is incomplete because evidential value depends on what the researcher is trying to establish.

If you want to understand how teachers experience the introduction of a new curriculum, in-depth interviews may provide highly relevant evidence. If you want to estimate how common a particular attitude is across a national population, interviews with ten conveniently selected participants would not be sufficient for that population-level estimate. If you want to estimate a causal effect of an intervention, still other design requirements arise.

The method is therefore not valid or invalid simply because of its label. You must examine the relationship among the research question, the source of information, the design, the measurement or observation process, the analysis, and the intended conclusion.

Evidence Can Take Many Forms

The word evidence should not be restricted to numbers. Different research questions require researchers to observe different aspects of the world.

These categories can overlap, and disciplinary terminology varies. Their purpose is not to create a rigid taxonomy but to show why evidence cannot be reduced to a single format.

Quantitative Evidence Is Not Automatically More Valid

Numbers can create an impression of precision, but numerical evidence is only as useful as the processes that produced it.

A precisely reported estimate based on a badly measured variable can still be misleading. A large survey with a severely biased sampling process may provide an inaccurate picture of the population. A statistically sophisticated analysis cannot repair every weakness in the underlying data or design.

Quantitative evidence is particularly useful when a research question requires numerical estimation, comparison, modeling, or measurement. Its validity still depends on issues such as construct measurement, sampling, study design, assumptions, analytical choices, missing data, and uncertainty.

Qualitative Evidence Can Be Valid Evidence

Interviews, observations, documents, narratives, and other qualitative materials can provide evidence for questions concerning meaning, experience, interpretation, process, context, social interaction, and other phenomena for which numerical representation may be insufficient or inappropriate.

The standards used to evaluate such evidence are not necessarily identical to those used in an experiment. Researchers may need to consider the appropriateness of sampling, depth and adequacy of data, transparency of analytical procedures, reflexivity, consistency of interpretation, attention to contradictory cases, and the relationship between interpretations and the underlying material.

Calling evidence qualitative therefore does not make it inherently subjective or weak, just as calling evidence quantitative does not make it inherently objective or strong. The relevant question is whether the approach supports a defensible answer to the question being asked.

The Source of Evidence Matters

Researchers also need to consider where the information came from.

Evidence may be generated directly for a study or obtained from existing sources. Existing evidence can include datasets, documents, published research, archives, registries, repositories, institutional records, and other materials.

Using an existing source does not automatically weaken a study. In some cases, an established dataset may be substantially larger or more appropriate than anything an individual researcher could feasibly collect. At the same time, researchers must understand how the information was originally generated, what it represents, what is missing, and whether it is suitable for the new question.

Validity Depends on Whether the Evidence Represents What You Think It Represents

Suppose you want to study student learning but measure only satisfaction with a course. Satisfaction may be a legitimate phenomenon to investigate, but it is not interchangeable with learning.

Likewise, publication count is not identical to research quality, citation count is not identical to societal impact, and self-reported behavior may not perfectly correspond to observed behavior.

A central question is therefore whether the observations provide an adequate representation of the concept being studied. Measurement validity, broadly understood, concerns whether the interpretation and use of measurements are justified for their intended purpose.

Good Evidence Must Be Relevant to the Inference

A finding can be accurate and still be insufficient for a particular conclusion.

An observational association between two variables may be excellent evidence that the variables co-vary in the studied data. It may provide much weaker support for the stronger claim that one variable causes the other.

This is where the distinction between research and evidence becomes especially important. Conducting research does not automatically authorize every conclusion that could be attached to its findings.

Rigorous Procedures Increase Confidence in Evidence

NIH defines scientific rigor as the strict application of the scientific method to support robust and unbiased experimental design, methodology, analysis, interpretation, and reporting. Although specific standards differ across research traditions, the broader principle is useful: confidence depends partly on how systematically potential sources of error and bias have been addressed.

Depending on the study, relevant considerations may include sampling procedures, controls, randomization, blinding, measurement quality, data completeness, transparency of coding, analytical assumptions, documentation, sensitivity analyses, or other safeguards appropriate to the methodology.

No single checklist applies equally to every form of research. A historical analysis should not be judged as though it were a clinical trial, nor should an experiment be evaluated using only the standards appropriate to ethnographic interpretation.

Evidence Has Scope and Boundaries

Evidence obtained from one population, period, setting, measurement system, or context may not automatically support conclusions about another.

A study of first-year engineering students at one university may provide valid evidence about the participants and conditions studied while offering uncertain evidence about primary-school pupils, working adults, or university students in substantially different educational systems.

This does not make the original evidence invalid. It limits the scope of the inference.

Evidence Should Usually Be Considered as Part of a Larger Body

Even high-quality evidence from one study has limitations. Sampling variation, contextual differences, measurement error, analytical decisions, and unforeseen sources of bias can affect results.

This is why one study is rarely sufficient for a definitive answer. Confidence may become stronger when findings are examined alongside other relevant investigations, including studies using different methods or conducted under different conditions.

Repeated and independent inquiry can reveal whether an apparent finding is robust, context-dependent, smaller or larger than initially estimated, or difficult to reproduce. The National Academies notes that both successful and unsuccessful attempts to reproduce or replicate research can contribute information to scientific inquiry.

The “Right” Evidence Changes With the Question

The program is the same, but the evidence required changes because the research question changes. There is no single dataset that automatically answers every question about it.

Common Misunderstandings About Valid Research Evidence

Choose Evidence by Working Backward From the Claim

Instead of beginning with the data source that is easiest to obtain, begin with what you need to know. Then ask what observations would actually allow you to answer that question.

This approach can prevent a common research-design problem: collecting large amounts of information that do not adequately address the intended claim.

Ultimately, what makes a research claim stronger or weaker is not simply the amount of information behind it. The fit between evidence, methods, reasoning, and the claim matters just as much.

Before Treating Information as Evidence, Check:

Frequently Asked Questions About Valid Research Evidence

Evidence Is Valid When It Can Defensibly Support the Claim Being Made

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.