01 · The Question
If You State a Hypothesis, Must You Plan How to Test It?
A hypothesis makes a specific claim that the study intends to evaluate. It therefore seems straightforward that every hypothesis should have a statistical test attached to it.
The underlying principle is sound, but “one hypothesis, one test” can become misleading. A hypothesis may concern a model parameter, contrast, interaction, trend, association, difference, or another estimable quantity. Several hypotheses can sometimes be evaluated within one model, while one substantive hypothesis may require more than one analytical step.
The important question is not whether the number of statistical tests equals the number of hypotheses. It is whether each hypothesis has a clearly identified analysis capable of evaluating the claim it actually makes.
03 · What You Need to Know
How a Hypothesis Should Map to Its Analysis
Start With the Claim the Hypothesis Actually Makes
A hypothesis is more specific than a general topic. “Student engagement and academic performance” is a topic. “Student engagement is positively associated with academic performance” makes a directional claim about a relationship. “Students receiving the intervention will improve more than students receiving the comparison condition” makes a claim about a difference in change.
The analysis needs to evaluate that specific claim.
Before choosing a statistical procedure, translate the hypothesis into the quantity that would provide evidence relevant to it. Is the hypothesis about a difference between means, a regression coefficient, an odds ratio, a correlation, a change over time, an interaction, a trend, or another parameter or contrast?
This is the same alignment principle that connects analysis to the research question and study design, but hypotheses usually make the intended comparison or relationship more explicit.
A Substantive Hypothesis and a Statistical Hypothesis Are Related but Not Identical
Researchers often use the word “hypothesis” for two related ideas.
Substantive research hypothesis
A claim about the phenomenon being studied, such as an intervention improving an outcome or two variables being associated.
Statistical hypothesis
A formal statement about a population parameter, probability distribution, or other quantity that can be evaluated using a statistical procedure.
The analysis plan has to connect the two. A substantive claim about an intervention should be represented by a statistical quantity that actually corresponds to the relevant treatment comparison. A claim about effect modification should map to an interaction or other direct comparison of effects, not merely to separate tests performed within subgroups.
This translation matters because a statistical test can be perfectly valid for its statistical hypothesis while still failing to evaluate the substantive hypothesis the researcher intended.
Directional Hypotheses Require More Than Directional Wording
A hypothesis may predict not merely that two quantities differ but that one is greater, lower, positively related, or negatively related to another. That direction should be established on substantive grounds before the relevant results are known.
Whether a one-sided statistical test is justified is a separate decision. A directional research hypothesis does not automatically require one-sided testing. The consequences of an effect in the unexpected direction, disciplinary conventions, study purpose, and statistical framework all matter.
Choosing a one-sided test after observing that the effect points in the predicted direction would not constitute genuine advance specification.
The Unit of Analysis Must Match the Hypothesis
Consider a hypothesis stating that classrooms using a particular teaching strategy will have better student outcomes than classrooms using another strategy. If the intervention is implemented at the classroom level but the analysis treats every student observation as independent, the statistical analysis may fail to reflect the design that generated the data.
Similarly, a hypothesis about within-person change requires an analysis that recognizes repeated observations from the same individuals. A hypothesis about differences among organizations cannot automatically be evaluated by treating measurements within those organizations as unrelated observations.
The hypothesis, design, and unit of analysis should therefore be checked together before the statistical procedure is chosen.
One Hypothesis Does Not Always Equal One Statistical Test
A simple study may genuinely have a neat one-to-one structure. Hypothesis 1 might correspond to one predefined comparison, Hypothesis 2 to another, and so forth. There is nothing wrong with that structure when the science supports it.
It should not, however, be treated as a universal requirement.
A single regression model can contain parameters or planned contrasts corresponding to several hypotheses. Conversely, one substantive hypothesis may require a primary estimate plus supporting or sensitivity analyses to evaluate how robust the conclusion is to reasonable analytical assumptions.
What matters is that the mapping can be followed. The reader should be able to identify which output actually evaluates each hypothesis.
Interaction Hypotheses Need a Direct Test of the Difference in Effects
Interaction hypotheses provide a particularly useful illustration of why the mapping matters.
Suppose the hypothesis is that an intervention is more effective for novice researchers than for experienced researchers. A common mistake is to test the intervention separately in each subgroup, find a statistically significant effect among novices and a non-significant effect among experienced researchers, and conclude that the intervention effect differs between them.
That conclusion does not follow merely from the difference between “significant” and “not significant.” The hypothesis concerns whether the effects themselves differ. The analysis therefore needs to evaluate that difference directly through an appropriate interaction, contrast, or equivalent method consistent with the design.
Null Hypotheses Should Correspond to the Quantity Being Evaluated
In conventional null-hypothesis significance testing, the null hypothesis is expressed in terms of a statistical quantity. For a simple comparison, it might state that a population difference equals zero. For a regression coefficient, it may state that the coefficient equals zero under the specified model.
The exact formulation depends on the analysis. This is another reason not to write generic null hypotheses mechanically before deciding what quantity the study is actually evaluating.
A null hypothesis should not become a ritual sentence detached from the research design. It should correspond to the statistical question that the planned analysis can genuinely address.
Not Every Research Study Needs Formal Hypotheses
The requirement that every formal hypothesis have a corresponding analysis does not mean every study needs hypotheses.
Descriptive research may be organized around estimation rather than hypothesis testing. Exploratory studies may ask questions without specifying directional predictions. Many qualitative methodologies are driven by research questions, theoretical propositions, sensitizing concepts, or iterative inquiry rather than formal statistical hypotheses.
Do not manufacture hypotheses merely because a methodology template contains a heading for them. The presence of hypotheses should follow from the study's purpose, design, epistemological commitments, and analytical logic.
Every Hypothesis Should Be Testable With the Data You Plan to Collect
A hypothesis can sound plausible yet be impossible to evaluate with the proposed data.
Suppose you hypothesize that an intervention “causes long-term improvement,” but the design measures the outcome only immediately after the intervention. The analysis cannot create long-term evidence. Similarly, a hypothesis about change cannot be directly evaluated if the necessary temporal information was never collected.
This is why the analysis should be planned before data collection. Mapping each hypothesis to its intended analysis forces you to ask whether the study will produce the required evidence.
Multiple Hypotheses May Create a Multiplicity Problem
When a study tests many hypotheses, the probability of obtaining at least some apparently noteworthy results by chance can increase under conventional repeated significance testing. The appropriate response depends on why the hypotheses exist, how they are related, which are primary, and what inferential framework is being used.
In some settings, particularly confirmatory trials, multiplicity is addressed explicitly through prespecified testing strategies or adjustments intended to control relevant error rates. In other research contexts, estimation, transparent reporting, and appropriate caution may be more informative than mechanically applying the same correction to every analysis.
The important point is to consider multiplicity during advance analysis planning, rather than discovering after twenty tests that one crossed a conventional significance threshold.
Do Not Rewrite Hypotheses After Seeing the Results
Data can generate new hypotheses. That is scientifically useful. What changes is their evidential status.
If an unexpected relationship appears and inspires a new hypothesis, that hypothesis can be reported as exploratory or hypothesis-generating and investigated in future data. Presenting it as though it had been specified before examining the results obscures the distinction between prediction and discovery.
Advance mapping between hypotheses and analyses provides a record of what the study originally intended to test and makes later exploration easier to distinguish rather than harder to conduct.
