03 · What You Need to Know
The Timing of a Hypothesis Changes What the Evidence Can Tell You
Changing Your Mind Is Part of Research
A hypothesis is not a personal commitment that researchers must defend regardless of what the evidence shows. One purpose of empirical inquiry is precisely to expose expectations to evidence and revise them when necessary.
Suppose you predict that students with greater generative AI use will demonstrate higher AI literacy. Instead, the association is weak overall, but exploratory analysis suggests that the relationship differs sharply according to how students use AI.
It is entirely reasonable to develop a new hypothesis that distinguishes productive from passive forms of use. The evidence has taught you something.
What changes is the evidential status of that new prediction. The current observations helped generate it, so they did not independently predict and test it in the same way they would have tested a hypothesis specified beforehand.
What Is an A Priori Hypothesis?
In this context, an a priori or prespecified hypothesis is one formulated before the researcher examines the results relevant to testing that prediction.
For example:
Before examining the outcome: Students receiving structured retrieval practice will achieve higher delayed-test scores than students receiving rereading.
The subsequent data can then be evaluated against a prediction that was not selected because it happened to fit those data.
Advance specification is particularly valuable in confirmatory research because it constrains the researcher's opportunity to tailor hypotheses and analytical choices to observed results.
What Is a Post Hoc Hypothesis?
A post hoc hypothesis is formulated or modified after relevant results have become known.
Suppose your original hypothesis predicts:
Greater AI use will be associated with higher academic performance.
The overall association is absent, but students who use AI primarily for feedback appear to perform better. You then formulate:
Using generative AI for feedback will be positively associated with academic performance.
That may be a worthwhile hypothesis. Its source is the observed pattern, however, so it should be reported accordingly.
What Is HARKing?
Norbert Kerr introduced the term HARKing, short for "Hypothesizing After the Results are Known," for presenting a post hoc hypothesis that was informed by the results as though it had been an a priori hypothesis.
The crucial problem is therefore not merely that a hypothesis was developed after seeing the data. It is the misrepresentation of its timing.
Exploratory reasoning can generate scientific insights. If researchers conceal that process and rewrite the introduction so that the eventual result appears to have been predicted all along, readers receive a misleading account of how strongly the data tested that prediction.
Post hoc hypothesis generation
The results suggest a new prediction, and the researcher reports transparently that the hypothesis emerged from those results.
HARKing
A hypothesis informed by the observed results is presented as though it had been specified before those results were known.
Why Does the Timing Matter?
Imagine throwing a dart at a wall and then drawing the target around wherever it lands. The resulting bullseye tells you much less about your accuracy than a target drawn before the throw.
Data analysis is more complicated than darts, but the underlying problem is similar. A sufficiently rich dataset can contain many possible relationships, subgroups, outcomes, transformations, covariates, and analytical choices. If researchers use the data to select a promising pattern and then present that pattern as an advance prediction, the apparent success of the prediction becomes overstated.
The distinction is particularly consequential when conventional inferential procedures are interpreted as though the hypothesis and analytical plan were selected independently of the observed results.
Changing the Hypothesis Can Also Change the Statistical Problem
Suppose your original hypothesis concerns an overall intervention effect. After seeing the results, you notice that the effect appears only among first-year students and change the hypothesis to predict an effect specifically for that subgroup.
The new hypothesis is not simply a better sentence. You have selected a subgroup because of what appeared in the data.
If several subgroups or analytical alternatives were available, the selected pattern emerged from a broader search space. Conventional p-values or confidence intervals calculated as though that subgroup had been the only planned analysis may not fully reflect that selection process.
This is one reason the American Statistical Association emphasizes that proper statistical inference requires transparency about the hypotheses and analyses explored rather than reporting only the result eventually selected for presentation.
Do Not Delete the Original Hypothesis Simply Because It Was Wrong
If the original hypothesis was genuinely part of the study and the study was designed to evaluate it, its lack of support is itself part of the scientific result.
Suppose you predicted a positive relationship and observed essentially no relationship. Removing the hypothesis from the paper because it "didn't work" distorts the record of what the study was intended to investigate.
The appropriate response is usually to report the original prediction and its result, then explain any exploratory patterns that motivated revised hypotheses.
This separation allows readers to distinguish what the study set out to test from what the study subsequently discovered.
Do Not Quietly Reverse the Direction Either
Suppose you predicted:
Higher AI dependence will be associated with lower critical-thinking performance.
The data instead suggest a positive association.
You should not revise the introduction to say that you predicted a positive association simply because that is what appeared. The original directional hypothesis was not supported.
The unexpected direction may be more interesting than the predicted one. Report it as such. Scientific surprise is not a formatting error.
A New Hypothesis Can Be Better Than the Original One
Post hoc does not mean worthless.
An unexpected pattern may expose a weakness in the original theory, reveal an overlooked moderator, suggest a new mechanism, or identify a more precise boundary condition. Some important scientific ideas originate in results researchers did not anticipate.
The new hypothesis can therefore be theoretically stronger and scientifically more interesting than the original one.
What it cannot do is travel backward in time and become an advance prediction of the observations that produced it.
What Should You Call the Revised Hypothesis?
Terminology varies across disciplines, but useful descriptions include:
- post hoc hypothesis;
- exploratory hypothesis;
- hypothesis generated from the observed findings;
- data-informed hypothesis;
- hypothesis for future testing.
The important point is not finding a ceremonial label. Readers should understand when and how the prediction arose.
Can You Test the New Hypothesis With the Same Dataset?
You can examine how the new hypothesis relates to the data that generated it, but this should not automatically be interpreted as an independent confirmatory test.
A stronger strategy is often to evaluate the new prediction using new data or an appropriately independent portion of existing data. In some settings, researchers may use data splitting, holding one subset for exploration and another for validation, although whether this is useful depends on sample size, design, and analytical goals.
The basic principle is straightforward: evidence that helped select a hypothesis and evidence used to test it do not play identical inferential roles.
This distinction follows from separating the evidence that generates a hypothesis from the evidence that subsequently evaluates it.
Preregistration Can Make Changes Easier to See
Preregistration records specified aspects of a research plan before the relevant results are known. Depending on the registration, this may include hypotheses, outcomes, exclusion criteria, sample-size decisions, and planned analyses.
If the eventual hypothesis or analysis differs from the preregistration, that does not automatically invalidate the study. Research plans sometimes need to change for legitimate reasons.
The important step is to disclose the deviation and explain why it occurred. A transparent report might distinguish the preregistered hypothesis from an exploratory hypothesis generated during analysis.
What if You Realize the Original Hypothesis Was Poorly Formulated?
Sometimes the problem is discovered before analysis. Perhaps you recognize that the hypothesis is ambiguous, the direction is unsupported, or an outcome was described incorrectly.
If the relevant results have not yet been examined, revising the hypothesis and documenting the change may still preserve its status as an advance prediction, depending on what information was available when the revision occurred.
If the result has already been seen, the situation changes. You can still correct a genuine wording error, but substantive changes that make the prediction fit the observed outcome should be disclosed.
What if the Analysis Reveals a Measurement or Coding Error?
Correcting an error is different from changing a hypothesis to accommodate an inconvenient result.
If a variable was miscoded, an instrument scored incorrectly, or an analysis implemented improperly, researchers should correct the error and document consequential changes. The relevant hypothesis does not become post hoc merely because the analysis had to be repaired.
However, if the correction leads researchers to formulate an entirely new substantive prediction, that new hypothesis should be distinguished from the original one.
Exploratory and Confirmatory Work Can Coexist in One Paper
You do not need to hide exploration to produce a coherent article.
A paper can report the prespecified hypothesis and its planned analysis, then present unexpected patterns as exploratory findings and formulate new hypotheses from them. The discussion can explain how those discoveries alter the original theoretical account and what future research should test.
This is precisely why exploratory research can legitimately generate hypotheses. Discovery and confirmation are both valuable when their roles remain visible.
Do Not Treat p <.05 as Permission to Rewrite the Story
A statistically significant exploratory result does not retroactively make the corresponding hypothesis prespecified. Nor does statistical significance alone establish scientific importance.
The American Statistical Association cautions that scientific conclusions should not depend solely on whether a p-value crosses a particular threshold and emphasizes full reporting and transparency about analyses conducted.
Report the estimated effect, uncertainty, study design, analytical context, and exploratory status rather than allowing one threshold to determine which version of the research story survives.
Watch Out
Do not change, remove, reverse, or narrow a hypothesis after seeing the results and then present the revised version as the study's original prediction. You may revise the scientific explanation; preserve the history of that revision.
