03 · What You Need to Know
How to Tell a Defensible Deviation From a Problematic One
Preregistration Was Never Meant to Make Researchers Follow Known Mistakes
A preregistration records what researchers intended to do before the relevant results were known. That timing can make it easier to distinguish advance decisions from choices made after researchers learned something from the data.
But advance decisions are not automatically correct decisions.
You can preregister an inappropriate statistical model. You can miscalculate a target sample size. You can misunderstand how a variable is coded. You can specify an exclusion rule that later proves impossible to implement. You can simply make a mistake.
Nosek and colleagues explicitly acknowledge that deviations can improve research when researchers discover errors in their original plans or learn that a better approach is available. The objective is therefore not perfect correspondence between preregistration and publication.
This is why preregistration does not mean that nothing can change.
Some Deviations Improve the Study
Imagine discovering that the statistical analysis you preregistered rests on an assumption that is clearly inappropriate for the data structure. You now know that another method provides a more defensible analysis.
Following the original method merely because it was preregistered would preserve adherence at the expense of methodological quality.
A better approach may be to use the more appropriate analysis and explain the deviation. Depending on the circumstances, reporting the preregistered analysis as well may help readers understand the effect of the change.
The same logic applies to corrections of coding errors, mistaken exclusion rules, inappropriate measurement procedures, and other problems discovered after registration.
Deviation to improve validity
The original plan is changed because new information reveals an error, methodological problem, or more defensible approach.
Deviation to improve the result
The plan is changed because another choice produces a more favorable, statistically significant, theoretically convenient, or otherwise preferred outcome.
The first can be entirely defensible. The second deserves considerably more scrutiny, particularly when the result influenced the choice and that influence is not disclosed.
Some Deviations Are Practically Unavoidable
Research plans interact with the world, and the world has not read your preregistration.
A participating site may withdraw. Recruitment may stop before the planned sample is reached. Equipment may fail. A software service may change. A planned dataset may become inaccessible. A researcher may leave the project. An intervention may need to move online.
These events can require changes even when the original plan was methodologically sound.
The existence of such a deviation says little by itself about research quality. The relevant questions concern what changed, why it changed, and whether the modification affects the study's scope, comparability, precision, or interpretation.
Ethical Reasons Can Require a Deviation
Ethical obligations take precedence over adherence to a preregistered procedure.
If a procedure creates an unforeseen risk to participants, new information affects informed consent, a privacy problem emerges, or an ethics body requires a modification, researchers should not continue the original procedure merely to preserve preregistration fidelity.
Such changes may also require formal approval under institutional or regulatory procedures. Those requirements depend on the research context and should be handled through the relevant ethics or oversight process.
Preregistration does not override those obligations.
Timing Changes How a Deviation Should Be Interpreted
Suppose two researchers make exactly the same change to a statistical model.
The first discovers an error in the preregistration before collecting any data. The second compares several analyses after seeing the primary result and selects a different model because it produces a more favorable finding.
The final analyses may be identical, but the decision processes are not.
This is why the timing of a deviation matters. Researchers should consider what information was available when the revised decision was made.
| Situation |
Result knowledge |
Typical interpretive concern |
| Error corrected before relevant data are collected |
Relevant outcome unknown |
The correction was not selected because of the eventual result. |
| Procedure changed during data collection for logistical reasons |
Relevant outcome may still be unknown |
The change may affect implementation, comparability, or generalizability. |
| Analysis changed after an assumption problem becomes visible |
Relevant data have been examined |
The change may be justified, but the decision is now data-informed. |
| Outcome or model changed after comparing which produces the preferred result |
Results known |
There is substantial risk of result-contingent selection. |
A Data-Informed Deviation Is Not Automatically Wrong Either
The fact that a researcher has seen the data does not make every subsequent methodological decision improper.
Some problems can only become apparent after examining the data. A model may fail to converge. A distribution may make a planned procedure untenable. A measurement problem may become visible only during cleaning. An unexpected structure may require a different statistical treatment.
The appropriate response is not to pretend that the decision remained independent of the data. It is to disclose that the change was made after relevant information became available and explain the methodological reason.
Readers can then evaluate whether the revised analysis is persuasive and how strongly it should be interpreted as confirmatory.
Exploratory Analyses Are Not Illicit Deviations
Suppose your preregistered analysis is completed exactly as planned. You then notice an unexpected pattern and conduct several additional analyses to understand it.
You have gone beyond the preregistration, but that does not mean you have done something wrong.
Exploration is a legitimate part of research. The key is to distinguish those analyses from the ones specified prospectively. An analysis inspired by an observed pattern should not be presented as though it were an advance test of a hypothesis that existed before the pattern was seen.
This preserves the distinction between exploratory and confirmatory research without treating exploration as a methodological embarrassment.
A Deviation Can Affect Only Part of the Study
Another common mistake is to treat preregistration as all-or-nothing.
Imagine that the research question, hypothesis, sample, primary outcome, and exclusion criteria all remain as preregistered, but the primary statistical model changes for a defensible reason.
It would be misleading to claim perfect adherence. It would also be unnecessarily crude to say that nothing about the study remained preregistered.
The more informative approach is component-specific: identify which parts followed the plan and which did not.
Willroth and Atherton describe preregistration deviations as discrepancies between the preregistration and the final article and argue for transparent reporting that allows readers to evaluate those differences rather than assuming that any deviation automatically invalidates the research.
Some Deviations Have Greater Inferential Consequences Than Others
Moving a laboratory session from one room to another is not equivalent to changing the primary outcome after seeing which outcome produces the strongest effect.
Changing a recruitment advertisement may have different consequences from changing an exclusion criterion after observing which cases weaken the result. Correcting a typographical error in an analysis script is different from testing several models and reporting only the statistically significant one.
When evaluating a deviation, ask:
- Could the change alter the study's primary conclusion?
- Was the decision influenced by knowledge of the result?
- Did it introduce additional analytical flexibility?
- Did it change the hypothesis, primary outcome, analyzed sample, or inferential procedure?
- Would a reasonable reader interpret the evidence differently if they knew about the change?
The more strongly the answer points toward consequential influence, the more important explicit reporting becomes.
Disclosure Is Necessary, but It Does Not Make Every Decision Sound
Transparency should not be confused with methodological absolution.
A researcher could openly state that the primary outcome was changed after seeing that another outcome produced a statistically significant result. The disclosure is better than concealment, but it does not restore the revised analysis to the evidential status of an independently prespecified primary test.
Likewise, reporting a poorly justified deviation does not make the methodological problem disappear.
Transparency gives readers the information needed to evaluate the change. It does not determine what their evaluation should be.
Watch Out
“We disclosed the deviation” and “the deviation does not affect the inference” are separate claims. Report the change transparently, then consider whether it alters the strength, scope, or confirmatory status of the conclusion.
Registered Reports May Impose Additional Requirements
When the study is being conducted as a Registered Report, deviations involve an additional consideration: the Stage 1 plan has undergone prospective journal review and may have received in-principle acceptance.
Publishers and journals can specify procedures for handling changes. For example, current Elsevier Registered Reports guidance advises authors to consult the handling editor before deviating from proposed materials and methods.
Researchers should therefore verify the target journal's current policy rather than assuming that the same amendment process applies to ordinary preregistration and Registered Reports.
This is one of the practical differences between preregistration and Registered Reports.
Good Preregistration Makes Deviations Interpretable, Not Impossible
A preregistration that is sufficiently specific gives later deviations meaning. If the original plan simply says “appropriate analyses will be conducted,” almost any eventual analysis could be claimed as consistent with it.
Specificity therefore creates the possibility of identifying differences between intention and execution.
That can feel uncomfortable because deviations become visible. But visibility is part of the point.
A study with several transparently reported deviations may provide a more faithful research record than a supposedly flawless preregistered study whose registration was too vague to constrain anything consequential.