03 · What You Need to Know
A changed result is something to investigate, not merely replace
First confirm that you are comparing the same study
Before interpreting a discrepancy, establish that the preprint and journal article actually describe the same underlying research.
Titles can change. Author order can change. Sample sizes can increase. A preprint may report an interim analysis while the journal article reports final follow-up. A large research project may also generate several papers addressing different questions.
Compare identifying characteristics such as:
- study registration number;
- authors and institutions;
- population and eligibility criteria;
- study setting;
- recruitment period;
- study design;
- intervention, exposure, or comparison groups;
- sample size;
- outcomes;
- follow-up period;
- study acronym, funding, or project identifiers.
If these characteristics indicate that the documents describe the same research, treat them as different reports or versions of one study. Do not count them as two independent studies.
Different results do not necessarily mean opposite conclusions
There are several levels at which preprints and journal articles can disagree.
| Type of difference |
Example |
Possible importance |
| Numerical difference |
An effect estimate changes from 0.72 to 0.78 |
May be minor if the substantive interpretation remains unchanged. |
| Precision difference |
A confidence interval becomes narrower after additional participants |
May strengthen or weaken confidence without changing effect direction. |
| Statistical significance difference |
A p-value changes from below to above a conventional threshold |
May alter wording but should not automatically be interpreted as a reversal of the underlying effect. |
| Direction difference |
An effect favoring the intervention later favors the comparison |
Potentially consequential and warrants close investigation. |
| Outcome difference |
An outcome appears in one version but not the other |
May raise questions about reporting changes or different analytic priorities. |
| Conclusion difference |
The preprint claims benefit while the journal article concludes that evidence is inconclusive |
Directly affects how the study should be represented in a review. |
Do not collapse these differences into a single category called "the results changed." A small numerical revision is not equivalent to a reversal of the primary conclusion.
Empirical research suggests that many versions remain broadly consistent
Differences between preprints and subsequent publications are real, but dramatic reversals do not appear to be the norm in the empirical literature examined so far.
A study of 547 medRxiv preprint-journal article pairs describing clinical trials, observational studies, and meta-analyses found that 97.6% reported concordant primary endpoints and 96.2% had concordant overall study interpretations. Exact numerical primary-endpoint results were concordant in 81.1% of pairs. Even among pairs with discordant numerical primary-endpoint results, most retained the same overall interpretation.
Another systematic review of 152 COVID-19 randomized trial preprints found that 119 had subsequently been published in journals. Differences in outcomes, analyses, results, or conclusions occurred in 65 of those 119 pairs, but the main conclusion contradicted the preprint in only two studies.
These findings suggest that changes are common enough to justify checking but that a changed number should not automatically be interpreted as a complete reversal of the research.
Some discrepancies are ordinary consequences of a study progressing
A preprint is often posted while research communication is still evolving. Between preprint posting and journal publication, legitimate changes may occur.
For example:
- additional participants may complete follow-up;
- data cleaning may identify errors or duplicate records;
- missing data may be updated;
- outcome definitions may be clarified;
- analyses may be corrected;
- sensitivity analyses may be added;
- statistical models may be revised;
- authors may respond to methodological criticism;
- tables or figures may be corrected;
- interpretation may become more cautious.
A difference is therefore not automatically suspicious. What matters is whether the change is transparent, methodologically defensible, and important to the inference.
Check whether the sample changed
Sample-size differences are one of the first things to investigate.
In the medRxiv comparison study, 86.4% of 535 pairs reporting sample size in both versions had exactly concordant sample sizes. Among pairs with different sample sizes, journal publications more often contained larger samples.
A larger published sample may reflect continued recruitment, additional follow-up, correction of exclusions, or differences in the analysis population. Conversely, a smaller sample may result from revised eligibility criteria, removal of invalid observations, or use of a different analysis set.
Ask whether the numerical result changed because the underlying data changed before assuming that the statistical analysis itself was responsible.
Check whether the outcome itself changed
A result can appear different because the two documents are not reporting exactly the same outcome.
Compare:
- outcome definition;
- measurement instrument;
- assessment time point;
- primary versus secondary designation;
- continuous versus categorical representation;
- composite versus individual outcomes;
- adjusted versus unadjusted estimates.
Research comparing COVID-19 preprints with subsequent publications found that outcomes sometimes appeared only in the preprint or only in the journal article. Differences also occurred in outcome descriptors, measures, metrics, assessment time points, numerical values, statistical significance, and subgroup analyses.
If the outcome changed, you may not be comparing two estimates of precisely the same thing.
Check whether the analysis changed
The dataset can remain largely the same while the result changes because the analysis differs.
Look for changes in:
- statistical model;
- covariate adjustment;
- handling of missing data;
- intention-to-treat versus per-protocol analysis;
- transformations;
- subgroup definitions;
- multiple-comparison procedures;
- exclusion rules;
- sensitivity analyses.
Do not assume that a more complex analysis is automatically better. Determine why it was used and whether it was prespecified or introduced later.
A change in statistical significance is not necessarily a reversal
Suppose a preprint reports an effect estimate with p = 0.04 and the journal article reports a similar estimate with p = 0.06.
It would be misleading to describe this mechanically as "the preprint found an effect, but the published paper found no effect." The underlying estimates may be extremely similar.
Statistical significance depends on the estimate, its uncertainty, sample size, analytic choices, and the selected threshold. Moving from one side of 0.05 to the other does not create a scientific discontinuity.
Compare effect estimates and confidence intervals rather than relying only on whether an asterisk disappeared from the table.
Watch Out
Do not equate "statistically significant" with "true" and "not statistically significant" with "no effect." When significance changes between versions, examine the magnitude and uncertainty of the estimates and the analytical changes that produced them.
Check the study registration or protocol when one exists
If a study was prospectively registered or has a protocol, those records can help you determine which outcomes and analyses were planned before the results were known.
Compare the preprint and journal article with:
- registered primary and secondary outcomes;
- planned time points;
- target sample size;
- prespecified analyses;
- protocol amendments;
- dated changes to the registration.
This matters because differences between preprint and publication do not occur in isolation from the study's earlier plans. A systematic review and meta-analysis of registration-publication comparisons found discrepancies in primary and secondary outcomes across substantial proportions of studies, although the literature was heterogeneous and often did not clearly establish whether discrepancies were disclosed or whether the registration was prospective.
The registration is not automatically the unquestionable truth either. Protocols can legitimately change. What matters is whether important changes are transparent and justified.
Do not assume that peer review caused the change
The sequence "preprint first, peer-reviewed article later" makes it tempting to attribute every difference to peer review.
That conclusion usually goes beyond the available evidence.
Authors may independently identify errors. Readers may comment publicly on a preprint. Editors may request revisions. Reviewers may identify methodological problems. Additional data may become available. A statistical collaborator may recommend another analysis. The manuscript may also change before it is even submitted to the journal.
Unless editorial correspondence, peer-review reports, author statements, or another reliable record establishes the cause, describe the difference as a change between the preprint and published version.
Do not claim that "peer review corrected the study" merely because the correction appeared before journal publication.
Should you trust the published paper instead?
For ordinary citation and interpretation, the subsequent peer-reviewed article will usually be the primary current report of the study. It has passed through additional editorial and peer-review processes and normally represents the formal publication.
But "use the published paper" should not become "ignore everything that came before it."
The preprint may reveal:
- an earlier outcome that disappeared;
- a different analysis;
- a larger or smaller claim;
- additional methodological detail;
- an earlier sample;
- a result that was subsequently corrected;
- information omitted from the journal article.
When those differences affect your interpretation, the publication history itself becomes relevant evidence.
Which version should you cite?
If the journal article supports the claim you are making and represents the same study, it will usually be the appropriate citation for the current result. The broader principles for choosing between a preprint and its peer-reviewed article still apply.
If you specifically discuss how the result changed, cite both versions. Readers need access to both documents to verify the comparison.
If information appears only in the preprint, cite the preprint for that information rather than attributing it to the journal article.
Reporting the study's current published finding
Usually cite the peer-reviewed journal article.
Discussing a discrepancy between versions
Cite both the preprint and journal article so the change can be verified.
For a systematic review, update the study rather than adding another study
If your systematic review originally included the preprint and a journal article subsequently appears, do not add the article as another independent study.
Cochrane guidance emphasizes identifying and linking multiple reports from the same underlying study because duplicate inclusion can bias a meta-analysis. It also recommends searching for peer-reviewed publications of included preprints before manuscript completion.
Revisit the study record and update:
- eligibility information;
- study characteristics;
- sample size;
- outcomes;
- effect estimates;
- risk-of-bias judgments;
- funding and conflicts;
- interpretation.
If the journal version changes the result used in a meta-analysis, rerun the synthesis with the updated value.
Reassess risk of bias when the later version adds methodological information
A journal article may explain procedures that were unclear in the preprint. Alternatively, comparison with a registration or protocol may expose outcome or analysis changes that were not previously apparent.
Your appraisal should therefore be revisited rather than copied forward automatically.
The later version might strengthen confidence because important methods are clarified. It might also create new concerns if selective reporting, changed analyses, or unexplained discrepancies become visible.
Publication status itself should not determine the risk-of-bias judgment. The newly available information should.
What if the published result weakens the finding?
Suppose the preprint reported a large statistically significant effect while the published article reports a smaller and more uncertain effect.
For the study's current result, use the published estimate unless there is a compelling methodological reason not to do so. Then investigate what changed.
If the published analysis corrected an error, used the final sample, or followed a more appropriate prespecified method, that context strengthens the rationale for relying on it. If the change is unexplained and involves a switch from a prespecified outcome to another analysis, the discrepancy may instead raise concerns that deserve discussion.
Do not select the estimate that better fits the narrative of your review.
What if the published result strengthens the finding?
The same scrutiny applies.
A result becoming larger or statistically significant after publication is not inherently suspicious, but neither should it be accepted without checking why. Additional data, longer follow-up, corrected errors, changed analysis populations, or revised statistical methods can all move an estimate.
Researchers sometimes scrutinize weakened findings carefully while welcoming strengthened findings without questions. That asymmetry is a recognizable form of confirmation bias wearing a methods section.
What if the direction of effect reverses?
A reversal deserves close examination because it may materially change the study's interpretation.
Check:
- whether the same outcome and time point are being compared;
- whether coding or reference categories changed;
- whether the analysis population changed;
- whether additional data were collected;
- whether an error was corrected;
- whether the effect measure is expressed in the opposite direction;
- whether adjusted and unadjusted estimates are being confused.
Sometimes an apparent reversal is merely a change in how an effect is expressed. For example, one report may model success while another models failure, producing reciprocal or sign-reversed estimates without a substantive contradiction.
Verify the definitions before announcing that the study reversed its conclusion.
What if an outcome disappears from the published article?
An outcome reported in the preprint but absent from the journal publication deserves attention, particularly if it was important to the research question or prespecified as a primary or secondary outcome.
Do not automatically infer selective suppression. The outcome may have been moved to supplementary material, reserved for another paper, removed because of space constraints, or excluded for a methodological reason.
Check the supplements, registration, protocol, and related publications. If the reason remains unclear and the missing outcome matters to your review, the discrepancy should be documented rather than silently ignored.
What if the published paper adds a new outcome or analysis?
Again, investigate rather than assume wrongdoing.
A new analysis may have been requested during review, added after legitimate methodological reconsideration, or conducted in response to new information. But if it becomes central to the paper's conclusion, knowing whether it was prespecified or post hoc can affect interpretation.
Registrations and protocols are particularly useful here because they provide a time-stamped reference point for planned analyses.
Major discrepancies should be transparent in your own review
If a difference between versions materially affects your conclusion, tell the reader.
You do not need to catalogue every changed decimal place. Focus on discrepancies that affect:
- eligibility;
- effect direction or magnitude;
- statistical uncertainty;
- primary outcomes;
- risk-of-bias assessment;
- the study's overall conclusion;
- your synthesis or certainty judgment.
A concise explanation is often sufficient: the study was initially available as a preprint, the published version reported a revised analysis or result, and the review uses the later estimate while noting the consequential difference.
Sometimes the discrepancy is itself a research-integrity signal
Most version changes are not evidence of misconduct. Revision is a normal part of research communication.
Concern increases when important changes are unexplained, inconsistent with a prospective registration, selectively favor a preferred conclusion, or occur alongside other reporting problems.
If a discrepancy appears serious, examine the protocol, registration, corrections, editorial notices, supplementary files, and other reports before drawing conclusions. In rare cases, formal corrections, expressions of concern, withdrawals, or retractions may provide essential context.
The appropriate response is evidence gathering, not speculation about motive.
