03 · What You Need to Know
The Answer Depends on How the Pilot Relates to the Main Study
Start With the Difference Between an Internal and External Pilot
The clearest starting point is whether the pilot is internal or external to the definitive study.
An external pilot is conducted separately before the main study. Its purpose is to investigate feasibility and refine the future research. Pilot data ordinarily remain separate from the definitive study, giving researchers greater freedom to modify procedures before the main study begins.
An internal pilot, by contrast, is designed as an initial phase of the definitive study. Participants recruited during that phase may contribute to the final analysis, provided the study proceeds and the conditions for retaining those data are satisfied.
External pilot
A separate preliminary study conducted before the definitive study; its data are ordinarily analyzed separately from the main-study data.
Internal pilot
A planned initial phase of the definitive study in which eligible participants and their data may contribute to the final analysis if progression occurs appropriately.
This distinction should ideally be established prospectively rather than invented after researchers see that the pilot data would be convenient to keep.
Why Internal Pilots Can Retain Participants
An internal pilot can reduce the inefficiency of recruiting participants solely for feasibility assessment and then starting the definitive sample from zero. The initial participants form part of the planned study, while early data provide an opportunity to examine specified feasibility parameters or design assumptions.
For example, an internal pilot within a randomized controlled trial might assess recruitment, retention, or other prespecified parameters during an initial phase. If progression criteria are met and no incompatible changes are required, recruitment continues and the internal-pilot participants remain part of the final trial dataset.
This approach requires careful planning because the feasibility assessment and definitive analysis are not completely independent. The protocol should explain the internal-pilot design, progression criteria, possible adaptations, and treatment of the initial participants.
External Pilot Data Are Different
An external pilot is intentionally separated from the definitive study so researchers can learn from it and make changes without necessarily preserving comparability with the later research.
The pilot may use an earlier version of an intervention, different questionnaire wording, different eligibility criteria, an incomplete follow-up procedure, a different recruitment strategy, or a data system that is subsequently redesigned.
Those differences may be exactly why the pilot was useful. They can also make its observations inappropriate for direct pooling with the main study.
If you originally designed the pilot as external, do not assume afterward that scientifically usable data should automatically become definitive data.
Ask What Changed After the Pilot
The most important practical question is often simple: what did you change?
Minor administrative modifications may have little effect on comparability. Correcting typographical errors, clarifying staff instructions without altering the participant procedure, or fixing a data-entry interface may not necessarily change what was measured or how participants were treated.
Other changes are more consequential. Researchers might modify the intervention, eligibility criteria, outcome measure, follow-up schedule, randomization process, recruitment population, assessment mode, or analytical strategy.
The more a change affects what participants experienced, what was measured, when it was measured, or who could enter the study, the harder it becomes to treat pilot and main-study observations as though they arose under one unchanged protocol.
If piloting leads to substantial modifications, consider whether the revised protocol remains the same study in a meaningful methodological sense.
Outcome Changes Require Particular Attention
Suppose the pilot shows that the planned primary outcome is difficult to collect, so the researchers replace it before the main study. Pilot participants may not have data on the new primary outcome at all.
Even subtler changes can matter. Revising questionnaire items, changing measurement timing, switching administration modes, or altering scoring can affect comparability.
Before pooling, ask whether the variable has the same meaning and measurement process across both phases. If not, simply placing observations in the same dataset does not make them methodologically equivalent.
Intervention Changes Can Make Pilot Participants Non-Comparable
A pilot may reveal that an intervention needs modification. Perhaps its duration changes, content is revised, staff training is strengthened, or delivery moves from face-to-face to online.
If pilot participants received a materially different intervention from main-study participants, combining their outcome data without accounting for that difference can blur the interpretation of the treatment being evaluated.
The appropriate response depends on the extent of the modification and the study design. Minor refinements may sometimes be compatible with a planned adaptive or internal-pilot framework. Major intervention changes may make pooling inappropriate.
Eligibility and Recruitment Changes Can Alter the Study Population
Suppose pilot recruitment is poor, so eligibility criteria are broadened before the main study. The pilot participants remain eligible under the new criteria, but the population from which later participants are recruited has changed.
That does not automatically prohibit pooling, but it requires methodological consideration. Changes to inclusion or exclusion criteria can alter the target population, event rates, baseline characteristics, intervention response, or generalizability of the study.
Similarly, adding new sites or recruitment channels may introduce differences that need to be understood rather than ignored.
Be Careful When Pilot Outcomes Influenced Main-Study Decisions
A particularly important issue arises when researchers inspect substantive pilot outcomes before deciding how to design or analyze the definitive study.
Suppose you examine which outcome favors the intervention and then designate that outcome as primary for the main study. Or you inspect subgroup effects and change the analysis accordingly. If those same pilot observations are then included in the definitive analysis, the data have influenced both the question and the answer.
This can introduce bias and undermine the independence of the planned analysis.
Testing whether the planned analytical workflow operates correctly is different from examining substantive pilot results and choosing a final analysis because it produces a favorable pattern.
Consent and Ethics Approval Must Cover the Intended Use
Methodological compatibility is not the only consideration. Participant data must be used consistently with the approved protocol, consent process, applicable regulations, and institutional requirements.
If pilot participants consented to a separate preliminary study, researchers should not assume that their data can automatically be repurposed as part of a definitive study simply because the variables are similar.
The exact requirements depend on the jurisdiction, institution, study type, consent language, and ethics or regulatory framework. When reuse was not prospectively planned, researchers should consult the relevant ethics review body or institutional authority rather than infer permission from methodological convenience.
Combining Data Is Not the Same as Reusing Participants
Two related questions should be separated.
You might ask whether data already collected from pilot participants can enter the definitive analysis. Alternatively, you might ask whether people who participated in the pilot can enroll again in the main study.
Allowing the same people to participate again can introduce additional concerns. Prior exposure may make them more familiar with study procedures, questionnaires, interventions, experimental tasks, or study hypotheses. That experience could change their behavior in the main study.
Whether repeat participation is acceptable depends on the design. It should not be assumed merely because existing pilot data will remain separate.
Plan the Decision Before the Pilot Whenever Possible
If you anticipate wanting pilot participants to contribute to the definitive study, address this during design rather than after seeing the pilot results.
Specify whether the pilot is internal, what progression criteria apply, what modifications are permissible while retaining the initial data, how adaptations will be documented, and what would require treating the pilot as external.
This prospective approach is methodologically cleaner because the decision to retain data is not made opportunistically after researchers know what those data contain.
Watch Out
Do not decide to pool pilot and main-study data merely because excluding the pilot would reduce your sample size. Statistical efficiency does not override differences in protocol, outcome measurement, intervention exposure, eligibility, consent, or data-dependent design decisions.
