When Does Changing Your Research Sample Turn It Into a Different Study?

When Is a Sample Change More Than a Recruitment Adjustment?

Research samples rarely unfold exactly as planned. One recruitment site may withdraw. An eligibility criterion may prove unnecessarily restrictive. A subgroup may be difficult to recruit. Researchers may need to add sites, extend recruitment, broaden an age range, or accept a different distribution of participants from the one originally anticipated.

Some of these changes leave the scientific identity of the study largely intact. Others alter the population represented by the evidence so substantially that the revised project is no longer answering quite the same question.

There is no universal percentage of participants, eligibility criteria, or recruitment sites that marks the boundary. The more useful question is whether the revised sample still provides evidence about essentially the same population, phenomenon, and target of inference as the study originally intended.

A Sample Change Becomes Fundamental When It Changes Who the Study Is About

The Number of Participants Is Not What Defines the Study Population

Sample Size and Sample Identity Are Different Questions

Suppose a study planned to recruit 300 participants but eventually recruits 240. The sample is smaller than intended, but it may still come from precisely the population originally specified.

Now consider another study that recruits all 300 participants but obtains the final 100 by broadening eligibility to people outside the original population. The numerical target has been achieved, yet the scientific meaning of the sample may have changed.

If the primary problem is simply that fewer participants can be recruited than planned, the more immediate issue is what to do when the planned sample size cannot be reached. A different question arises when solving that shortfall requires changing who can enter the study.

Ask What the Eligibility Criteria Were Doing Scientifically

Eligibility criteria are not merely administrative filters. They help define the population represented by the study and may also serve methodological or ethical purposes.

An age restriction may identify a developmental population. A diagnostic criterion may define the condition under investigation. Previous experience may be relevant to how participants respond to an intervention. A language requirement may reflect the availability of validated measures. A clinical exclusion criterion may protect participants from unacceptable risk.

Changing one criterion can therefore have very different consequences depending on why it existed.

The same procedural change can therefore be minor in one study and transformative in another.

Adding Another Site Does Not Automatically Mean Adding Another Population

A recruitment site is a location or organizational source, not necessarily a population definition. Adding another university, clinic, community, school, or organization may simply provide additional access to the population already specified.

But settings can also structure the population. A specialist hospital may serve people with different severity profiles from community clinics. Private and public schools may differ in ways relevant to an educational intervention. Online recruitment may reach participants with different characteristics from those approached through a local organization.

When adding sites, ask what selection mechanism the new setting introduces. If the setting changes who has a realistic chance of entering the study, it can change more than recruitment speed.

Do Not Use Demographic Similarity as the Only Test

Researchers sometimes compare the original and revised samples using age, sex, education, or another short demographic table and conclude that the populations are equivalent because the percentages look similar.

That can be reassuring, but it does not settle the methodological question. Two samples can look similar on measured demographics while differing in motivation, severity, prior exposure, institutional practices, access, or other characteristics relevant to the study. Conversely, samples can differ demographically without those differences necessarily changing every target inference.

Focus on characteristics that have a plausible relationship to the phenomenon, intervention, exposure, measurement, or conclusion rather than treating demographic resemblance as proof of interchangeability.

A Sampling Change Can Alter Generalizability Without Creating a Completely Different Study

Not every shift in sample composition transforms the study. Sometimes the central question remains intact, but the range of populations to which the findings can reasonably apply becomes narrower.

For example, a study intended to recruit participants from several geographic regions may ultimately recruit mostly from one region. The study may still provide useful evidence about the phenomenon among the participants studied while offering weaker support for broader geographic generalization.

In such cases, the appropriate response may be to narrow the claim rather than declare the project a new study.

If the people who can realistically be recruited differ from those originally intended, assess the implications of a recruitable population that differs from the planned study population before changing the study's identity.

Changing the Sample Can Change the Effect or Phenomenon Being Studied

Population differences are sometimes discussed only as a problem of generalizability. That can underestimate their importance.

If intervention effects vary across participant characteristics, changing the population can change the average effect the study estimates. If an exposure operates differently across contexts, changing recruitment settings can alter the association of interest. If an instrument functions differently across populations, the measurement process can change as well.

A population change may therefore alter the scientific quantity or phenomenon itself rather than merely limiting where the result can be generalized.

Changing the Sampling Strategy Can Alter the Inference Even When Eligibility Stays the Same

Suppose the eligibility criteria remain unchanged but the study moves from probability-based sampling to open volunteer recruitment because the original strategy proves infeasible. The formal population definition may be identical, yet the mechanism determining who enters the study has changed.

That can matter if the original research question requires population estimates supported by the sampling design. The volunteer sample may still answer useful questions, but it should not automatically inherit the inferential properties of the original sampling strategy.

STROBE reporting guidance emphasizes describing eligibility criteria and the sources and methods of participant selection. This information matters precisely because readers need to understand how the observed sample arose.

Consider Whether Data Collected Before and After the Change Belong Together

A mid-study sample change can produce phases with meaningfully different participant populations.

Suppose the first 150 participants satisfy one eligibility definition and the next 150 are recruited under broader criteria. Pooling all 300 participants may be appropriate in some circumstances. In others, the modification may introduce heterogeneity that needs to be represented explicitly in analysis and interpretation.

Before combining the observations automatically, consider whether the same construct, intervention, exposure, and outcome retain comparable meanings across the original and expanded populations.

Changing the Sample After Seeing Outcomes Requires Additional Caution

The reason for a sample change matters. Broadening eligibility because an independently documented recruitment problem makes the original population inaccessible is different from changing eligibility after discovering that a subgroup produces inconvenient results.

Excluding, adding, or redefining participants in response to observed outcomes can introduce selection bias and undermine the distinction between prespecified and data-dependent decisions.

Ethics Approval and Methodological Identity Are Separate Questions

Changing eligibility criteria, recruitment sites, or participant populations may require review or approval under the rules governing the study. That administrative determination does not, by itself, answer whether the project remains scientifically the same study.

A change could receive approval as an amendment while still altering the target population substantially. Conversely, a change requiring formal ethics review may have relatively limited implications for the research question.

Determine separately which changes require revisiting ethics approval, the protocol, or preregistration.

Several Sample Changes Can Accumulate

A study may broaden its age range, add new sites, relax an exclusion criterion, and shift recruitment from institutions to social media. Each modification might initially appear manageable.

Collectively, however, they may create a participant population and selection process substantially different from the original design. Evaluate the cumulative effect rather than classifying each change in isolation.

The relevant comparison is between the original study population and the population actually represented by the final design.

A Study Broadens Its Sample to Solve a Recruitment Problem

Adding participants is not inherently an improvement when the added participants change the population embedded in the research question. A smaller study of the intended population may sometimes be more coherent than a larger study whose sample answers a different question.

Common Misconceptions About Changing a Research Sample

Compare Who the Study Was About With Who It Is About Now

When the sample changes, write down the original population definition and the revised one. Then identify every difference that could plausibly affect the phenomenon, intervention, exposure, measurement, or intended inference.

The boundary is ultimately substantive. Ask whether the study you would describe today is still investigating essentially the same people for essentially the same scientific reason.

If the sample change is only one part of a broader methodological transformation, consider whether changing the research method has also turned the project into a different study.

Has Changing the Sample Changed the Study?

Questions About Sample Changes and Study Identity

The Study Changes When the Sample Changes What the Research Is About

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.