When Do Inclusion and Exclusion Criteria Create an Unrepresentative Sample?

Can Reasonable Eligibility Rules Produce the Wrong Sample for the Question?

Every study needs boundaries. Researchers may specify an age range, diagnosis, educational level, geographic location, prior exposure, language ability, health status, medication history, or other conditions that determine who can participate.

Those boundaries can be essential. Without them, the study population may become poorly defined, participant safety may be compromised, or the research question may be impossible to answer clearly.

But eligibility criteria also remove people from the pool from which the sample is recruited. If the criteria systematically exclude substantial parts of the population to which researchers later want to apply the findings, a tension emerges: the study may become easier or more internally controlled while providing evidence about a population narrower than the one implied by its conclusions.

Eligibility Criteria Become a Problem When They Narrow the Population Without Adequate Justification

Eligibility Criteria Define the Population Before Sampling Even Begins

Researchers often think of representativeness as a sampling issue: Was random sampling used? Did enough people respond? Does the demographic distribution resemble the population?

But sampling happens only after the eligible population has been defined.

If eligibility rules have already removed large or systematically different segments of the target population, even excellent sampling from the remaining eligible pool cannot restore those excluded people. This makes eligibility criteria part of the logic of population inference.

Inclusion and Exclusion Criteria Serve Legitimate Purposes

Inclusion criteria identify characteristics required for participation. Exclusion criteria identify conditions that make an otherwise relevant person ineligible.

Criteria may be necessary to define the phenomenon being studied, protect participants, ensure that an intervention is appropriate, reduce particular sources of ambiguity, satisfy regulatory requirements, or make measurement meaningful.

A study of first-year university students appropriately excludes students in later years if the research question specifically concerns transition into university. A clinical investigation may need particular diagnostic or safety criteria. A language study may legitimately require proficiency in the language being investigated.

The mere existence of exclusion therefore tells us very little about whether the study is well designed.

The First Question Is Representative of What?

Representativeness cannot be evaluated without identifying the target population.

If researchers deliberately define their population as adults aged 18 to 30 with a particular diagnosis, excluding people outside that age range does not make the sample unrepresentative of that narrowly defined population merely because older adults also experience the diagnosis.

The problem arises if the resulting findings are later discussed as though they apply to everyone with the diagnosis.

This is why representation and representativeness should always be evaluated in relation to the population and inference being claimed.

Criteria Can Quietly Redefine a Broad Target Population

Suppose researchers say they want to study adults with a chronic condition. Their criteria then exclude adults above age 70, people taking common medications, people with several coexisting conditions, people unable to travel independently, and people who do not speak the study team's language.

The operational study population is no longer simply “adults with the condition.” It is a substantially narrower subset.

That may still be scientifically justified. Early-stage efficacy or mechanistic research, for example, can require tightly controlled populations. But researchers should describe the resulting population accurately and avoid allowing a broad label to conceal the restrictions built into eligibility.

Safety Exclusions Should Be Based on Actual Risk

Participant protection is a strong justification for exclusion when a study poses a risk that cannot be adequately managed for particular individuals or groups.

However, researchers should distinguish evidence-based safety considerations from precaution that has become habitual. Excluding an entire population because its participation is assumed to be complicated can leave persistent evidence gaps if the actual risks and possible safeguards are never examined.

CIOMS guidance on equitable selection specifically states that exclusion of groups needing special protection should be justified and cautions that categorical exclusion can contribute to or worsen health disparities. The guidance also states that inclusion and exclusion criteria should not rely on potentially discriminatory characteristics such as race, ethnicity, economic status, age, or sex without sound scientific or ethical justification.

This principle does not require inclusion regardless of risk. It requires the reason for exclusion to withstand scrutiny.

Convenience Can Masquerade as Eligibility

Some criteria make research considerably easier: participants must live nearby, attend appointments during office hours, use a particular technology, speak the researchers' language, have no common comorbidities, or be available for an intensive schedule.

Sometimes these requirements are unavoidable. At other times, they reflect how the study has been organized rather than characteristics necessary to answer the research question.

This distinction matters because study procedures can unintentionally exclude participants even when formal eligibility criteria appear reasonable.

A useful test is to ask: if the study procedure changed, would the person still need to be excluded? If the answer is no, the restriction may be operational rather than intrinsic to the target population.

Common Exclusions Can Produce an Unusually Healthy or Advantaged Sample

Research sometimes excludes participants with multiple conditions, medication use, disability, limited literacy, unstable housing, limited internet access, pregnancy, advanced age, or other circumstances that complicate study procedures or analysis.

Individually, each criterion may have a rationale. Collectively, however, they can produce a sample quite different from the population that will eventually encounter the intervention, policy, service, or phenomenon in ordinary settings.

This is particularly important when characteristics used for exclusion are common in the real-world population. The cleaner the study population becomes, the more carefully researchers may need to describe the population to which the findings directly pertain.

Restricting Heterogeneity Can Help One Objective While Hurting Another

Eligibility decisions often involve a genuine methodological trade-off. A narrower population can reduce certain sources of variation and make a specific causal or mechanistic question easier to investigate. A broader population can provide evidence across a wider range of participants and conditions.

The appropriate balance depends on what the study is trying to establish. There is no universal rule that broader eligibility is always methodologically superior.

Eligibility Criteria Are Only One Source of an Unrepresentative Sample

Even perfectly justified criteria do not guarantee a representative sample. Recruitment channels, nonresponse, accessibility, location, scheduling, participant burden, withdrawal, and other selection processes can alter who actually participates.

Conversely, an unrepresentative sample does not prove that the eligibility criteria caused the problem.

Researchers should therefore distinguish the eligible population from the recruited sample and the final analyzed sample. Different selection mechanisms operate at each stage.

At every transition, people can disappear from the evidence for different reasons.

A Diverse Sample Is Not a Substitute for Reviewing Eligibility Criteria

A final sample may appear demographically diverse while still excluding important segments of the target population. Conversely, a deliberately narrow study can be scientifically appropriate without attempting broad demographic diversity.

This is why making a sample more diverse does not automatically make it representative. Researchers need to evaluate the selection process as well as the composition of the final dataset.

Ethical Selection Requires More Than Avoiding Harm

The Belmont Report connects the principle of justice with fair procedures and outcomes in participant selection. CIOMS similarly emphasizes equitable distribution of research benefits and burdens and selection for scientific rather than arbitrary or convenient reasons.

These principles raise two concerns. Researchers should avoid disproportionately placing research burdens on populations simply because they are easy to recruit, and they should also scrutinize exclusions that unnecessarily prevent relevant populations from contributing to or potentially benefiting from research.

How Several Reasonable-Sounding Criteria Can Add Up to a Narrow Sample

The point is not that every original criterion was inappropriate. The audit reveals which restrictions follow from the science and which follow from procedures that can potentially be redesigned.

Common Mistakes When Defining Who Is Eligible

Audit Every Criterion Against the Population You Want to Understand

Before recruitment begins, place the target population beside the eligibility criteria and examine what happens to that population as each criterion is applied.

The central design problem is ultimately one of balancing inclusion with a clearly defined study population. Broad eligibility can increase relevance across heterogeneous populations, while narrower eligibility can sometimes be essential to answer a precise question. Neither strategy is automatically superior.

Before Finalizing Eligibility Criteria, Check What Each One Removes

Questions About Eligibility Criteria and Representativeness

The Right Criteria Define the Study Without Quietly Redefining Its Claims

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.