01 · The Question
Why Should It Matter Who Makes Up a Research Sample?
Suppose a study reports that an educational intervention improves student performance, a health program changes behavior, or a new technology is easy to use. Before applying that conclusion broadly, there is another question worth asking: Who actually participated in the study?
People can differ in ways that affect exposure, experience, access, response, and outcomes. If a sample captures only a narrow portion of the population relevant to the research question, some of that variation may never enter the evidence.
This is why sample diversity can matter scientifically as well as ethically. Yet “more diverse” is not a sufficient research objective by itself. Researchers still need to ask which forms of diversity matter for the phenomenon being studied, why they matter, and what the study is designed to infer.
03 · What You Need to Know
Diversity Matters When Human Differences Matter to the Research
A research sample is diverse when its participants vary across characteristics relevant to the study. Those characteristics might include age, sex, gender, race or ethnicity, socioeconomic circumstances, disability, language, geography, educational background, occupation, cultural context, or other dimensions. What counts as meaningful diversity therefore depends on the research question.
A study does not become scientifically better merely by accumulating demographic categories. Diversity has analytical value when variation among participants helps researchers understand the phenomenon they are studying or appropriately reflects the population about which they intend to draw conclusions.
Diversity Can Reveal Differences Hidden by a Narrow Sample
Research frequently reports an average result. Yet an average can combine people whose experiences or responses differ substantially.
Consider an intervention that appears moderately effective overall. The same intervention might work differently across age groups, contexts, baseline levels of need, or other relevant characteristics. If the study includes little variation on a characteristic that plausibly influences the outcome, researchers may have limited ability to investigate that heterogeneity.
This does not mean that every study must test every possible subgroup difference. Such analyses require theoretical justification, suitable measurement, and adequate statistical power. It means that a narrow participant pool can constrain which forms of variation the study is capable of observing in the first place.
Diversity Can Affect How Relevant the Evidence Is to the Population of Interest
Researchers often want their findings to inform people beyond the individuals who participated. The plausibility of doing so depends partly on how the study sample relates to the population and settings to which the conclusions will be applied.
If a population is heterogeneous in ways relevant to the outcome, repeatedly studying only a narrow segment can leave uncertainty about everyone else. Evidence obtained primarily from highly connected urban participants, for example, may not tell researchers enough about populations experiencing different infrastructure or access conditions when those conditions influence the phenomenon being studied.
This is closely connected to who receives a realistic opportunity to enter a study. Diversity in the final sample can be shaped long before analysis begins through eligibility criteria, recruitment channels, accessibility, language, scheduling, and willingness or ability to participate.
Relevant Diversity Is Not Always Demographic Diversity
Age, sex, gender, race, ethnicity, and other demographic variables are often the most visible dimensions of sample diversity. They are not automatically the variables most important to every research question.
For a study of technology adoption, relevant variation might include digital access, prior experience, institutional resources, or frequency of technology use. For an educational study, variation in prior achievement, school context, instructional language, or socioeconomic circumstances may matter. In qualitative research, diversity of experiences or positions relative to the phenomenon may sometimes be more consequential than reproducing a population's demographic proportions.
Researchers should therefore resist treating demographic diversity as a generic checklist. The better question is: Which differences could plausibly change what we observe or whose experience the study captures?
Diversity Can Help Expose the Boundaries of a Finding
Sometimes diversity matters not because researchers expect everyone to respond differently, but because they do not yet know whether a finding persists across relevant variation.
A sample containing meaningful heterogeneity may allow researchers to observe whether a pattern appears relatively consistent or whether important differences emerge. In some designs, this can support investigation of effect modification or subgroup differences. In others, particularly qualitative studies, it may reveal contrasting experiences that complicate an initially simple explanation.
There is an important statistical qualification. Having several groups represented does not guarantee that the study contains enough participants in each group to estimate differences reliably. Subgroup analyses that are underpowered, numerous, or devised after seeing the data can produce unstable or misleading conclusions.
Repeatedly Narrow Samples Can Create Gaps in the Evidence Base
The consequences of limited diversity extend beyond a single study. When particular populations are consistently absent or sparsely included across a body of research, much less may be known about whether established findings apply to them.
This is one reason underrepresentation in research can become consequential. The concern is not simply that a demographic table looks imbalanced. It is that systematic gaps in participation can become systematic gaps in knowledge.
In health research, for example, the U.S. National Institutes of Health requires inclusion of women and racial and ethnic minority groups in NIH-defined clinical research unless there is a compelling justification for exclusion. NIH policy explicitly connects inclusion with the need to ensure that research findings can address populations affected by the condition under study. These requirements are specific to their policy context and should not be treated as universal rules for all research.
Diversity and Fairness Can Be Related, but They Are Not Identical
There is also an ethical dimension. If a population bears the burden of a problem or is expected to benefit from research-informed policies, services, treatments, or technologies, its systematic absence from the evidence deserves scrutiny.
The Belmont Report's principle of justice directs attention to fair participant selection and to the distribution of research burdens and benefits. CIOMS similarly addresses equitable distribution of research benefits and burdens. These principles do not establish a simple numerical formula for a “diverse enough” sample. They do, however, make participant composition more than a cosmetic concern.
Watch Out
Do not assume that demographic categories are natural explanations for observed differences. Characteristics such as race and ethnicity may be associated with social, historical, environmental, structural, or other conditions rather than functioning as simple biological variables. Researchers need a defensible reason for how such variables are defined, measured, analyzed, and interpreted.
This issue becomes especially important when researchers use race and ethnicity as research variables or other demographic classifications whose meanings depend heavily on context.
A Diverse Sample Is Not Necessarily a Representative Sample
This distinction is essential. Imagine a target population that is 80% Group A and 20% Group B. A researcher deliberately recruits 50 participants from each group. The resulting sample is visibly diverse and may be extremely useful for comparing the two groups. Yet its composition does not mirror the target population.
That is not necessarily a methodological flaw. Researchers sometimes intentionally recruit groups disproportionately because doing so serves an analytical purpose. The problem arises when diversity is presented as proof that the sample represents the population.
Diversity
The sample contains variation across characteristics relevant to the research.
Representativeness
The sample adequately reflects a defined target population for the inference researchers intend to make.
The conceptual difference is important enough that researchers should examine representation and representativeness separately rather than using the terms as synonyms.
04 · A Practical Example
How Sample Composition Can Change What a Study Is Able to See
Hypothetical Example
Evaluating a university's new digital learning platform
A university evaluates a new learning platform by recruiting 300 students. Most participants study full-time on campus, own personal laptops, have reliable home internet access, and already use digital learning tools regularly.
The study could still provide valid evidence about those participants. Suppose, however, that the university intends to use the findings to claim that the platform works well for its entire student population.
Question Does the platform support students across the university population?
Potentially relevant variation Students differ in connectivity, device access, digital experience, study mode, accessibility needs, and other circumstances that could affect platform use.
Narrow sample If most participants have similar technological circumstances, the study may reveal little about barriers experienced under different conditions.
More diverse sample Including participants with meaningfully different access conditions gives the research an opportunity to observe experiences that were previously missing.
Interpretation Greater diversity broadens what can be observed, but the researchers must still determine whether the sampling design supports claims about the whole university population.
The example illustrates why diversity should be tied to the phenomenon. Adding variation that has little relationship to platform use may contribute little to the research question, while overlooking digital access could conceal an important source of different experiences.
06 · What This Means for You
Decide Which Diversity Matters Before You Recruit
The most useful time to think about diversity is during study design, not after a demographic table reveals who happened to participate.
Begin with the research question and target population. Identify characteristics that theory, previous evidence, context, or the intended application suggests could affect the phenomenon. Then ask whether the sampling and recruitment strategy gives relevant variation a reasonable chance of appearing in the study.
A simple decision framework
If a characteristic could plausibly affect the phenomenon or intervention
Consider whether the study needs sufficient variation on that characteristic to investigate or acknowledge it.
If the study intends to generalize to a heterogeneous population
Evaluate whether the sampling design and achieved sample adequately support that intended inference rather than relying on diversity alone.
If an important population is difficult to recruit
Examine barriers to recruitment and participation before assuming low enrollment reflects low willingness to participate.
If an analytically important group would otherwise be too small
If diversity is not relevant to the central inference
Do not manufacture demographic comparisons merely to make the study appear more inclusive; justify the population actually required by the question.
Finally, describe the achieved sample transparently and calibrate conclusions accordingly. Diversity can increase what a study is capable of observing. It cannot repair every limitation in sampling, measurement, analysis, or design.
07 · A Quick Checklist
Before Claiming That Your Sample Is Sufficiently Diverse, Check Why
When planning sample diversity, check:
Define the population your research question is intended to address.
Identify which participant differences are plausibly relevant to the phenomenon rather than selecting demographic variables by habit.
Review prior evidence for populations or experiences that may have been systematically understudied.
Examine whether eligibility criteria and recruitment procedures create avoidable barriers for relevant participants.
Determine whether important subgroup analyses require specific recruitment targets or larger subgroup sample sizes.
Do not use demographic diversity as evidence of population representativeness without evaluating the sampling design and target population.
Report relevant sample characteristics and recruitment procedures transparently.
Limit claims about populations for whom the study provides insufficient evidence.
