Should You Exclude Research From a Different Country or Population?

Does Research From Somewhere Else Still Belong in Your Review?

You are studying university students in the Philippines and find an excellent study from South Korea. Or your review concerns public-school teachers, but one paper includes teachers from private schools. Perhaps your target population is undergraduate students, while a potentially useful study combines undergraduate and postgraduate participants.

Should these studies be excluded because their populations are not exactly the same as yours?

Not necessarily. No study population will perfectly reproduce every characteristic of the population to which you eventually want to apply its findings. At the same time, some differences in geography, culture, age, institutional setting, socioeconomic conditions, disease characteristics, educational system, or other population features can materially change what the evidence means.

The task is therefore not to find studies conducted among people identical to your intended population. It is to determine which population and contextual differences are important enough to define the boundaries of the review, and which should instead be considered when interpreting the evidence.

Different Does Not Automatically Mean Ineligible

The Real Question Is Whether the Difference Matters to What You Are Trying to Learn

Country and Population Are Not Automatic Exclusion Criteria

A study conducted in another country does not become irrelevant merely by crossing a national border. Likewise, a study population does not necessarily need to match your target population on every characteristic.

Cochrane guidance states that population and setting criteria should be broad enough to encompass relevant diversity while remaining narrow enough to produce a meaningful answer when studies are considered together. Restrictions based on population characteristics or settings should have a sound rationale.

This is an important starting point because geographical restrictions are sometimes introduced almost reflexively:

“My study is about the Philippines, so I should include only Philippine studies.”

That may be appropriate for some questions. For others, it unnecessarily removes useful international evidence.

The relevant question is not simply where the study was conducted. Ask what characteristics of that location might affect the phenomenon you are reviewing.

Separate Eligibility From Applicability

Two questions are easily confused.

A study can be eligible yet not perfectly applicable to every setting in which its findings might later be used.

Cochrane describes applicability, generalizability, external validity, and transferability as related concepts concerned with how well evidence applies beyond the particular populations and circumstances in which studies were conducted. Some difference between study populations and target populations is unavoidable. The question is whether those differences are sufficiently important to alter interpretation.

Excluding every study that differs in some way from your local context can therefore be unnecessarily restrictive. Conversely, including highly dissimilar populations and then assuming their findings transfer without qualification can be equally problematic.

Ask What the Country Difference Actually Represents

Country is often a proxy for other characteristics rather than the causal characteristic itself.

Two studies conducted in different countries may operate under remarkably similar educational, clinical, technological, or institutional conditions. Two studies conducted in the same country may take place in contexts so different that treating them as equivalent would be questionable.

Consider a review of university students' experiences with online learning. A Philippine university and a South Korean university differ geographically, but both may use comparable learning-management systems, similar assessment practices, and high-speed internet infrastructure.

Meanwhile, two universities within the Philippines could differ substantially in:

• internet access and technological infrastructure;
• institutional resources;
• student socioeconomic characteristics;
• language of instruction;
• class size;
• teaching practices;
• institutional policies; or
• urban and rural context.

“Country” can therefore be too crude a variable to capture the contextual difference you actually care about.

Sometimes Geography Really Is Part of the Question

There are also many questions for which location is substantively important.

Suppose your review asks:

What barriers affect implementation of the Philippine Universal Health Care Act in public hospitals?

Research from Canada or Germany may provide useful comparative or theoretical insight, but it cannot directly investigate implementation of a Philippine law within Philippine institutions.

Likewise, a review of educational outcomes following implementation of a particular national curriculum may legitimately focus on jurisdictions where that curriculum applies.

JBI's scoping review guidance uses Population, Concept, and Context as core elements of inclusion criteria. When context is central to the review question, geographical, cultural, institutional, or other contextual boundaries may appropriately determine eligibility.

The correct geographical boundary therefore follows from the question rather than from a general preference for local or international research.

Population Differences Need the Same Kind of Reasoning

Population criteria can involve age, educational status, diagnosis, disease severity, sex, occupation, socioeconomic characteristics, or other features relevant to the question.

Cochrane recommends defining these characteristics prospectively and cautions that restrictions should have a sound rationale. The population should be sufficiently broad to capture relevant diversity while still allowing the review to answer a meaningful question.

Suppose your review concerns undergraduate students. A study of elementary-school pupils is probably too distant if educational level is central to the phenomenon. A study combining undergraduate and postgraduate students is more complicated.

The relevant questions might include:

• Are separate data available for the eligible subgroup?
• How much of the sample belongs to the target population?
• Is the population difference expected to change the phenomenon or effect?
• Did the protocol specify how mixed populations would be handled?

Cochrane specifically recommends prespecifying a strategy for studies that contain both eligible and ineligible participants.

Mixed Populations Require a Rule, Not an Impression

Suppose your review includes participants aged 18 years and older, but a study reports an age range of 16 to 25 years.

Automatically excluding the study may discard useful adult evidence. Automatically including it may introduce participants outside your intended population.

Possible strategies include:

• including only separately reported data for eligible participants;
• using a prespecified proportion rule for mixed samples;
• contacting authors for subgroup data where appropriate; or
• excluding studies when eligible data cannot be separated, if that is the rule established for the review.

There is no universal percentage that makes a mixed sample acceptable. The strategy should be appropriate to the review question and, for systematic reviews, ideally specified in advance.

This is another reason eligibility criteria need consistent operational rules. “Mostly the right population” is difficult to reproduce unless you define what “mostly” means.

Do Not Assume Demographic Differences Are Either Always Important or Never Important

Age, sex, ethnicity, socioeconomic position, educational level, disease severity, and other population characteristics may affect outcomes in some research questions and have little bearing on others.

Cochrane recommends considering whether population characteristics could modify intervention effects. When relevant subpopulations exist, reviewers may either narrow the review to exclude certain groups or maintain a broader scope and address subgroup differences analytically.

The important methodological move is to justify the choice.

For example, restricting a review of an intervention to adolescents might make sense when developmental differences plausibly affect how the intervention works. The same age restriction could be difficult to justify for another intervention whose mechanism is unlikely to depend meaningfully on age.

Demographic criteria should therefore follow substantive reasoning rather than assumptions about who seems sufficiently similar.

Cultural Differences May Matter Without Requiring Exclusion

Culture can shape beliefs, behavior, educational practices, healthcare use, communication, family roles, and responses to interventions. That makes cross-cultural differences potentially important.

But recognizing cultural variation does not require excluding all studies conducted elsewhere.

For qualitative evidence in particular, differences across contexts can sometimes be analytically valuable. They may reveal which experiences appear repeatedly across settings and which depend strongly on local conditions.

A broader review might intentionally include diverse cultural contexts and examine variation. A locally focused review might instead restrict context because transferability to one particular setting is central to the question.

The review purpose determines which approach is more appropriate.

Different Populations Can Sometimes Strengthen a Review

Variation is not always methodological noise.

Including studies across different countries, institutions, socioeconomic conditions, or demographic groups may reveal whether findings are robust across contexts or concentrated in particular circumstances.

For example, if students report similar concerns about generative AI feedback across universities in several countries, that consistency may be informative. If concerns differ markedly according to institutional policies or technological infrastructure, that variation may be even more informative.

A review designed to examine such heterogeneity should not eliminate it at the eligibility stage.

Broad eligibility does, however, place greater demands on synthesis. Researchers need to describe the populations and contexts accurately and avoid collapsing substantively different studies into a single undifferentiated conclusion.

Do Not Use Local Relevance as a Shortcut for Evidence Quality

A study conducted locally is not automatically better evidence than one conducted elsewhere.

Likewise, an international study is not automatically more rigorous because it comes from a prestigious institution or journal.

Country, relevance, methodological quality, and applicability are different dimensions. A methodologically weak local study may be highly applicable but uncertain. A rigorous international study may provide strong evidence but require caution when transferred to a substantially different context.

This echoes the broader distinction between relevance and methodological quality. Neither geography nor prestige can substitute for evaluating what the study actually contributes.

Be Careful With Country Restrictions Added for Convenience

A reviewer might restrict studies to one country because international screening seems difficult, because the resulting literature is smaller, or because local studies are easier to interpret.

Those are practical considerations, but they are not necessarily scientific justifications.

Cochrane emphasizes that restrictions on populations and settings should have a sound rationale and notes the importance of making reviews widely relevant. JBI similarly expects the reasons for including or excluding particular participants and contexts to be explained clearly.

Language and Country Are Different Eligibility Decisions

Geographical restrictions are sometimes unintentionally created through language restrictions.

For example, including only English-language publications may disproportionately exclude evidence from particular countries even when those countries themselves were not intended to be outside the scope.

Cochrane guidance states that language restrictions should not be imposed in Cochrane intervention reviews, although practical arrangements for translating or extracting information may still be required.

More generally, researchers should distinguish:

These decisions have different rationales and different implications for the evidence base.

Do Not Add a Geographical Restriction After Seeing the Results

Suppose your original protocol includes studies from any country. During screening, you notice that studies from one region produce different findings from studies conducted locally. You then decide that foreign studies are probably not applicable and exclude them.

That is a substantive eligibility change made after learning something about the evidence.

Cochrane guidance allows sensible post hoc eligibility decisions when unforeseen issues arise but states that changes should remain faithful to the objectives of the review and must not be driven by study findings or synthesis results.

If a genuine contextual problem becomes apparent, use the safeguards required when changing inclusion criteria after screening has begun. Do not simply remove the studies whose results complicate the story.

Sometimes the Better Solution Is Analysis Rather Than Exclusion

If a population characteristic or setting may modify findings but does not make the study irrelevant, consider whether the difference should be examined rather than eliminated.

Depending on the review and available data, this might involve:

• subgroup analysis;
• meta-regression;
• stratified synthesis;
• separate presentation by setting;
• qualitative comparison across contexts; or
• explicit discussion of indirectness and applicability.

Not every review will support these analyses, and subgroup investigations need methodological care. The broader point is that heterogeneity does not automatically require exclusion.

Sometimes the interesting research question is precisely why evidence differs across populations.

Applicability Should Be Discussed Even When Studies Are Eligible

Eligibility is a threshold decision. Applicability is often a matter of degree.

A study may satisfy your criteria but still differ from the population to whom you hope to apply the findings. Those differences should be acknowledged during interpretation.

Cochrane's guidance on interpreting results emphasizes examining how closely the body of evidence matches the population, intervention, comparator, and outcomes specified by the review question. Differences between the evidence and the target context may contribute to concerns about indirectness or applicability.

This gives you an alternative to over-restrictive screening:

Include evidence that legitimately answers the review question, then be appropriately cautious about where and to whom its conclusions can be generalized.

Your Review Type Changes How Context Is Defined

Different evidence-synthesis approaches conceptualize population and context somewhat differently.

For intervention reviews, PICO frameworks typically make population central, with setting incorporated where relevant. Cochrane emphasizes that population restrictions should be sufficiently precise for meaningful synthesis but broad enough to capture relevant diversity.

For scoping reviews, JBI recommends the PCC framework: Population, Concept, and Context. Context may include geographical location, cultural factors, or specific settings, depending on the review objective.

For qualitative evidence synthesis, context may be particularly important to transferability and interpretation because experiences and meanings can be closely connected to social and institutional environments.

Do not therefore copy a country or population restriction from another review simply because its eligibility table looks tidy. Define the boundary your own question requires.

Should a South Korean Study Be Included in a Review Focused on Filipino University Students?

Now change the question to: “What barriers do students encounter when complying with Philippine university policies on generative AI-generated coursework?”

The South Korean study may no longer provide direct evidence about the phenomenon as defined because Philippine institutional policy is now part of the question itself. It could remain useful comparative literature, but the reason for exclusion would be the specific policy context, not simply the fact that the study is foreign.

Common Mistakes When Judging Different Countries and Populations

Identify the Difference That Matters Instead of Using Country as a Shortcut

When a potentially relevant study comes from another country or population, ask what specific characteristic creates concern. “Different country” is often too broad to be analytically useful.

A useful test is to replace the country name with the characteristic you think actually matters. Instead of saying, “Studies from other countries may not apply,” try to complete the sentence:

“Studies conducted elsewhere may differ because...”

If the answer is a specific mechanism such as policy, curriculum, healthcare system, technological infrastructure, cultural practice, or socioeconomic environment, you now have something that can be evaluated. If the sentence ends simply with “because they are elsewhere,” the geographical restriction probably needs more thought.

Before Excluding a Study Because Its Country or Population Is Different

Questions About Countries, Populations, and Study Eligibility

Exclude a Different Population Only When the Difference Actually Matters

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.