What Happens When a Research Question Mixes Individual-Level and Group-Level Explanations?

What Is Wrong With Asking Whether “Institutional Support Influences Faculty Adoption”?

The question sounds reasonable:

Does institutional support influence faculty adoption of generative AI?

But what exactly is “institutional support”?

Is it each faculty member's personal perception of support?

Is it a shared university support climate?

Is it an objectively measured institutional policy or resource?

And what does “faculty adoption” mean?

Is it one faculty member's behavior, the percentage of faculty adopting AI, or the university's institutional adoption level?

Depending on those answers, the question could be entirely individual-level, entirely organizational-level, or explicitly cross-level.

Problems arise when the research question moves between those levels without saying so. The variables may then be measured at one level, analyzed at another, and interpreted at yet another.

A mixed-level question is not inherently wrong. Many important scientific questions are genuinely multilevel. The solution is to make the levels explicit and design the study accordingly.

Mixing Levels Is Valid Only When the Cross-Level Logic Is Explicit

Before Mixing Levels, Identify What Each Variable Actually Describes

Start by assigning a level to every construct

Suppose researchers want to study faculty AI adoption.

Potential variables include:

Simply placing these variables in one conceptual framework does not make them all equivalent.

The distinction between individual-level and group-level variables is part of the substantive meaning of the model.

A purely individual-level question is straightforward

Consider:

Do faculty members who perceive greater personal support report greater individual AI adoption?

Both variables belong to faculty members.

The comparison is among individuals.

Even if faculty are nested inside universities and clustering must be handled statistically, the substantive relationship itself is individual-level.

A purely group-level question is also straightforward

Consider:

Do universities with stronger AI-support climates have higher institutional AI adoption rates?

Both variables belong to universities.

The comparison is among universities.

Faculty may provide observations used to construct the climate measure, but the substantive relationship is organizational-level.

A cross-level question deliberately combines the two

Now consider:

Do faculty members working in universities with stronger AI-support climates report greater individual AI adoption?

The predictor is university-level.

The outcome is faculty-level.

This is a legitimate cross-level question.

The nouns in the question often reveal the level

Compare:

Faculty members with greater self-efficacy...

with:

Universities with greater average self-efficacy...

The first refers to people.

The second refers to organizations.

Similarly:

Students who perceive supportive teachers...

differs from:

Classrooms characterized by supportive teaching climates...

A well-written multilevel research question should make these entities visible.

The same word can conceal a level mismatch

Terms such as support, performance, adoption, efficacy, engagement, and climate can exist at several levels.

For example, “performance” could mean:

• individual employee performance;
• team performance;
• department performance;
• organizational performance.

The variable name alone does not identify the level.

Measurement should match the level implied by the question

If the question concerns individual perceptions, an item such as:

“I receive adequate support for using AI.”

may be appropriate.

If the construct is organizational support climate, items may instead use a shared referent:

“Faculty in this university receive adequate support for using AI.”

That does not automatically validate the group construct, but it aligns measurement with the intended referent.

This is especially important when individuals provide data for a group-level research question.

Do not solve a conceptual level mismatch merely by averaging

Suppose researchers ask faculty:

“I am satisfied with the support I personally receive.”

They then average responses by university and rename the mean:

Institutional support climate.

The arithmetic does not establish that the underlying individual measure represents a shared organizational climate.

Aggregation requires a theory of how lower-level responses compose into the higher-level construct.

A group mean can be a legitimate contextual variable

Suppose individual digital competence is measured for each employee.

The organizational average can be included as a separate group-level variable.

The study can then ask two questions:

Individual effect: Are more competent employees more likely to adopt AI?

Contextual effect: Are employees in organizations with generally more competent colleagues more likely to adopt AI, beyond their own competence?

This is a multilevel decomposition rather than an accidental mixing of levels.

Within-group and between-group relationships should be separated

Suppose X is individual workload and Y is burnout.

The raw relationship between X and Y may combine:

Within organizations: whether employees working more than their colleagues are more burned out.

and:

Between organizations: whether organizations with higher average workloads have higher average burnout.

These relationships can differ.

Multilevel modeling or appropriate within-between specifications can estimate them separately.

Group-mean centering can make the within-group question explicit

The university mean can then be included separately as a higher-level predictor.

This prevents the raw individual coefficient from silently combining different sources of variation.

A contextual effect is not simply a group-level correlation

Suppose university-average digital competence predicts an individual's AI adoption even after the individual's own competence is represented.

This can be described as a contextual effect under an appropriate model.

Conceptually, it asks whether being located in a higher-competence environment matters above and beyond the person's own competence.

This differs from merely correlating university-average competence with university-average adoption.

Cross-level moderation asks whether context changes an individual relationship

Suppose self-efficacy predicts faculty AI adoption more strongly in universities with robust infrastructure.

The individual-level relationship is:

Self-efficacy → Adoption

The organizational-level moderator is:

University infrastructure

The research question becomes:

Does university infrastructure change the strength of the individual self-efficacy–adoption relationship?

This is a genuine cross-level relationship.

Merely placing a higher-level variable in an individual regression does not solve everything

Suppose every faculty member within University A receives the same infrastructure score.

An ordinary regression can technically include that variable alongside individual predictors.

But the repeated values are not independent observations of university infrastructure.

Faculty within a university remain clustered, and the uncertainty for higher-level effects depends strongly on the number of universities rather than simply the total number of faculty.

The number of higher-level units matters for higher-level claims

Suppose researchers have 5,000 faculty members from six universities.

There is extensive individual information.

But an organizational-level predictor has only six independent institutional values.

Mixed-level questions therefore affect sampling

If the research question involves both faculty and universities, the sample must be planned at both levels.

Researchers need to consider:

• number of faculty per university;
• number of universities;
• variation in university characteristics;
• balance or imbalance in cluster sizes;
• power for individual, group, and cross-level effects.

One total N is not sufficient for describing the design.

The hierarchy should be reflected in the analysis

Mixed-level research often produces nested or hierarchical data.

If faculty are nested in universities, observations from the same university may be correlated.

A multilevel model can represent:

• within-university variation;
• between-university variation;
• individual predictors;
• university predictors;
• cross-level interactions.

Other approaches may also be appropriate depending on the estimand, but the hierarchy should not simply be ignored.

The level of the outcome is particularly important

Suppose the predictor is university policy.

If the outcome is:

individual faculty adoption

the relationship is cross-level.

If the outcome is:

university adoption rate

both variables are organizational-level.

The same predictor therefore participates in different kinds of research questions depending on the outcome.

Mixing levels can create an ecological fallacy

Suppose researchers find:

Universities with greater average faculty self-efficacy have greater institutional AI adoption.

They then conclude:

Faculty members with greater self-efficacy are more likely to adopt AI.

That individual conclusion does not follow automatically from the university-level association.

The move from higher-level evidence to lower-level inference is the ecological fallacy.

The atomistic fallacy moves upward instead

Suppose researchers establish at the individual level that faculty with greater self-efficacy report greater adoption.

They then conclude:

Universities with more self-efficacious faculty will necessarily have stronger institutional AI adoption.

That conclusion moves from individual relationships to an organizational relationship without directly estimating it.

This is the atomistic or individualistic fallacy.

Rigorous multilevel research therefore keeps conclusions at the appropriate level.

These two errors are sometimes called cross-level fallacies

Recognizing them is essential whenever a conceptual framework contains variables from more than one level.

A relationship can genuinely differ across levels

One reason cross-level inference is dangerous is that the same variable pair can produce different relationships at different levels.

For example, among employees within organizations, individual autonomy might be positively related to satisfaction.

Across organizations, average autonomy could have a weaker, stronger, or different relationship with average satisfaction.

The two effects compare different sources of variation.

Group context can modify individual relationships

Another reason the levels differ is that the higher-level group itself can modify a lower-level association.

Suppose personal technology competence predicts adoption only where organizational infrastructure allows competent employees to act.

Then the individual competence–adoption relationship is conditional on context.

A single pooled individual coefficient can conceal that heterogeneity.

The conceptual framework should label levels visibly

For multilevel research, it can help to arrange the framework into sections:

Individual level

Faculty self-efficacy → Faculty adoption

University level

Institutional infrastructure

Cross-level pathway

Infrastructure → Faculty adoption

Cross-level moderation

Infrastructure changes the self-efficacy → adoption relationship

This makes the theoretical structure visible before statistical analysis begins.

Do not use arrows that hide the unit being affected

An arrow labeled:

Organizational culture → performance

is ambiguous.

Does culture affect:

• individual employee performance?
• team performance?
• organization-wide performance?

Those are different theoretical propositions.

The framework and accompanying text should identify the target explicitly.

Cross-level causality requires causal reasoning at both levels

Suppose universities with formal AI policy have faculty who adopt AI more frequently.

A causal interpretation still requires researchers to consider why some universities adopt policies, what other institutional characteristics influence faculty adoption, whether faculty composition differs across universities, and whether the policy precedes the behavior.

Multilevel modeling can represent hierarchy, but it does not automatically solve confounding.

Statistical control does not repair a badly defined level

If “institutional climate” is actually measured as a person's private satisfaction score, adding university fixed effects or random effects does not transform it into an organizational construct.

Statistical hierarchy and construct hierarchy must both be defensible.

Qualitative research can also mix levels

Suppose a qualitative study asks:

How do university AI policies shape faculty teaching practices?

The policy exists at the institutional level.

Teaching practices exist at the individual faculty level.

This is a cross-level research question even without regression or multilevel modeling.

The study should gather evidence at both levels and explain how institutional processes connect to individual experiences.

Mixed methods do not automatically make a mixed-level design

A survey and interviews can both focus entirely on individual faculty members.

Conversely, one quantitative dataset can support a multilevel design if it contains individuals nested in organizations and variables at multiple levels.

Number of methods and number of analytical levels are separate design decisions.

A multilevel question should identify the direction of the cross-level process

Compare:

Are institutional support and faculty adoption related?

with:

Do faculty members in universities with stronger support climates report greater individual AI adoption?

The second wording identifies:

• the higher-level context;
• the lower-level outcome;
• the direction of the hypothesized relationship.

It is therefore much easier to align with a multilevel design.

Some apparently mixed-level questions are actually individual-level questions in disguise

Suppose “institutional support” is measured using each faculty member's personal perception and no aggregation or group-level construct is intended.

Then:

Does perceived institutional support predict individual AI adoption?

may be an entirely individual-level question.

The word “institutional” in the variable name does not make the variable organizational-level.

This is why level comes from the construct definition and referent, not merely terminology.

Some apparently individual questions contain an implicit group explanation

Consider:

Why do faculty members in some universities adopt AI more than faculty members in others?

This wording signals between-university variation even though the outcome belongs to individual faculty members.

The question implicitly invites institutional explanations.

A multilevel design can make that structure explicit rather than treating university differences as unexplained noise.

Mixed-level questions can be scientifically richer when designed deliberately

Multilevel research can distinguish:

Who is more likely to adopt AI?

from:

Which institutional environments encourage adoption?

and from:

For whom does institutional context matter most?

These are complementary questions rather than competing ones.

The problem is not complexity. It is ambiguity about where the variables and conclusions live.

Turning an Ambiguous Question Into a Multilevel Research Model

The revised question is not merely more precise stylistically. It now tells the researcher what must be sampled, measured, aggregated, and modeled.

Common Mistakes When Research Questions Mix Levels

Rewrite the Question Until Every Variable Has an Identifiable Level

A useful test is to rewrite every research question using explicit nouns:

Which individuals?

Which groups?

What characteristic belongs to each?

At what level is the outcome?

If those questions are answered clearly, the multilevel structure usually becomes much easier to design.

Before Finalizing a Mixed-Level Research Question, Check This

Frequently Asked Questions About Mixed-Level Research Questions

Mixing Levels Is a Problem Only When the Study Fails to Admit That It Is Multilevel

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.