03 · What You Need to Know
Changing the Level Changes What Variation the Relationship Represents
Start with an individual-level relationship
Suppose faculty members are nested within universities.
At the individual level, researchers ask:
Within universities, are faculty members with greater AI competence more likely to adopt AI?
This compares one faculty member with other faculty members.
If the relationship is positive, individuals who are more competent than others tend to show greater adoption under the specified model.
Now move to the university level
The university-level question is:
Do universities with higher average faculty AI competence have higher average AI adoption?
This compares universities.
The predictor is no longer simply one person's competence. It is a university-level aggregate.
The outcome may also be an institutional average or rate.
The relationship therefore uses a different source of variation.
Within-group relationship
Compares lower-level units relative to others in the same group.
Between-group relationship
Compares groups using group means or other higher-level characteristics.
There is no mathematical requirement that the two coefficients match
The within-group association might be:
strongly positive
while the between-group association is:
weakly positive
zero
or:
negative.
These possibilities are not inherently contradictory because the coefficients answer different questions.
An intuitive example shows why
Imagine two universities.
Within each university, faculty with greater AI competence use AI more often.
However:
University A has highly competent faculty but a restrictive AI policy.
University B has less competent faculty but strong infrastructure, incentives, and institutional requirements for AI use.
Within both universities:
Higher competence → More adoption
But when comparing university averages:
Higher average competence could coexist with lower average adoption.
The higher-level institutional environment changes the between-university pattern.
This does not mean one result is wrong
Researchers may be uncomfortable when the coefficients differ because they assume there must be one “true” relationship between X and Y.
But the level is part of the definition of the relationship.
“How do people differ within organizations?” and “How do organizations differ from one another?” are different scientific questions.
Both relationships can be correctly estimated and still differ.
Raw individual regression can mix within- and between-group information
Suppose X varies both among individuals within universities and across university means.
A simple regression using raw X without appropriately representing the group structure can blend those sources of variation.
The coefficient may then fail to correspond cleanly to either:
the within-university relationship
or:
the between-university relationship.
Multilevel methods can separate them explicitly.
Group-mean centering isolates within-group variation
Suppose Xᵢⱼ is faculty competence.
Researchers can calculate:
This removes between-university variation from the centered individual predictor.
The associated coefficient can then describe the within-university relationship under the specified model.
The group mean provides the between-group component
The university mean:
X̄ⱼ
can be entered separately.
Its coefficient captures how differences among university average values relate to the outcome under the model.
Methodological discussions of group-mean centering emphasize that including these components allows researchers to ask richer multilevel questions rather than forcing within- and between-group relationships into one homogeneous coefficient.
A simple model can show the decomposition
That pattern may appear paradoxical only if the two coefficients are incorrectly assumed to represent the same relationship.
The difference between within and between effects can itself be informative
If the within-group and between-group relationships differ, researchers should ask why.
Possible explanations include:
- contextual influences;
- group composition;
- different confounding structures at different levels;
- selection into groups;
- measurement differences;
- nonlinear relationships;
- different causal processes.
The discrepancy can reveal scientific structure rather than merely a statistical nuisance.
A contextual effect can be defined through this difference
In some multilevel formulations, a contextual effect is represented by the difference between the between-group and within-group coefficients for a corresponding predictor.
Interpretation depends on how the model is parameterized and what X̄ⱼ represents theoretically.
The calculation should therefore not be separated from the substantive contextual question.
The group mean may represent composition rather than context
Suppose University A has a higher average faculty age than University B.
The university mean age describes the composition of the institution.
It does not necessarily represent an environmental property in the same sense as institutional policy or climate.
Researchers should be precise about whether a group mean is being interpreted as composition, context, or a shared collective construct.
Aggregation can produce dramatically different correlations
When individual data are averaged within groups, both the numerator and denominator of the statistical relationship can change because within-group variability disappears.
The aggregate correlation therefore need not resemble the individual correlation.
This is not an error introduced by correlation itself. It reflects that the two analyses are describing different distributions.
Group-level relationships often appear stronger, but that is not a universal rule
Aggregation can reduce individual measurement noise and within-group heterogeneity, which sometimes produces stronger correlations among group means.
But group-level relationships can also weaken or reverse.
Researchers should therefore not assume that aggregation will systematically strengthen or weaken an association.
Aggregation reduces the number of cases
Suppose 3,000 students are nested within 50 schools.
An individual analysis may use information from 3,000 students while recognizing the clustering.
A fully aggregated school analysis has 50 school-level cases.
Even if the school-level association appears numerically large, its uncertainty depends on the 50 schools rather than 3,000 independent schools.
Strong group-level correlations can therefore still be imprecise
A correlation of.60 across eight organizations may look impressive.
But it is based on very little higher-level information.
Effect size and precision should be interpreted together.
The ecological fallacy occurs when group findings are transferred downward
Suppose countries with higher average education levels have lower crime rates.
It does not automatically follow that individuals with more education are less likely to commit crimes.
The aggregate association concerns countries.
Inferring the individual relationship from that aggregate result is the ecological fallacy.
The core problem is a level-of-inference mismatch.
The atomistic fallacy moves in the opposite direction
Suppose individuals with higher income report greater technology adoption.
It does not automatically follow that higher-income communities have greater aggregate adoption.
Using an individual-level relationship to infer a group-level relationship can produce the atomistic fallacy.
| Inference error |
Incorrect movement |
|
Ecological fallacy
|
Group-level finding → individual-level conclusion |
|
Atomistic fallacy
|
Individual-level finding → group-level conclusion |
Neither fallacy means aggregate analysis is inherently inferior
Group-level analysis is entirely appropriate when the scientific question concerns groups.
If researchers want to know whether countries with higher research investment produce more publications, the country-level relationship is directly relevant.
The mistake would be interpreting that result as evidence that individual researchers receiving more funding necessarily publish more.
The problem is inappropriate transfer across levels, not aggregation itself.
The same principle applies to individual analysis
Individual-level studies are not inherently superior merely because they avoid aggregation.
If the substantive question concerns school systems or organizations, an exclusively individual-level analysis may fail to address the actual target of inquiry.
The appropriate level follows from the research question.
Group-level factors can explain why the relationships differ
Return to faculty competence and AI adoption.
Within universities, more competent faculty may adopt AI more frequently.
Across universities, however, policy and infrastructure differ.
These higher-level variables can shift university averages enough to produce a different between-university association.
Context can therefore help explain why within- and between-group coefficients diverge.
Selection into groups can also produce differences
People do not always enter groups randomly.
High-performing employees may select into demanding organizations. Wealthier families may choose particular neighborhoods. High-achieving students may attend selective schools.
Such sorting can make between-group comparisons reflect both contextual effects and compositional differences.
The resulting group-level relationship may therefore differ from the within-group relationship even without a direct contextual effect.
Different confounders may operate at different levels
An individual relationship may be confounded by personal characteristics.
A group-level relationship may be confounded by institutional, geographic, economic, or policy factors.
Because the confounding structures differ, coefficients at different levels need not coincide.
Within- and between-group modeling has been used precisely to reveal when an overall association conceals different relationships across levels.
Nonlinearity can create additional differences
Suppose the individual relationship between X and Y is curved rather than linear.
Group averages taken from different parts of that curve can produce an aggregate relationship that differs from a simple individual-level linear coefficient.
Researchers should therefore avoid assuming that all cross-level discrepancies must be caused by one phenomenon such as ecological bias.
Restricted ranges can differ across levels
Within one university, faculty competence may vary only slightly.
Across universities, average competence may vary substantially.
In another study, the reverse may be true.
The variance available at each level affects what relationship can be estimated.
Measurement reliability can differ across levels
Individual survey scores contain individual measurement error.
Averages based on multiple respondents may sometimes be more reliable estimates of a group mean, although the reliability depends on group size and the measurement structure.
Differences in measurement reliability can contribute to differences in estimated relationships.
This is another reason the process of aggregating individual data to the group level should be treated as more than arithmetic.
The group-level construct may not even be the same construct
Individual efficacy and collective efficacy are not merely the same variable at two resolutions.
Personal support and organizational support climate may also differ theoretically.
If constructs change meaning across levels, expecting identical coefficients is even less reasonable.
Between-person and within-person relationships create the same problem over time
This multilevel principle is not restricted to people nested in organizations.
Suppose people who generally sleep more are generally less stressed.
That is a between-person relationship.
It does not automatically follow that when a particular person sleeps more than usual, that person will experience less stress than usual the next day.
The latter is a within-person relationship.
Between-person and within-person associations can differ just as individual- and group-level relationships can.
Person-mean centering can separate those longitudinal effects
Suppose sleep varies daily.
A person's average sleep represents between-person information.
Daily deviation from their own average represents within-person information.
These can be modeled separately.
Methodological work on longitudinal models has emphasized that failing to disaggregate within-person and between-person effects can produce consequential interpretive errors.
A pooled coefficient can therefore answer neither question cleanly
If within- and between-level effects differ, a model that blends both can produce a coefficient somewhere between them.
Researchers might then describe it as “the effect of X” even though it does not correspond precisely to either the within-level or between-level process of theoretical interest.
This is why identifying the relevant level of analysis should come before coefficient interpretation.
Simpson's paradox is related but should not be used as a synonym for every level difference
Simpson's paradox refers to situations in which an association observed within subgroups differs substantially, sometimes reversing, when groups are combined.
It illustrates how conditioning and aggregation can change associations.
But not every difference between within- and between-group coefficients is properly described as Simpson's paradox.
The broader principle is simply that relationships depend on which variation and conditioning structure are being analyzed.
Different levels can even imply different interventions
Suppose individual self-efficacy strongly predicts adoption within universities.
An individual-focused intervention might therefore provide confidence-building training.
But suppose between universities, infrastructure is the stronger determinant of adoption.
An organizational intervention might focus on systems, policy, and access.
Multilevel differences can therefore have practical consequences for where interventions are targeted.
A group-level intervention should not be justified only by an individual-level coefficient
If the evidence only shows that individual confidence predicts individual behavior, it does not establish that increasing the average confidence of an institution will produce the corresponding organizational change.
The intervention itself may operate at a different level from the observed relationship.
This is another reason cross-level causal claims require explicit theory.
Cross-level relationships can explain why levels differ
Suppose university infrastructure changes how strongly individual competence predicts adoption.
Then individual-level slopes themselves differ according to higher-level context.
This is a cross-level interaction.
The question is no longer simply whether the average X–Y relationship differs at two levels, but whether context systematically alters the lower-level relationship.
Do not compare significance labels instead of coefficients
Suppose the individual-level coefficient is statistically significant and the group-level coefficient is not.
That does not by itself establish that the coefficients are statistically different.
The reverse is also true.
If the scientific question concerns whether relationships differ across levels, the contrast should be examined directly under an appropriate model rather than inferred from one p-value crossing.05 and another not doing so.
The level-specific question should be stated explicitly
Instead of asking:
Is X related to Y?
ask:
Within groups, are individuals with greater X more likely to have greater Y?
and, if relevant:
Across groups, do groups with higher average X have higher average Y?
The two questions immediately reveal why the coefficients need not be identical.
A strong multilevel study can estimate both
Rather than choosing one level and assuming the other behaves similarly, researchers can model the within- and between-group components directly.
This allows the data to reveal whether:
- the relationships are similar;
- the group-level relationship is stronger;
- the individual-level relationship is stronger;
- one exists while the other does not;
- the signs differ.
Any of these patterns can be scientifically meaningful.
