03 · What You Need to Know
Measurement Changes Behavior Because Institutions Respond to What Counts
Rankings and Accreditation Do Not Have the Same Purpose
Rankings and accreditation should first be distinguished. Global university rankings compare institutions using selected indicators and weights. Accreditation and quality-assurance systems generally evaluate institutions or programs against specified standards, although terminology, procedures, and consequences vary among jurisdictions.
Research may appear in both systems, but for different reasons.
A ranking may use citations because they provide a scalable indicator of scholarly influence. A quality-assurance framework may examine research because an institution offers research degrees and therefore needs qualified supervisors, appropriate infrastructure, research-integrity systems, and credible research activity.
Combining the two into a generic category of “external requirements” can obscure these different functions.
External Evaluation Can Produce Genuine Improvement
Institutional response to external evaluation is not inherently undesirable. A university that discovers weak research capacity may invest in researcher development, laboratories, research administration, postgraduate programs, ethics systems, data infrastructure, or external partnerships.
Accreditation and quality assurance can encourage institutions to demonstrate that their research capabilities are consistent with what they claim to provide. Rankings can provide comparative evidence that prompts institutions to examine why their research visibility, citation performance, funding, or international collaboration differs from peers.
These responses can create benefits that extend beyond the external evaluation itself.
The Problem Begins When a Proxy Becomes the Target
Universities cannot measure broad concepts such as research excellence, societal contribution, or institutional quality directly with one number. Evaluation systems therefore rely on indicators and proxies.
A publication count can represent research output. Citation indicators can represent aspects of scholarly use or influence. Grant income can indicate success in attracting resources. Reputation surveys capture perceptions among specified groups.
Each provides information. None is identical to the underlying concept.
Institutional objective
The underlying outcome the university wants research to achieve, such as credible knowledge, researcher development, educational enrichment, societal contribution, or sustainable research capability.
Indicator
A measurable quantity or category used as evidence about some aspect of that objective.
Target
A desired level of performance on the indicator, often attached to planning, accountability, incentives, or evaluation.
Distortion becomes more likely when the target begins determining behavior independently of whether improving it still advances the underlying objective.
Rankings Can Favor Research That Is Globally Visible
Major global rankings rely partly on bibliometric data, academic reputation, and international research indicators. These measures can favor research that enters internationally visible scholarly networks.
This creates a potential tension for universities whose missions include regional or national priorities. Research addressing a local educational problem, endangered language, specialized community, national policy issue, or geographically specific environmental concern may have substantial value while attracting modest international citation attention.
If ranking performance becomes the dominant allocation criterion, such research may appear strategically unattractive despite its institutional or societal importance.
Database Visibility Can Influence Which Outputs Appear Valuable
Bibliometric ranking indicators depend on databases that do not represent every form of scholarship equally. Coverage varies across disciplines, languages, publication types, regions, and journals.
Universities seeking stronger bibliometric performance may consequently encourage researchers toward outputs and venues visible to the databases underlying rankings.
That can improve international discoverability. It can also reduce incentives for books, local-language scholarship, practitioner publications, community outputs, or other legitimate forms of research communication when those outputs receive less recognition within the measurement system.
Disciplinary Priorities Can Shift
Publication and citation patterns differ substantially across disciplines. Ranking organizations use normalization and subject-specific methodologies partly because of these differences.
Institutional incentives may nevertheless favor disciplines capable of producing the kinds of outputs that generate visible ranking returns quickly. Journal-intensive fields with large citation networks can appear more attractive than disciplines whose scholarship develops through books, archival work, creative outputs, local engagement, or slower publication cycles.
The danger is not that universities support internationally competitive science. The problem arises if one measurement model becomes an implicit judgment about which disciplines deserve institutional investment.
Accreditation Can Also Redirect Research Activity
Accreditation or regulatory requirements can influence institutional behavior when research forms part of the standards. Universities may establish research offices, require faculty research, expand postgraduate programs, or introduce output targets to demonstrate compliance.
Such responses can strengthen institutional research. Evidence from Ecuador, for example, has shown substantial growth in indexed research production following national evaluation and accreditation reforms that gave research greater prominence.
Yet increased output demonstrates behavioral response, not necessarily equivalent improvement in research quality. As explained in whether producing more research improves accreditation outcomes, institutions need to identify what the applicable standard actually requires rather than assuming that more publications solve every research-related quality problem.
Internal Requirements Can Become More Demanding Than External Ones
An interesting institutional phenomenon occurs when external criteria are translated internally. An accreditor may require evidence of active research appropriate to institutional mission. University leadership converts this into a publication expectation. Faculties convert the expectation into annual targets. Departments convert targets into individual quotas.
By the time the requirement reaches the researcher, it may be considerably more specific than the external standard that initiated it.
This is why institutions should trace internal research requirements back to their source. “Required for accreditation” should be treated as a verifiable claim, not an incantation capable of ending policy discussion.
Rankings Can Shift Resources Toward What Moves the Score
Research strategy involves scarce resources: faculty positions, research time, internal grants, laboratories, doctoral scholarships, travel, databases, research staff, incentives, and administrative support.
Once ranking performance becomes a strategic priority, resources may be allocated toward activities expected to improve highly weighted indicators. Universities may recruit highly cited researchers, incentivize publications in particular databases, support international collaboration, or concentrate resources in areas expected to generate stronger bibliometric performance.
Some of these investments can strengthen research. The concern is whether other valuable activities are displaced simply because their contribution to ranking indicators is less visible.
External Targets Can Change the Time Horizon of Research
Some worthwhile research takes years before substantial outputs appear. Longitudinal studies, difficult fieldwork, infrastructure development, fundamental research, archival projects, community partnerships, and researcher development may generate modest short-term metrics.
Annual or short-cycle performance targets can favor projects with faster measurable returns.
This creates a temporal distortion: activities valuable over a decade may compete poorly with activities that produce a countable output this year.
Distortion Can Occur at Several Levels
| Level |
Possible external pressure |
Potential distortion |
| Institution |
Improve global ranking |
Resources shift disproportionately toward ranking-visible research |
| Faculty or school |
Meet institutional research targets |
Disciplines are evaluated using poorly matched output expectations |
| Department |
Increase annual publications |
Short-cycle output receives priority over longer-term research development |
| Individual researcher |
Publish for promotion or performance review |
Topic, method, or dissemination choices become shaped by rewarded outputs |
| Student researcher |
Contribute to institutional research production |
Educational research experiences risk becoming output-oriented rather than developmental |
Researcher Motivation Can Change as Institutional Priorities Change
An institutional target eventually reaches individual people. A faculty member who once conducted research primarily because of curiosity or professional contribution may begin researching because publication is required. A doctoral student may learn that particular outputs are valued more strongly than others. Research leaders may choose projects partly according to what contributes to institutional metrics.
This is how organizational incentives can influence why researchers themselves conduct research.
The change is not automatically harmful. External incentives can encourage productive activity. The concern is whether controlled pressure becomes sufficiently strong to displace more autonomous reasons for undertaking worthwhile research.
Distortion Does Not Require Misconduct
Institutional research priorities can become distorted even when every researcher behaves ethically.
Imagine that faculty members simply choose internationally visible topics because those studies receive greater institutional support. Over time, locally important questions receive less attention. No misconduct has occurred. The research portfolio has nevertheless changed because the incentive system made some questions more attractive than others.
This distinction is important because debates about research incentives sometimes jump immediately to misconduct. Much of the effect of incentives occurs through perfectly legitimate choices about topics, collaborations, methods, publication venues, and allocation of time.
Distortion Can Eventually Create Integrity Risks
Although distortion does not require misconduct, excessively demanding output environments can create pressures relevant to research integrity. Researchers may experience stronger incentives to produce positive, publishable, rapid, or numerous outputs.
The National Academies has emphasized that research organizations influence integrity through their policies, practices, and incentive structures. Responsible research environments therefore require attention not only to individual conduct but also to the organizational conditions under which researchers work.
Watch Out
Do not assume that a researcher responding strategically to institutional incentives is behaving improperly. Researchers operate within real career and organizational constraints. The more useful question is whether the incentive system makes responsible, valuable research easier or harder to pursue.
Responsible Research Assessment Offers an Alternative
Research-assessment reform initiatives have emerged partly because narrow quantitative indicators can produce unintended consequences.
The San Francisco Declaration on Research Assessment recommends evaluating research on its own merits rather than relying on journal-based metrics as proxies for individual contributions. The Coalition for Advancing Research Assessment similarly promotes assessment approaches that recognize diverse research contributions and emphasize qualitative judgment supported by responsible quantitative indicators.
The Leiden Manifesto proposes principles for research metrics that include supporting qualitative expert assessment, measuring performance against institutional research missions, accounting for field differences, and scrutinizing indicators regularly.
These approaches do not require abandoning measurement. They seek to keep measurement subordinate to the purposes being evaluated.
The Institutional Mission Provides a Useful Reference Point
A university cannot avoid external evaluation, but it can decide how external evidence fits within its mission.
A research-intensive university may reasonably prioritize globally influential scholarship. A regional university may combine international research aspirations with research addressing local economic, educational, environmental, cultural, or public-service needs. A teaching-oriented institution may emphasize scholarship connected with faculty expertise and student research opportunities.
External indicators become less distorting when institutions can explain why their research priorities make sense independently of the indicator.
The Test Is Whether the Research Would Still Matter Without the Metric
A simple counterfactual can clarify strategy: if the ranking indicator or accreditation requirement disappeared tomorrow, would the research priority still have a compelling scientific, educational, institutional, professional, or societal rationale?
If yes, the external system may be reinforcing an already defensible priority.
If no, the institution should examine whether it is investing primarily in the metric rather than the underlying purpose.
