01 · The Question
When Does Asking Research to Matter Begin to Change Research for the Worse?
There is a strong case for asking researchers what difference their work might make. Research consumes resources, often involves public funding or participant contributions, and can potentially improve policy, professional practice, technology, services, communities, or people's lives. Researchers should not be surprised when funders, institutions, and the public ask what those investments accomplish.
But incentives do not merely measure research after it happens. They can also change what researchers do before the research begins.
If researchers believe that funding, promotion, institutional recognition, or future opportunities depend on demonstrating visible impact, they may begin selecting questions partly for their ability to produce an attractive impact story. Long-term, foundational, uncertain, locally relevant, critical, or difficult-to-measure contributions may become harder to defend.
The problem, then, is not whether research should matter. It is whether the demand to demonstrate impact starts determining what kinds of research are considered worth doing.
03 · What You Need to Know
Impact Assessment Does More Than Observe Research
Research assessment creates incentives
Evaluation systems do not simply describe what researchers have accomplished. When funding, appointments, promotion, institutional resources, prestige, or other rewards depend on particular criteria, researchers and organizations have reasons to respond to those criteria.
This is not necessarily undesirable. Incentives can encourage behaviors a research system legitimately wants, including attention to research quality, openness, collaboration, engagement, or societal relevance.
The difficulty is that indicators and assessment criteria can also have systemic effects. Responsible research-assessment initiatives explicitly recognize this problem. The Leiden Manifesto, for example, includes principles calling for evaluators to recognize the systemic effects of assessment and indicators, protect excellence in locally relevant research, account for disciplinary variation, and use quantitative evaluation to support rather than replace expert judgment.
The broader lesson is simple: once researchers know how they will be evaluated, the evaluation criteria can influence the activity being evaluated.
Impact can become a target rather than an outcome
Consider two different questions:
What contribution could this research make?
The question begins with the research problem and then considers plausible pathways through which the resulting knowledge might matter.
What research can I propose that will demonstrate impact?
The desired evaluative outcome begins shaping the research question itself.
The second question is not automatically illegitimate. Applied research is often intentionally designed around a practical problem. Research intended to improve health, education, public services, professional practice, technology, or policy may properly begin with a desired outcome.
The distortion occurs when researchers pursue particular kinds of outcomes primarily because the assessment system rewards them, even when another question would be more scientifically or socially important.
This distinction extends the question of whether research should always aim for impact. Asking researchers to consider consequences can be productive. Treating a particular kind of consequence as the universal endpoint of worthwhile research is much harder to justify.
Short-term visibility can crowd out long-term value
Some research has a relatively short pathway to application. An evaluation of an existing public program, for example, may generate evidence that decision-makers can use within months.
Other research operates on a different timetable. Foundational knowledge may require years of subsequent investigation before an application becomes conceivable. Methodological work may first improve the quality of later studies. Theoretical research may change understanding long before anyone can identify a practical consequence.
If assessment disproportionately rewards impacts that can be demonstrated within a convenient evaluation period, research with longer or less predictable pathways may become comparatively difficult to justify.
Watch Out
What is easiest to demonstrate within an assessment period is not necessarily what is most valuable in the long term. Visibility, speed, and ease of attribution are properties of an impact pathway, not automatic measures of its importance.
This matters because research benefits can be indirect or take years to appear. A system optimized only for readily observable effects risks undervaluing precisely those contributions whose consequences cannot yet be known.
Researchers may begin overpromising future benefits
Impact pressure can also affect how researchers describe work that has not yet been conducted.
In a research proposal, the actual findings are unknown. Whether stakeholders will use them is also unknown. Whether use will produce the desired consequence introduces still more uncertainty.
Researchers can reasonably identify intended beneficiaries, describe plausible pathways to use, plan engagement, and explain what kinds of effects could follow if particular findings emerge. The problem arises when these conditional possibilities are written as though they were predictable outcomes.
Defensible prospective claim
The findings could inform institutional decisions about the intervention if the study produces sufficiently credible and relevant evidence.
Overstated prospective claim
The research will transform institutional practice and improve outcomes across the sector.
The second statement may sound more impressive in an application, but unless the pathway is unusually direct and well established, it claims knowledge about a future that the researcher does not possess.
When assessment systems reward impressive impact narratives, they can inadvertently reward confidence in prediction rather than accuracy about uncertainty.
Attribution can encourage researchers to tell an artificially simple story
Wider changes rarely have a single cause. A new policy may emerge from years of accumulated evidence, political negotiation, public pressure, economic circumstances, professional expertise, stakeholder advocacy, and institutional constraints.
A research project may contribute meaningfully without being able to claim that it caused the final outcome.
Pressure to demonstrate attributable impact can therefore create an awkward incentive. Researchers may feel encouraged to draw a straighter line from their study to an outcome than the historical evidence warrants.
This is especially problematic when research primarily helps people make better decisions under uncertainty. Evidence can alter the information available to decision-makers without determining which decision they ultimately make.
Measurable impact can displace meaningful impact
Some consequences leave convenient traces. Citations can be counted. Policy documents can be searched for references. Patents, licenses, commercial revenue, downloads, media mentions, and engagement activities can sometimes be documented.
Other contributions are harder to observe. Research may change how practitioners conceptualize a problem, improve the quality of public discussion, challenge an assumption, strengthen a community's capacity to investigate its own concerns, or influence decisions through conversations that leave little documentary evidence.
Measurement difficulty does not make these effects imaginary. It does mean they can be disadvantaged if assessment systems equate what can be counted with what matters.
Reviews of research metrics have therefore emphasized responsible use rather than assuming that indicators can substitute for contextual evaluation. The Metric Tide proposed principles including robustness, humility, transparency, diversity, and reflexivity, with particular attention to the effects that indicators themselves can have on research systems.
Impact pressure can disadvantage locally relevant research
A study can be extremely important to a particular city, institution, professional community, cultural group, or region while having little prospect of generating international visibility.
If impact is implicitly associated with scale, international reach, or easily aggregated indicators, researchers may be encouraged to pursue problems with larger apparent audiences even when a smaller population has a pressing research need.
That would confuse breadth with importance.
The Leiden Manifesto explicitly calls for protecting excellence in locally relevant research. This is particularly important when considering the value of research whose benefits remain local. A geographically limited contribution can still be substantial for the people affected.
Some fields have easier pathways to demonstrable impact than others
Disciplines differ in their relationships to application, publication, audiences, timescales, and evidence. A clinical intervention, engineering technology, historical analysis, mathematical proof, educational study, and philosophical argument do not produce consequences in the same way.
Applying identical impact expectations across these domains can therefore reward disciplinary characteristics rather than research quality.
Current research-assessment reform initiatives reflect this concern. The Agreement on Reforming Research Assessment calls for recognizing diverse research outputs, practices, and activities and for assessment based primarily on qualitative judgment, supported by responsible use of quantitative indicators. It also emphasizes that assessment should reflect differences in research missions and contexts.
The principle is relevant beyond formal assessment exercises: judge research against the contribution appropriate to the kind of research being conducted.
Impact pressure can discourage uncertain or risky inquiry
Research sometimes matters precisely because nobody knows what it will reveal. Exploratory, foundational, and curiosity-driven work may investigate questions for which neither the answer nor its eventual usefulness can be predicted confidently.
A strong requirement for predetermined impact can make such uncertainty appear undesirable. Researchers may instead favor projects with clearer beneficiaries, familiar methods, established partners, and outcomes that can be described convincingly in advance.
Yet uncertainty is not always a weakness in research. It can be the reason the research is needed.
A healthy research system therefore has reason to support both curiosity-driven and problem-driven research. Requiring both to present identical impact pathways would erase an important difference in why they are undertaken.
The impact agenda can also correct genuine weaknesses in research culture
A critical discussion of impact pressure should not imply that researchers ought to be insulated from questions of relevance or accountability.
Impact-oriented policies partly reflect legitimate concerns about whether research systems engage sufficiently with societal problems, communicate beyond academic audiences, recognize diverse beneficiaries, and demonstrate value to those who support research.
Research on impact management also shows that academics and university leaders can disagree about what impact means and how it should be incentivized. This is significant because poorly aligned expectations may generate conflict or encourage behavior different from what institutions intended.
The useful response is therefore not to abandon impact. It is to ask whether impact expectations improve the relationship between research and society without narrowing what counts as worthwhile research.
