03 · What You Need to Know
Citations Tell You Something, but Not Everything You Want to Know
What does a citation actually tell you?
At its simplest, a citation records that one scholarly work has included another work in its references. When many later publications cite a paper, that pattern can provide evidence that the paper has attracted scholarly attention or influenced subsequent literature.
That is useful information. It is also narrower than several claims researchers sometimes make about citations.
The San Francisco Declaration on Research Assessment, commonly known as DORA, notes in its guidance on research indicators that citations may reflect the influence of a research article, but influence differs from research quality and significance. Citation data also do not tell you simply from the count whether a work was cited positively, critically, methodologically, historically, or for some other reason.
Citation impact
The extent to which a publication is cited in subsequent scholarly literature, interpreted with appropriate attention to field, time, document type, and other relevant factors.
Research importance
The significance of the question, knowledge, evidence, method, explanation, or other contribution, which cannot be reduced to the number of references the publication later receives.
The two can certainly be related. Important research may become highly cited. But treating them as interchangeable creates problems both for evaluating published work and for deciding what research to conduct next.
High citation potential can partly reflect the size of the conversation
Imagine two equally rigorous studies. One addresses a topic investigated by tens of thousands of researchers. The other concerns an important problem within a small specialty.
The first paper has many more potential citers simply because many more papers are being produced around it.
This is one reason raw citation counts differ substantially across disciplines and research areas. Bibliometric research has repeatedly documented differences in citation practices across fields, and citation indicators are often normalized by field and publication year when comparisons are required.
A question can therefore have lower citation potential because fewer researchers work on it, not because the question is less important.
Publication age matters because citations need time to accumulate
Citation counts are also time-dependent. A paper published ten years ago has had far more opportunity to accumulate citations than one published six months ago.
DORA consequently describes citation performance as a lagging indicator that may take years to become a robust signal. This makes citation counts particularly problematic when evaluating recent scholarship or comparing work with substantially different publication ages.
The same logic matters prospectively. Predicting how highly a future paper will eventually be cited requires anticipating not only the quality and usefulness of work that has not yet been conducted, but also the future development of its research area.
That is a considerable amount of uncertainty to build into the choice of a research question.
Different kinds of papers have different citation opportunities
Document type also affects citation behavior. Reviews, methodological papers, datasets, guidelines, and empirical articles can occupy different roles in a literature and may accumulate citations differently.
A broadly useful review, for example, may be cited whenever researchers need a convenient synthesis of a field. A specialized empirical study may be relevant to a much narrower set of later papers even if its evidence is exceptionally rigorous.
This makes a simple equation such as “more citations = better research question” difficult to defend.
The paper's function in scholarly communication affects its citation opportunities.
Citations do not directly measure research quality
Perhaps the most consequential mistake is treating citation counts as if they were direct measurements of scientific quality.
A large study in the behavioral and brain sciences examined whether citation counts and journal impact factors predicted several indicators related to research quality, including statistical reporting accuracy, evidential value, and replicability. The relationships were weak and inconsistent. This does not establish that citations contain no useful information, but it illustrates why citation counts should not simply be equated with methodological or evidential quality.
DORA's guidance reaches a similar practical conclusion from the perspective of research assessment: no single indicator captures the complexity of research quality, and citation data should be used contextually and with expert judgment rather than as substitutes for it.
If citation count is an imperfect retrospective measure of quality, expected citation count is an even shakier prospective criterion for deciding whether a question deserves to exist.
Research can be important to a small audience
Suppose a rare disease affects relatively few people and is studied by a comparatively small research community. Or imagine a highly specialized methodological problem that affects only researchers using a particular analytical technique.
The number of potential citing papers may be modest. The consequences of resolving the question could nevertheless be substantial for the people who depend on the answer.
The same can happen with research questions arising from local settings. A study could provide crucial evidence for a community, institution, profession, or policy decision while attracting relatively little international scholarly attention.
Citation potential therefore partly reflects the size and citation behavior of the scholarly audience. It does not simply measure how much the answer matters.
A citation is not necessarily an endorsement
Researchers cite papers for many reasons.
A publication may provide supporting evidence, introduce a method, define a concept, supply a dataset, represent a competing explanation, provide historical context, or serve as an example of a claim being criticized.
DORA explicitly cautions that citation data do not reveal whether articles are being cited for positive or negative reasons. A controversial or flawed paper can attract substantial attention precisely because researchers are responding to it.
High citation counts can therefore indicate scholarly influence without establishing that the underlying claims are correct.
Some valuable research becomes infrastructure
Citation potential becomes more complicated when research creates tools, datasets, software, protocols, measures, or other resources.
Such work may become heavily cited because many researchers reuse it. That citation pattern can provide meaningful evidence of scholarly influence. Yet actual use and citation are not always identical. Researchers may use resources without citing them consistently, and citation conventions vary among fields and output types.
The broader lesson is that research contribution can take forms that a single citation count captures only partially.
Citation practices vary across disciplines
Raw citation counts should not be compared casually across fields. Some disciplines publish more frequently, produce longer reference lists, have larger research communities, or rely more heavily on journal articles than others.
Bibliometric methods therefore often normalize citations by field, publication year, and sometimes document type. The NIH-developed Relative Citation Ratio, for example, was designed to assess article-level influence relative to a co-citation network rather than interpreting raw citation counts without context.
Normalization can improve particular comparisons, but it does not turn citations into a complete measure of research importance. It addresses some bibliometric differences rather than solving the conceptual problem of defining value.
Citation potential can tempt you toward crowded questions
If you deliberately optimize for expected citations, a predictable strategy emerges: work where many other researchers are already working.
Large, active fields provide more potential readers and citers. Fast-moving topics may offer frequent opportunities for publication and scholarly attention.
That can be entirely reasonable when important unanswered questions exist there. But it can also create an odd incentive: questions receive more research attention partly because they already receive more research attention.
Meanwhile, neglected questions may remain neglected precisely because fewer researchers work on them and the expected bibliometric payoff is lower.
Watch Out
Do not infer that a topic with low expected citation potential has low research value. A small literature may indicate a narrow or trivial question, but it can also reflect an understudied population, neglected problem, emerging field, disciplinary publication pattern, or issue whose primary beneficiaries are not academics.
Optimizing for citations can distort what you ask
Metrics become particularly consequential when they shift from measuring research after publication to shaping research before it begins.
Imagine that you are choosing between two questions. One concerns a fashionable topic with a large literature and high citation activity. The other addresses a consequential uncertainty for a smaller research community but is unlikely to attract comparable attention.
If expected citation count becomes the deciding criterion, you may select the first question even when the second has greater scientific or practical importance.
This is one manifestation of a broader concern in responsible research assessment: metrics can influence behavior when they become targets rather than contextual indicators.
DORA therefore recommends that quantitative indicators be used transparently, specifically, contextually, and fairly, alongside qualitative assessment rather than allowing metrics to lead evaluation.
Do not confuse citation potential with discoverability
There is an important difference between selecting a question because it is likely to attract citations and making worthwhile research easier for the right audience to discover.
The latter is good scholarly communication.
A carefully chosen title, informative abstract, appropriate keywords, clear terminology, relevant indexing, open dissemination where feasible, conference presentation, data sharing, and engagement with the appropriate research community can help a study reach people who may genuinely use it.
Those activities do not require you to redesign the research agenda around citation counts. They help useful work enter the scholarly conversation.
Optimizing the question for citations
Choosing what to investigate partly because you expect the resulting paper to accumulate a favorable metric.
Optimizing dissemination
Helping research that was already worth doing become discoverable, accessible, understandable, and usable by the audiences who may need it.
The second is usually much easier to defend.
Citation potential can legitimately enter career strategy
None of this requires pretending that citation metrics have no consequences for researchers.
Institutions, rankings, funding systems, promotion processes, and hiring committees may use citation-based indicators. The extent and sophistication of that use vary considerably. A researcher working in such an environment may reasonably consider whether a portfolio of projects will produce visible scholarly contributions.
That is a career constraint, not evidence that the most citable question is scientifically the most important.
A useful distinction is between choosing your entire research agenda to maximize citations and considering likely scholarly visibility when choosing among several questions that are already worth pursuing.
| Reasonable strategic consideration |
Metric-driven warning sign |
| Considering likely scholarly audience when choosing among several worthwhile projects |
Rejecting an important question mainly because the relevant research community is small |
| Building a portfolio containing both specialized and broadly relevant work |
Choosing topics primarily according to expected citation volume |
| Making completed research discoverable to people likely to use it |
Designing studies around fashionable keywords simply to attract attention |
| Understanding how citations are used in your actual evaluation system |
Treating citation count as the objective definition of research success |
| Using contextualized metrics alongside qualitative evidence of contribution |
Comparing raw citation counts across fields as though citation practices were identical |
Expected citations are especially uncertain for new research areas
A new field creates an interesting paradox. It may currently have few publications and therefore few potential citers, yet it could later become scientifically important.
If researchers had avoided every emerging question because the citation market was initially small, some important areas would have struggled to develop at all.
The opposite is also true. A rapidly expanding topic can appear destined for high citation activity and then lose scholarly attention before your study is complete.
This is why timeliness should be distinguished from trend-chasing. Research agendas often require a longer horizon than citation predictions encourage.
Ask whether you want citations or scholarly use
A subtle but useful reframing is to ask what you actually want citations to represent.
Presumably, you do not want other researchers merely to type your surname into their reference lists. You want the work to contribute to subsequent thinking, evidence, methods, decisions, or investigation.
That shifts the objective from maximizing citation count to producing research that others have a reason to engage with.
A methodological paper might be valuable because researchers reuse its procedure. An empirical study might resolve an uncertainty needed for later research. A theoretical paper might provide a better explanation. A local study might inform a consequential institutional decision even if few papers cite it.
Once the desired contribution is specified, citation count can be interpreted as one possible trace of scholarly influence rather than the purpose of the research itself.
The most citable question is not necessarily the question you should study
Choosing research inevitably involves trade-offs. Scientific importance, practical relevance, feasibility, personal expertise, available data, ethics, funding, career requirements, and scholarly opportunity can all matter.
Citation potential can be included in that landscape, particularly when you are making realistic career decisions. But it should remain subordinate to a more basic test:
If nobody could tell you how many citations the eventual paper would receive, would you still have a convincing reason why this question deserves an answer?
If yes, you have an intellectual justification independent of the metric. If no, the research question may be serving your citation strategy more clearly than it serves the development of knowledge.
