03 · What You Need to Know
Match the Evidence to the Research Problem You Are Claiming
Start by Identifying the Claim That Needs Support
Before counting sources, ask what you are actually trying to establish.
Research problem statements often contain several distinct claims:
- the phenomenon exists;
- it occurs at a particular magnitude or frequency;
- it has changed over time;
- it affects a particular population;
- it has important consequences;
- existing knowledge is insufficient;
- previous studies conflict;
- an explanation is inadequate;
- a methodological weakness limits existing evidence; or
- research is needed to reduce a consequential uncertainty.
Each claim may require different evidence. A government dataset might establish the prevalence of a practical condition but tell you little about whether the scholarly literature adequately explains it. A systematic review may characterize uncertainty in a body of studies but provide no evidence that your particular organization experiences the same practical problem.
The first principle is therefore simple: support the claim with evidence capable of supporting that kind of claim.
There Is No Magic Number of Sources
Researchers sometimes ask whether three, five, ten, or twenty references are enough to establish a problem. No defensible general rule works that way.
Ten weak or irrelevant sources do not necessarily provide stronger support than one rigorous synthesis directly addressing the question. Conversely, one article cannot normally establish that an entire field has consistently overlooked a problem simply because its authors describe a gap.
The relevant considerations are quality, relevance, breadth, recency where recency matters, and consistency with the claim being made.
A source count can tell you how many documents you cited. It cannot tell you whether your problem statement is justified.
Enough citations
A numerical idea: you have accumulated a certain number of references.
Enough evidence
An evidential judgment: the available sources collectively provide adequate support for the specific research-problem claim you are making.
Broader Claims Require Broader Evidence
Consider the difference between these statements:
“This university recorded a decline in use of its academic-support service during the past three academic years.”
“University students increasingly avoid academic-support services.”
The first is a bounded institutional claim. Appropriate administrative records from the university may be sufficient to establish it, assuming the records and comparison are valid.
The second makes a much broader claim about university students generally and a trend over time. Evidence from one institution cannot establish that.
This principle applies throughout research problem development. The larger the population, period, literature, causal claim, or level of generality, the more comprehensive the supporting evidence normally needs to be.
One Study Can Establish Something Without Establishing Everything
It would be equally mistaken to say that one study is never enough.
A single high-quality study can provide credible evidence that a phenomenon occurs under the conditions investigated. A local administrative dataset may establish that a documented process has changed. A carefully conducted experiment may provide strong evidence about a particular relationship under defined conditions.
What one study usually cannot do is justify a much broader conclusion than its design and evidence support.
For example, one study finding no association does not establish that “research has shown no relationship.” One paper identifying a limitation does not establish that the entire literature suffers from that limitation. One community experiencing a problem does not establish that the problem is widespread nationally.
The correct question is not “Is one study enough?” but “Enough for which claim?”
Systematic Reviews Can Be Especially Useful for Claims About a Body of Evidence
When your research problem concerns what an entire literature does or does not establish, evidence syntheses can be particularly informative.
Systematic reviews use explicit methods to identify, select, appraise, and synthesize relevant research. When well conducted and sufficiently current, they can help establish whether evidence is limited, inconsistent, imprecise, methodologically weak, or otherwise insufficient for a particular conclusion.
Research-gap frameworks developed from systematic reviews similarly emphasize that a gap can arise because information is insufficient, biased, inconsistent, imprecise, or otherwise unable to provide an adequate answer.
This makes a rigorous synthesis useful when your claim is about the state of evidence rather than merely about one study.
But a systematic review is not automatically definitive. Its relevance depends on its question, search coverage, inclusion criteria, methodological quality, date, and whether important new research has appeared since its search was completed.
“More Research Is Needed” Is a Lead, Not Proof
Researchers routinely conclude papers by recommending further research. Those statements can help you identify possible problems, but they should not be treated as self-validating evidence that another study is necessary.
An author may recommend research because the study had limitations, because another population might be interesting, because a mechanism remains uncertain, or simply because future research is a conventional part of scholarly discussion.
More importantly, the recommendation can become outdated. Subsequent studies may already have addressed the question.
If a paper says more research is needed, ask:
- What specific uncertainty did the authors identify?
- Does their evidence actually support that uncertainty?
- How consequential is it?
- Has later research addressed it?
- Does the issue remain unresolved across the broader literature?
Use another researcher's recommendation as a clue to investigate, not as permission to stop investigating.
A Knowledge Gap Requires Evidence About What Is Already Known
If your research problem is that something is insufficiently understood, you need a defensible basis for that conclusion.
This does not require proving the absolute absence of every relevant publication. Such claims are usually impossible to establish conclusively.
Instead, show what the literature currently establishes and where its limits remain. Relevant evidence might include recent reviews, primary studies, unresolved questions repeatedly identified across credible sources, and your own transparent search of the relevant literature.
The stronger formulation is often not “nothing is known about X” but “existing evidence establishes A and B, but does not adequately establish C under these conditions.”
This is particularly important when claiming that a lack of understanding constitutes the research problem.
Claims of Conflicting Evidence Need More Than Two Convenient Papers
You can almost always find two studies with different numerical results in a sufficiently large literature. That does not automatically establish a meaningful conflict.
If your research problem is inconsistency, examine whether the relevant studies are sufficiently comparable, whether their estimates genuinely differ, how precise those estimates are, whether methodological differences explain the pattern, and what the broader body of evidence shows.
A claim that “the literature is mixed” should therefore be based on a reasonable assessment of the literature rather than a pair of selectively chosen articles.
When conflicting evidence is the research problem, the inconsistency itself is something you need to demonstrate.
A Methodological Problem Requires Evidence of Consequence, Not Just Imperfection
Every study has limitations. Pointing out that previous research used self-report, cross-sectional data, convenience sampling, one measurement instrument, or another imperfect approach does not automatically establish a methodological research problem.
You need to show why the limitation matters to the conclusion.
Does it prevent temporal inference? Does the measurement fail to capture an important construct adequately? Does the sampling strategy systematically exclude a population whose inclusion could change interpretation? Does a recurring risk of bias materially weaken confidence in the evidence?
Ideally, the limitation should be visible across the relevant evidence rather than selected from one convenient article, especially when you claim that the field as a whole has a methodological problem.
This is the difference between identifying an ordinary limitation and establishing a methodological weakness that genuinely warrants further research.
Practical Problems Need Evidence From the World, Not Only From the Literature
If the problem concerns what is happening in a particular organization, population, community, ecosystem, or service, scholarly literature alone may not establish it.
Suppose your research problem is that waiting times have increased at a particular clinic. Articles showing that long waiting times occur in other health systems do not establish that your clinic has the same problem.
You may need administrative records, monitoring data, official statistics, audits, observations, evaluations, surveys, or another credible source appropriate to the claim.
The literature may help explain why the problem matters, what mechanisms are plausible, how it has been studied elsewhere, and what methods are appropriate. Local or population-specific evidence establishes whether the claimed condition actually exists where your study says it does.
Different Claims May Need Different Sources
A strong research problem often rests on a chain of evidence rather than one source type.
| Claim |
Potentially Useful Evidence |
What to Check |
| A practical condition exists locally |
Administrative records, monitoring data, official statistics, audits, credible observations |
Coverage, definitions, completeness, comparison period, data quality |
| A phenomenon is widespread |
Representative surveys, surveillance, population data, appropriate syntheses |
Population coverage and representativeness |
| Existing knowledge is insufficient |
Systematic reviews, recent primary literature, transparent literature searching |
Search coverage, recency, relevance, what is actually unresolved |
| Evidence conflicts |
Comparable primary studies, systematic reviews, meta-analyses where appropriate |
Comparability, precision, direction, methods, heterogeneity |
| A method is inadequate |
Methodological research, validation studies, risk-of-bias assessments, recurring limitations |
Whether the weakness materially affects conclusions |
| The problem matters to stakeholders |
Stakeholder research, needs assessments, consultations, documented decisions, outcome data |
Whose perspective is represented and what claim it supports |
No row supplies a universal recipe. The appropriate evidence depends on the discipline and claim. The table is useful because it prevents one source from being asked to establish something it cannot support.
Recency Matters When the Problem Can Change
A source can be authoritative and still be too old for a time-sensitive claim.
If you are establishing a current policy problem, recent prevalence, present service use, current technology adoption, contemporary publication practices, or another changing condition, older evidence may no longer describe the situation accurately.
For stable theoretical concepts or foundational methods, older sources may remain entirely appropriate.
Do not use recency mechanically. Ask whether the underlying fact could plausibly have changed. If it could, verify the current evidence.
Quality Matters More Than Citation Volume
A large stack of citations can create an illusion of certainty.
Suppose fifteen articles repeat the same claim, but all ultimately rely on one weak original dataset. The number of citations does not create fifteen independent pieces of evidence.
Likewise, many studies with similar biases may collectively leave substantial uncertainty.
Consider the quality and independence of the evidence, not merely how often a claim appears in print. Where formal appraisal tools are appropriate to your field, use them. Where they are not, critically evaluate study design, data quality, relevance, precision, and limitations rather than treating publication as automatic validation.
Triangulation Can Strengthen a Practical Problem
Some practical problems are better established through multiple kinds of evidence.
Suppose administrative data show declining service use, interviews indicate recurring access difficulties, and operational records show increased waiting times during the same period. These sources answer different questions, but together they may provide a more informative picture of the problem.
Triangulation does not mean that three weak sources automatically produce one strong conclusion. Nor does agreement among sources prove a particular cause. Its value comes from examining whether different forms of evidence converge, complement one another, or reveal contradictions that need explanation.
Stakeholder Experience Is Evidence, but Match It to the Claim
Practitioners, community members, patients, students, employees, and other stakeholders can provide essential evidence about experiences, priorities, implementation, and problems that may be poorly visible in formal datasets.
But stakeholder testimony should not be stretched beyond what it establishes.
If several staff members report that a process is frustrating, that is evidence about their experience. It does not by itself establish how common the experience is across an entire workforce or prove that a particular policy caused it.
Different evidence answers different questions. Respecting that distinction strengthens rather than diminishes stakeholder knowledge.
Absence of Evidence Is Especially Difficult to Establish
Claims such as “no research exists,” “this has never been studied,” or “there is no evidence” are stronger than they appear.
Your search may have missed different terminology, another discipline, non-English research, grey literature, older work, or newly published studies. Database coverage is also imperfect.
When possible, use language proportionate to your search:
“Our search did not identify studies examining X under Y conditions.”
That is more defensible than:
“No study has ever investigated X.”
The first describes what your search supports. The second makes an absolute claim about the entire universe of research.
Watch Out
Absolute absence claims require unusually strong support and are often unnecessary. You usually need to establish that consequential evidence is insufficient for the question, not prove that nobody anywhere has ever published anything related to it.
Do Not Confuse Evidence That the Problem Exists With Evidence About Its Cause
Suppose records establish that employee turnover has increased. That evidence supports the existence of the observed problem.
It does not establish why turnover increased.
If managers believe workload caused the increase, that is a separate claim requiring separate evidence. The same distinction applies to declining student attendance, low service uptake, poor program outcomes, or any other practical problem.
Keeping symptoms and explanations separate prevents you from building a study around a causal assumption that has not yet been established. This is why distinguishing the research problem from its symptoms and suspected causes matters when evaluating evidence.
Evidence About Existence and Evidence About Importance Are Also Different
You may establish convincingly that a problem exists and still need to show why it deserves research attention.
For example, records may demonstrate that a minor administrative error occurs several times per year. That establishes the problem. It does not establish that a major research project is justified.
Significance requires another layer of reasoning: what consequences follow, who is affected, what decisions depend on better evidence, and what could be gained by reducing the uncertainty?
A research problem must be both defensible as a factual claim and worthwhile as a research priority.
Stop Collecting Evidence When the Claim Is Adequately Supported, Not When the Bibliography Looks Impressive
There is no virtue in citing every paper that has ever touched the topic.
You need enough evidence to understand the state of knowledge, represent important variation and disagreement, identify relevant limitations, and justify the claims required by your problem statement. Beyond that point, additional citations may add little.
The threshold will differ by project. A dissertation literature review normally requires broader coverage than a short proposal. A systematic review requires explicit comprehensive methods beyond what an ordinary problem statement requires. A local quality-improvement study may rely heavily on institutional evidence while still consulting relevant external research.
The governing principle remains the same: evidential sufficiency is determined by the claim and purpose, not by a universal number.