How Do You Know When Your Literature Search Is Too Broad?

Does Getting Thousands of Results Mean Your Search Is Too Broad?

You run a literature search and the database reports 8,000 results.

Your immediate reaction may be: This search is far too broad.

Perhaps it is. But the number alone cannot tell you that.

A search can retrieve thousands of records because the topic has a large literature, because your strategy prioritizes sensitivity, because one term has several meanings, or because an important concept is missing. Conversely, a search producing only a few hundred records can still be too broad if almost none of them addresses your actual question.

The more useful question is therefore not simply how many records you retrieved. It is whether the search is retrieving so much irrelevant material, for identifiable reasons, that its current structure no longer serves the purpose of the search.

A Search Is Too Broad When Its Extra Retrieval Stops Being Useful

Broadness Is About Retrieval Performance, Not Just Result Count

There is no universal number that means a search is too broad

It would be convenient if literature searching came with a simple threshold: more than 1,000 results means too broad, fewer than 100 means acceptable.

No such general rule exists.

The appropriate number of records depends on the research question, size of the literature, database, purpose of the search, review methodology, and resources available for screening. A systematic review may legitimately retrieve thousands of records because its search is designed to minimize the risk of missing eligible studies. A narrowly focused student research project may have little reason to screen the same volume.

Even within the same project, different databases can return very different numbers because their coverage, indexing, interfaces, and search processing differ.

The result count is therefore a signal to investigate, not a verdict.

Look at precision, not only volume

A useful way to understand excessive breadth is through precision.

In information retrieval, precision refers to the proportion of retrieved records that are relevant. If a search retrieves 1,000 records and only 100 are relevant, its precision is 10%.

That final limitation matters. A search can have excellent precision because it retrieves only a small set of obviously relevant papers while missing much of the literature.

Precision therefore needs to be considered alongside sensitivity, also called recall.

Cochrane's current guidance emphasizes this trade-off for systematic reviews: searches should maximize sensitivity while striving for reasonable precision. Increasing comprehensiveness will commonly reduce precision because additional irrelevant records are retrieved along with relevant ones.

A broad search is therefore not automatically a bad search. Low precision can sometimes be the cost of the sensitivity your research method requires.

Too broad and appropriately sensitive are not the same thing

Imagine two searches that each retrieve 5,000 records.

The difference is not the number 5,000. It is why those records were retrieved and whether the irrelevant retrieval contributes anything useful.

Repeated patterns of irrelevance are your best clues

When a search feels too broad, inspect the irrelevant results rather than immediately adding restrictions.

Look for patterns.

Suppose you are searching for literature on generative AI in academic writing. You notice that a large proportion of the results concern:

• AI-assisted computer programming;
• automated clinical documentation;
• creative writing outside education;
• machine-generated scientific abstracts;
• professional rather than student writers.

Those patterns tell you more than the total result count does. They reveal where the search is conceptually leaking into adjacent literatures.

You can then ask which search term is responsible, which concept is missing, or whether one term is being interpreted more broadly than intended.

An ambiguous term can make an otherwise sensible search explode

Sometimes one word is responsible for a disproportionate amount of irrelevant retrieval.

Imagine a search involving the term engagement. Depending on the field, records might discuss student engagement, employee engagement, stakeholder engagement, user engagement, civic engagement, treatment engagement, or social media engagement.

The word itself is not wrong. It is simply polysemous: it carries several meanings.

Inspecting irrelevant results may reveal that the search needs a more specific phrase, an additional concept, an appropriate field restriction, a controlled vocabulary term, or a different expression of the concept.

This is preferable to adding arbitrary restrictions elsewhere in the strategy.

A missing concept can make the search broader than intended

Suppose your research question concerns generative AI in academic writing among university students, but your search contains only a generative AI concept and a writing concept.

If the result set is dominated by professional writing, journalism, scientific writing by researchers, and creative writing, a population or educational-context concept may improve precision.

That does not mean every detail from the research question should be added. The previous conceptual work still matters: only concepts that contribute usefully to retrieval should become mandatory search blocks.

A good refinement solves a diagnosed problem. It should not merely make the number smaller.

Overly generous synonyms can also broaden the search

Within a concept block, alternative terms are commonly joined with OR. This improves sensitivity because the database can retrieve different expressions of the concept.

But every term added with OR can also introduce records.

Suppose your concept is academic writing, and you add the single word writing as an alternative. That term may retrieve a much wider literature than phrases specifically related to academic or scholarly writing.

The appropriate response is not to avoid synonyms. A robust search often needs substantial terminological variation. Instead, check whether each term genuinely represents the intended concept and whether its contribution to retrieval is useful.

Broad phrase handling can produce unintended combinations

Search systems vary in how they process multiple words. If a meaningful multiword concept is searched too loosely, its individual words may retrieve contexts you did not intend.

For example, academic integrity has a particular meaning when the words occur together. Depending on the database and syntax, searching the words independently could allow records containing academic in one context and integrity in another.

Selective phrase searching can sometimes improve precision. But it should be used deliberately because making phrases too strict can create the opposite problem and cause relevant variants to disappear.

This is why searching an entire research question as one phrase is not a sensible cure for an overly broad search.

Searching very broad fields can increase irrelevant retrieval

Many databases allow searching within particular fields, such as title, abstract, author keywords, or subject headings. Searching across all available fields may retrieve a term from places where its presence says little about the main subject of the article.

Field searching can sometimes improve precision by requiring a term to appear in a more informative part of the record. However, restricting fields can also reduce sensitivity if relevant records do not contain the expected terminology in those fields.

As with other refinements, field restrictions should be tested rather than treated as universally superior.

Database filters can make the result count smaller without making the search conceptually better

When confronted with thousands of records, it is tempting to use whatever filters are visible in the interface: last five years, English only, journal articles only, peer reviewed, subject category, age group.

The count falls. The search feels fixed.

But a smaller number is not evidence that the underlying retrieval problem has been solved.

If irrelevant results are appearing because one keyword is ambiguous, restricting the search to the last five years does nothing to correct that ambiguity. It merely removes older relevant and irrelevant records alike.

Restrictions involving the date range of a literature search, language, or peer-reviewed literature should be justified by the research purpose or eligibility criteria, not used as emergency brakes for an imprecise search.

Do not confuse database relevance ranking with search precision

Some search systems rank records so that those judged most relevant appear first. This can make a broad search feel surprisingly good because the first page contains several useful articles.

But relevance ranking does not necessarily change which records were retrieved. Thousands of weakly related records may still exist further down the result set.

If you need to screen the complete result set, reproduce the search, or report a systematic search, the quality of the top ten results is not enough to evaluate the strategy.

For exploratory searching, ranking may make a broad search perfectly usable. For systematic evidence identification, the methodological requirements are different.

Your purpose determines how much irrelevant retrieval is acceptable

Search breadth should always be interpreted in relation to the job the search must perform.

Cochrane notes that systematic review searches can have high yield and low precision because sensitivity is prioritized. Its guidance also points out that database records can often be screened relatively quickly, so substantial irrelevant retrieval may sometimes be preferable to a search that looks efficient but misses eligible evidence.

That is a methodological trade-off, not a defect to be eliminated at all costs.

Test a narrowing change against what you already know is relevant

Suppose you identify a term that appears to cause hundreds of irrelevant results. You remove it, and the result count drops from 4,500 to 1,300.

That looks promising.

Before celebrating, ask what else disappeared.

If you know several publications that should be retrieved, check whether they remain. Examine relevant records found by the broader strategy and see whether the narrower version still captures them.

Cochrane recommends checking whether a developing strategy retrieves key publications or studies included in similar reviews as one basic test of search performance. However, retrieving known papers alone does not prove comprehensiveness, because a search can become inadvertently tailored to literature you already know.

A useful refinement should therefore improve the signal without simply optimizing the strategy around a small set of familiar papers.

Narrow one identifiable problem at a time

If possible, avoid changing several dimensions of the search simultaneously.

Suppose you respond to excessive results by:

• removing three synonyms;
• adding a population concept;
• requiring an exact phrase;
• restricting the date range;
• limiting the search to English.

The result count falls dramatically, but you no longer know which change produced the improvement or which one removed relevant literature.

A more diagnostic sequence is:

This turns narrowing from trial and error into search strategy development.

Diagnosing a Search With Thousands of Irrelevant Results

Notice what the researcher did not do: decide in advance that 6,400 was an unacceptable number and keep narrowing until only 200 remained.

The target was not a particular result count. The target was a search in which each restriction had a defensible retrieval function.

The reverse problem is also possible. If refinement removes so much that expected literature disappears, you need to consider whether the search has become too narrow.

Common Mistakes When a Search Returns Too Much

Diagnose the Source of Breadth Before You Narrow the Search

If your result count looks alarming, do not begin by asking how to make it smaller. Ask why the irrelevant records are there.

That change in question makes search refinement considerably more defensible.

If you are still at the exploratory stage, some excess retrieval may actually be useful because it helps reveal terminology and neighboring literatures. This is part of deciding how broad the initial search should be.

For a systematic review or another search where missing eligible evidence carries substantial consequences, the tolerance for low precision may be considerably greater. Screening hundreds of irrelevant abstracts is tedious. Missing an important eligible study can be methodologically worse. Academic glamour, regrettably, rarely resides in the screening queue.

Before Deciding That Your Search Is Too Broad

Questions About Literature Searches That Retrieve Too Much

A Search Is Too Broad Because of What It Retrieves, Not Simply How Much

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.