03 · What You Need to Know
Documentation Is Useful for More Than Proving a Search Was Systematic
Search documentation is often discussed in the context of systematic reviews, where the rationale is particularly strong.
Cochrane distinguishes documenting the search process from reporting it in the published review. Its guidance requires review authors to maintain sufficiently detailed internal records so that the searches can later be reported correctly and reproduced to the extent possible. PRISMA-S addresses the reporting side, providing a 16-item checklist for reporting literature searches in systematic reviews and related review types.
Those standards do not automatically apply to every search you conduct. PRISMA-S itself is a reporting guideline for systematic reviews, not a general requirement for ordinary literature searching.
Still, the practical benefits of remembering what you searched do not disappear simply because your review is not systematic.
Documentation and Reporting Are Not the Same Thing
This distinction is important.
Documentation is the internal record you maintain while conducting the search. It helps you remember and reconstruct what happened.
Reporting is what you eventually communicate to readers in a thesis, article, review, report, protocol, or supplementary file.
Document the search
Keep an internal record of useful queries, databases, dates, decisions, terminology, supplementary methods, and other information you may need later.
Report the search
Disclose the search methods in the final research output at the level required by the study design, reporting guideline, journal, institution, or other applicable standard.
You can document considerably more than you eventually report.
This is normal research practice. Laboratory researchers retain records that never appear verbatim in a journal article. Analysts keep code and intermediate outputs beyond what fits in the methods section. Search documentation serves a similar function: it preserves the research trail from which an appropriate account can later be produced.
A Search Log Reduces Your Dependence on Memory
Literature searching becomes surprisingly difficult to remember once a project extends beyond a few days.
You may recall that you searched Scopus, but not whether you later modified the query. You may remember discovering an important synonym, but not which paper introduced it. You might search the same combination repeatedly because you forgot that it had already produced little useful material.
A simple search log externalizes that memory.
At its most basic, it can record:
- where you searched;
- when you searched;
- what you searched;
- what you learned from the search;
- what you decided to do next.
The fifth item is often neglected. Search documentation becomes more useful when it records not only queries but also consequential decisions.
“Added automation bias after finding it in a relevant review” tells your future self much more than a spreadsheet containing an unexplained search string.
Documentation Helps You See How Your Search Developed
Literature searching is often iterative.
You begin with the terminology you know. Relevant papers introduce new terms. Some queries retrieve unexpected literatures. Others reveal that two concepts you thought were equivalent are actually distinct. Your research question may narrow or shift as your understanding develops.
If you record these developments, the search becomes an intelligible sequence rather than a collection of disconnected queries.
Initial search Record the starting concepts and terminology.
Discovery Note important synonyms, concepts, subject headings, authors, theories, or citation trails revealed by relevant papers.
Revision Record consequential changes to the query or scope and why they were made.
Focused search Preserve the searches that become particularly important to the final research question.
Update Record when the literature is searched again so you know how current your evidence base is.
This can be particularly helpful when searching leads you to revise the research question. You do not need a forensic record of every passing thought, but documenting major changes can help you reconstruct why the final project differs from the one you initially imagined.
You Do Not Need to Record Every Search You Ever Make
Proportion matters.
If you search Google Scholar for the title of a paper because you need its DOI, there is little value in creating a formal search-log entry.
If you try three quick searches to understand an unfamiliar term before a meeting, extensive documentation is probably unnecessary.
If you spend several weeks developing the literature base for a dissertation, however, the calculation changes. The search will influence how you understand the field, which evidence you cite, how you justify the research problem, and perhaps how you formulate the question.
Document the searches that would be inconvenient, difficult, or consequential to reconstruct later.
| Search activity |
Useful documentation |
Typical level |
| Looking up one known paper |
Usually none beyond your reference manager |
Minimal |
| Exploring an unfamiliar concept |
Useful terms, promising sources, key papers, brief notes |
Light |
| Developing a research question |
Databases or platforms, important queries, terminology discovered, major changes in scope |
Moderate |
| Building the literature base for an empirical thesis or paper |
Major databases, consequential searches, dates, citation searching, relevant inclusion boundaries, update searches |
Moderate |
| Conducting a systematic review or related synthesis requiring reproducible evidence identification |
Complete documentation required by the applicable methodology and reporting guidance |
Detailed |
The boundaries are deliberately approximate. A doctoral program, journal, supervisor, funder, or review methodology may impose more specific requirements.
Record Enough to Reconstruct Important Searches
For a consequential non-systematic search, a useful record often includes the database or search platform, date, search terms or query, and brief notes about the outcome.
You may also want to record relevant filters, the approximate result count, promising papers discovered, terminology harvested from those papers, citation-searching activity, and decisions about how the search should change.
For formal systematic-review searching, considerably more detail is expected. PRISMA-S calls for reporting individual databases and platforms, search strategies, search dates, citation searching, limits and restrictions, search updates, record counts, deduplication, and other aspects where applicable. It recommends reporting search strategies exactly as run because apparently small details such as fields, truncation, Boolean combinations, and limits affect reproducibility.
A non-systematic search log does not need to reproduce that checklist mechanically. It can borrow the underlying principle: preserve details that would otherwise be difficult to reconstruct accurately.
The Database and the Platform Can Both Matter
Researchers sometimes record only “MEDLINE” or “PsycINFO” and assume that is sufficient.
For formal searches, the platform matters because the same database may be accessed through different interfaces with different syntax and search functionality. PRISMA-S therefore asks authors to report both the database and platform.
For an informal exploratory search, that level of precision may not always matter. For a search you expect to rerun months later, it can.
“PsycINFO via EBSCOhost” is considerably easier to reconstruct than “searched psychology database.”
This illustrates a useful principle for non-systematic documentation: record enough context for your future self to know what the entry actually means.
Record the Query When the Query Matters
It is tempting to keep only a list of keywords.
That may be enough during very early exploration, but it can become inadequate once the search develops structure.
Compare:
Keywords: AI, feedback, students, writing
with:
Search: the actual combination of terms, phrases, fields, operators, and limits used in a particular database.
The second record preserves how the concepts interacted. That becomes important if you later want to reproduce the search, understand why it retrieved a particular body of literature, or modify only one component.
For systematic reviews, PRISMA-S recommends preserving the strategy exactly as run rather than reconstructing it later from memory.
The same habit can be useful for any search important enough that you expect to revisit it.
Record Why You Changed the Search, Not Just What Changed
Suppose your search log contains five increasingly complex queries.
Without notes, you may know that the queries changed but not why.
A short explanation can preserve the intellectual logic:
- removed “academic performance” because relevant studies did not consistently mention the outcome in titles or abstracts;
- added a newly discovered synonym used in several relevant papers;
- separated two constructs after reading conceptual literature showing they were not equivalent;
- searched an additional database because the topic crossed into another discipline;
- narrowed the population after the research question was refined.
These notes are particularly useful when a project is collaborative. Your coauthors do not share your memory of why a search changed, and six months from now you may not either.
This also makes it easier to distinguish a deliberate search strategy from an unexplained sequence of keyword experiments.
Documentation Helps Prevent Duplicate Work
Long research projects often involve repeated searching.
You search one database during proposal development, return to the topic while writing the literature review, conduct another search after receiving supervisor feedback, and update the literature during manuscript revision.
Without a record, you may repeat old searches unintentionally or spend time determining whether a particular concept has already been explored.
A search log makes previous work visible.
This becomes particularly valuable in collaborative projects. Two researchers can see which databases have already been searched, which queries were productive, which terminology has been tested, and which areas still require attention.
Documentation therefore serves workflow efficiency as well as methodological transparency.
Documentation Makes Updating the Literature Much Easier
Research projects age while you are working on them.
A dissertation may take several years. Peer review may add many months between manuscript submission and publication. Fast-moving topics can accumulate substantial new literature during that period.
If you preserved your important search strategies, updating is relatively straightforward. You can rerun or adapt them and restrict attention to material added since the previous search where the platform permits.
If you did not preserve them, you may need to reinvent the search and then wonder whether differences in retrieval reflect new literature or changes in your reconstructed query.
PRISMA-S explicitly treats search updates as something that should be reported in systematic reviews, including the methods used to update searches and the dates on which searches were last conducted.
Even when you are not conducting a systematic review, the practical lesson transfers well: today's search record is tomorrow's update strategy.
Documentation Helps You Judge Whether the Search Is Still Adequate
Search adequacy is difficult to assess retrospectively if you cannot remember what you searched.
Suppose you later decide that your topic has an important psychological dimension. Did you already search a psychology database? Did your earlier terminology represent that construct? Were those searches conducted before a major wave of new publications appeared?
Your documentation lets you audit the existing search rather than start from an impression.
This becomes especially useful when evaluating whether the literature search is good enough for the study you are actually conducting.
A search log cannot make an inadequate search adequate, but it can reveal where the weaknesses are.
Documentation Can Make Strong Claims Easier to Defend
The stronger your claim about the literature, the more useful a search record becomes.
Consider the difference between:
“Several studies have examined this issue.”
and:
“Very little research has examined this issue.”
The second statement implicitly makes a claim about what the literature does not contain.
If challenged, you may need to explain where you searched, which terminology you considered, whether adjacent disciplinary literatures were examined, and how recently the search was conducted.
A documented search gives you something more substantial than “I searched quite a lot and did not find much.”
Watch Out
Documentation does not transform an exploratory search into evidence that no relevant studies exist. It preserves what you did. The strength of any claim still depends on whether the search itself was capable of supporting that claim.
Keeping Records Does Not Make the Search Systematic
This misconception deserves explicit attention.
A researcher can meticulously record every Google Scholar query used during an exploratory search. The documentation may be excellent. The search can still remain exploratory.
Likewise, a reproducible search is not necessarily comprehensive or conceptually sound.
Documentation improves transparency and reproducibility. It does not retroactively change the purpose, design, or evidence coverage of the search.
This is why systematic, reproducible, and comprehensive searching are different properties.
You should document a non-systematic search because the record is useful, not because documentation earns the search a more impressive methodological label.
Do Not Report More Rigor Than You Actually Used
Good records make accurate reporting easier. They should not tempt you to redescribe an exploratory process as systematic.
Suppose you began with broad searches, followed interesting papers, changed terminology repeatedly, searched additional sources when they became relevant, and stopped once you had enough literature for your research purpose.
That may have been an entirely appropriate process.
If you kept careful notes, you can explain it accurately. You might report the principal databases used, broad search concepts, citation-searching approach, and the period covered if those details matter to the research output.
You should not claim that the evidence was systematically identified according to a predefined strategy if that is not what occurred.
Transparency is not the same as methodological inflation.
Systematic Reviews Require a Different Level of Documentation
When the search determines the evidence entering a systematic review, documentation moves from useful research housekeeping to a methodological requirement.
Cochrane requires review authors to document the sources searched, when searches occurred, who conducted them, and which terms were used, with sufficient detail for the searches to be reported correctly and reproduced to the extent possible.
PRISMA-S expands the reporting framework across 16 items and explicitly covers database and platform names, multi-database searching, registries, online resources, citation searching, full search strategies, limits and restrictions, prior search strategies, updates, search dates, peer review, record counts, and deduplication.
PRISMA-S is a reporting guideline rather than a tool for evaluating search quality or prescribing how a systematic review must be conducted.
That distinction matters. For a systematic review, consult the methodological requirements of the review approach alongside the reporting guidance that applies to the final output.
You Can Scale Documentation Instead of Choosing Between Everything and Nothing
The practical alternative to over-documentation is not no documentation.
Use a level appropriate to the consequences of the search.
A proportionate documentation framework
If the search is a quick lookup
Usually rely on your reference manager, notes, or browser history. A dedicated search log is rarely necessary.
If the search is exploratory and helping you learn a topic
Record useful terminology, important papers, productive sources, and searches you expect to revisit.
If the search materially shapes a thesis, dissertation, or empirical research project
Keep a more structured log of major databases, dates, consequential queries, citation searching, terminology changes, scope decisions, and important updates.
If the search determines the evidence set for a systematic or similarly structured review
Follow the documentation, conduct, and reporting requirements applicable to that methodology, preserving complete search details rather than relying on a lightweight personal log.
The objective is not bureaucracy. It is recoverability.
You should be able to recover enough of your search process to perform the next research task accurately, whether that means rerunning a productive query, explaining where the literature came from, checking whether the evidence is current, or producing the methods section months later.
