03 · What You Need to Know
Direct and Conceptual Replication Ask Different Evidential Questions
Direct Replication Tries to Recreate the Relevant Conditions of the Original Test
A direct replication attempts to repeat an earlier study as closely as possible in the respects believed to matter for producing and measuring the original finding. Depending on the study, that might include the experimental manipulation, instruments, instructions, outcome measures, sampling criteria, timing, analytical procedures, or other methodological features.
The objective is not literal duplication of every circumstance. That standard would usually be impossible. Different participants must be recruited, time has passed, researchers may differ, and laboratories, classrooms, clinics, communities, or technologies may not be identical.
For this reason, direct replication is often better understood as preserving the methodological features considered necessary for an informative retest rather than reproducing every incidental feature of the original study. Even the term itself does not have a universally agreed technical definition, and the boundary between direct and conceptual replication can be fuzzy.
The central question is approximately:
If we conduct an appropriately similar test again with new data, does the finding recur?
This makes direct replication particularly useful when uncertainty concerns the reliability of a specific empirical finding under conditions close to those originally studied.
Conceptual Replication Deliberately Tests the Idea in Another Way
A conceptual replication changes one or more meaningful aspects of the original method while attempting to test the same underlying hypothesis, theoretical construct, or phenomenon.
The operationalization might change. A different measure might represent the same construct. Researchers might use another manipulation, task, design, population, or context. In some conceptual replications, several of these features differ simultaneously.
The question therefore shifts toward something like:
Does the underlying claim continue to receive support when we test it in a meaningfully different way?
This can provide evidence that a finding is not tied exclusively to one particular operationalization or methodological arrangement. It can also probe the generality of the theoretical interpretation behind an original result.
| Question |
Direct Replication |
Conceptual Replication |
| Primary concern |
Whether the finding recurs under closely similar methodological conditions |
Whether the underlying idea, hypothesis, or phenomenon survives a different test |
| Method |
Preserved as closely as feasible in relevant respects |
Deliberately changed in one or more meaningful respects |
| Operationalization |
Usually kept similar |
May be intentionally different |
| Interpretive strength |
Closer methodological similarity can make discrepancies easier to relate to the original finding |
Successful results can support broader generality, but discrepant results may have more competing explanations |
| Typical question |
"Does this result occur again?" |
"Does this broader claim hold when tested differently?" |
Direct Does Not Mean Literally Exact
The phrase "exact replication" is sometimes used interchangeably with direct replication, but it can create an unrealistic impression.
No empirical study can reproduce every feature of an earlier investigation exactly. New participants are already a difference. Researchers may work at another time and location. Equipment or software may have changed. Even seemingly minor contextual conditions can differ.
Researchers therefore need to distinguish changes that are unavoidable or plausibly irrelevant from changes that alter the evidential meaning of the replication.
Suppose an original experiment used a standardized questionnaire administered immediately after an intervention. A direct replication might use the same intervention, questionnaire, instructions, timing, eligibility criteria, and analysis with a new sample. Conducting the study in another laboratory would not automatically prevent it from being considered direct if the relevant procedures remain sufficiently comparable.
Conversely, replacing the intervention, changing the outcome construct, and recruiting a theoretically distinct population while calling the study "direct" simply because the research topic remains similar would obscure substantial differences.
Conceptual Does Not Mean You Can Change Anything You Want
Conceptual replication permits deliberate methodological variation, but the changes still require theoretical logic.
If you replace one measure with another, both should defensibly operationalize the construct relevant to the claim. If you use another manipulation, it should provide a meaningful alternative way of instantiating the theoretical variable. If you change the population, the new population should help test the intended scope or generality of the claim.
Otherwise, the new study may no longer test the same proposition.
This is an important distinction because superficial similarity of topic does not establish replication. Two studies can both concern academic motivation, artificial intelligence, anxiety, leadership, or social media while asking fundamentally different questions.
Changing the population is particularly easy to misclassify. Whether that modification constitutes conceptual replication, another form of replication, an extension, or a new study depends on what the population difference is intended to test. The relevant reasoning is therefore more nuanced than simply declaring that a different population makes the study new.
The Most Important Difference Is What a Discrepant Result Would Mean
Imagine that a direct replication closely reproduces the original procedure but does not obtain a similar result. Interpretation still requires care, but the methodological similarity reduces some alternative explanations. Researchers can ask whether the original effect is less reliable than previously believed, whether sampling variation explains the discrepancy, or whether seemingly minor differences actually matter.
Now imagine a conceptual replication that changes the manipulation, measure, population, and context and does not obtain the predicted effect.
What failed?
Perhaps the underlying claim does not generalize. But perhaps the new manipulation did not adequately instantiate the theoretical construct. Perhaps the alternative measure captured something different. Perhaps the population introduced a boundary condition. The methodological differences that make conceptual replication informative about generality can also make discrepant findings harder to interpret.
Watch Out
A conceptual replication should not be considered supportive when it produces the expected result but dismissed as "not really comparable" whenever it does not. Before collecting data, specify what outcomes would increase or decrease confidence in the claim and which methodological differences could reasonably alter that interpretation.
This Is Why Some Scholars Disagree About the Term Conceptual Replication
The direct-versus-conceptual distinction is widely used, but replication terminology is not completely settled.
Nosek and Errington propose defining replication by its relationship to a prior claim rather than by methodological similarity alone. Under their account, a study is a replication when its possible outcomes would be diagnostic of evidence supporting the prior claim. If a result consistent with the original claim would count as support but an inconsistent result would simply be dismissed because the methods differed, the study provides an asymmetrical test and may not function as a replication in a strong evidential sense.
This critique is particularly relevant to conceptual replication. A researcher might test a theory with a substantially different method and describe a positive result as replication. But if a negative result could not reasonably lower confidence in the original claim because the new operationalization is too different, calling the study a replication becomes debatable.
Other scholars use conceptual replication more broadly to describe testing the same theoretical idea with different methods. There is therefore no single taxonomy accepted across every discipline.
For your own research, the safest approach is not to rely on the label to do the explanatory work. Describe the methodological relationship to the original study explicitly.
Direct and Conceptual Replication Are Better Viewed as a Continuum Than Perfectly Separate Boxes
Real studies do not always fit neatly into two categories.
A replication may retain the original intervention and outcome measure but recruit a somewhat different sample. Another may preserve the procedure but update outdated stimuli. Another may use the same constructs while deliberately changing both the manipulation and measurement.
Educational replication research, for example, has also used an intermediate category such as approximate replication for studies that largely reproduce an original investigation but modify one or more substantive aspects. Other taxonomies use terms such as close replication, exact replication, or constructive replication.
Rather than spending excessive effort deciding which side of a terminological border your study occupies, report the actual similarities and differences. A reader should be able to understand what was held constant, what changed, and why.
Your Choice Should Follow the Uncertainty You Want to Resolve
If you doubt whether a particular result is replicable even under conditions closely resembling the original study, deliberately changing several important methodological features may be premature. A direct or close replication provides a cleaner test of that concern.
If there is already reasonable evidence that the effect recurs under similar conditions, a more conceptually varied test may provide greater information about whether the underlying claim extends beyond one methodological implementation.
Match the design to the question
If your question is "Will this particular finding occur again under closely comparable conditions?"
A direct or close replication is usually the more natural design.
If your question is "Does the underlying hypothesis survive another operationalization?"
A conceptual replication may be more appropriate, provided the alternative operationalization genuinely tests the same claim.
If your question is "Does the claim generalize to a theoretically meaningful new condition?"
A deliberately varied replication may be useful, but specify exactly what aspect of generality the change is intended to test.
If your question adds a new theoretical relationship or outcome
You may be moving beyond replication toward an extension or a different study.
This last boundary deserves attention. Researchers sometimes call almost any study inspired by earlier work a conceptual replication. Yet introducing an additional mechanism, outcome, moderator, or hypothesis may create a new inferential target. In that case, understanding the difference between replication and extension may be more useful than stretching the term conceptual replication to cover the entire project.
06 · What This Means for You
Define the Inferential Target Before You Design the Replication
Do not begin by deciding that your project "will be a conceptual replication" and then search for features to change. Begin with the uncertainty in the literature.
Write the original claim in one sentence. Then write the question your proposed study is intended to answer. If those two sentences are vague, classifying the replication will not solve the underlying problem.
Next, identify the features of the original study that are relevant to the claim. For each planned difference, explain whether it is unavoidable, methodologically beneficial, or deliberately introduced to test generality. This makes the logic of the design visible.
A simple decision framework
If confidence in the original empirical result is itself uncertain
Keep the replication sufficiently close that the new result provides a clear test of whether the finding recurs.
If the original result already appears reproducible but may depend on one operationalization
Consider deliberately changing the operationalization while preserving the theoretical claim you intend to test.
If you deliberately change the population or context
State what aspect of generalizability or boundary conditions the change is intended to examine.
If several substantive elements must change because the original method cannot be implemented
Evaluate whether the resulting study can still provide interpretable evidence about the original claim before assigning a replication label.
If your new design primarily introduces an additional theoretical claim
Consider describing the project as an extension, or as containing separate replication and extension components.
You should also be explicit about methodological departures. If materials, procedures, measures, populations, or analyses differ, readers should not have to reverse-engineer those differences from two methods sections.
Most importantly, decide in advance what a result consistent with the original finding would mean and what an inconsistent result would mean. If only the confirming outcome can be interpreted as evidence about the original claim, you may not have designed a sufficiently diagnostic replication.
When practical constraints prevent close reproduction of important features, acknowledge them rather than quietly relabeling the study. The next question is then whether a useful replication remains possible despite those differences.
