03 · What You Need to Know
The Identity of a Study Comes From More Than Its Topic
Researchers sometimes assume that two questions belong to the same study because both concern the same general topic.
For example:
“How do students experience institutional rules governing generative AI?”
and:
“Does generative AI use improve students' writing performance?”
Both concern students and generative AI. Yet the first seeks an understanding of experiences surrounding institutional rules. The second asks about an outcome and potentially a causal effect. They require different evidence and probably different designs.
Topic continuity is therefore a weak test of study continuity.
Same topic
The revised question remains within the same broad subject area.
Same study
The revised question preserves enough of the original scientific purpose, population, phenomenon, evidence, and methodological logic that the existing study remains an appropriate way to answer it.
Minor wording changes usually do not create a new study
Suppose the original question is:
“How do students experience AI use in academic writing?”
You revise it to:
“How do undergraduate students experience using generative AI in academic writing?”
If the study always concerned undergraduate students and generative AI, the revision may simply make previously implicit boundaries explicit.
The participants do not change. The phenomenon does not change. The interview protocol remains appropriate. The analysis still seeks the same kind of understanding.
This is refinement rather than reinvention.
Narrowing the population does not automatically create a different study
Suppose your original question concerns “university students,” but feasibility analysis shows that the study can credibly recruit only first-year undergraduate students.
You revise the question accordingly.
Whether this constitutes a new study depends partly on why the population matters. If first-year students are simply a more precisely defined subset of the population the study always intended to investigate, the change may be manageable.
If the phenomenon operates fundamentally differently among first-year students and the theoretical rationale must now be reconstructed around transition to university, the change is more substantive.
The label of the population matters less than what changing it does to the scientific rationale and intended generalization.
Changing the population entirely is a stronger signal
Consider:
Original question:
“How do students interpret university guidance concerning generative AI?”
Revised question:
“How do faculty members develop course-level rules concerning generative AI?”
The topic remains AI policy. Almost everything else has shifted.
The participants are different. The phenomenon has changed from interpretation to policy development. The literature may change. The interview questions change. The sampling strategy changes. The eventual claims concern faculty rather than students.
Calling this a minor revision because “both are about AI rules” would obscure the magnitude of the change.
Changing the central phenomenon is one of the clearest boundaries
Suppose a qualitative study begins by investigating students' experiences of AI-related academic-integrity accusations.
During fieldwork, participants frequently discuss anxiety. The researcher becomes interested in student mental health and changes the question to:
“How do university students experience anxiety during assessment?”
Anxiety may have emerged from the original phenomenon, but it has now become the central phenomenon itself.
The revised study might require a different literature, ethical risk assessment, sampling logic, interview protocol, and analytical focus.
At that point, the question may no longer be a refinement of the AI-integrity study. It may be a new study inspired by it.
Changing from description to causation can create a fundamentally different study
Compare:
“How frequently do students use generative AI for academic writing?”
and:
“What is the effect of generative AI use on students' independent writing performance?”
The first is descriptive. The second is causal.
The original survey might be perfectly capable of estimating reported prevalence while being wholly inadequate for estimating a causal effect.
Moving from one question to the other changes the inferential target, comparison, data requirements, design assumptions, analysis, and possibly sample-size requirements.
This is why earlier guides distinguished questions about effects from questions about associations or description. A change in inferential level can be a change in the study itself.
Changing from association to prediction is also substantive
Suppose the original question asks:
“Is academic self-efficacy associated with university withdrawal?”
The revised question asks:
“Can academic self-efficacy predict which students will withdraw from university?”
These questions may use some of the same variables, but prediction has different methodological requirements. A predictive study should evaluate how well a model predicts outcomes, ideally in data not used simply to fit the model, and must address issues such as overfitting, calibration, discrimination, and validation as appropriate.
A regression coefficient demonstrating association is not automatically a prediction model.
Thus, even when the variables remain unchanged, the scientific job can change.
Changing the primary outcome can change the study
Imagine a trial designed around:
“Does AI-supported feedback improve independently assessed writing performance?”
Writing performance determines the primary outcome, sample-size calculation, intervention evaluation, and principal analysis.
Halfway through the study, the question becomes:
“Does AI-supported feedback improve students' satisfaction with writing instruction?”
Satisfaction may already be a measured secondary outcome, but promoting it to the central question changes what the study is principally trying to establish.
In confirmatory research, changes to prespecified primary outcomes are particularly consequential because they can alter the interpretation of the study and create concerns about selective outcome reporting when made after outcome data are known. Reporting standards such as CONSORT require important changes to trial outcomes after commencement to be identified and explained.
A new outcome does not always mean a completely new study
Context matters.
If an instrument used for the original outcome becomes unavailable and researchers replace it with a validated measure of the same construct before relevant outcome data are examined, the scientific question may remain essentially unchanged.
If the construct itself changes from writing performance to student satisfaction, the scientific target has changed.
Ask whether the revised measure is another way of observing the same intended outcome or whether you are now interested in a different outcome altogether.
Changing the comparison can change the causal question
Causal effects are defined relative to alternatives.
Suppose your original question asks whether AI-supported feedback improves writing compared with instructor feedback alone.
The revised question compares AI-supported feedback with no feedback.
The intervention may be unchanged, but the causal contrast is different. The resulting effect answers a different decision problem.
If participants have already been allocated according to the original comparison, the existing design may not even contain the data needed for the revised question.
Changing the time horizon can alter the phenomenon being studied
Consider:
“Does AI-supported practice improve writing performance immediately after instruction?”
versus:
“Does AI-supported practice improve independent writing performance one year later?”
The intervention is the same. The population may be the same. The outcome construct may appear similar.
But the second question concerns retention or longer-term development and requires a very different follow-up structure. Attrition becomes more important, subsequent exposures may matter, and the interpretation changes.
Extending a question from immediate performance to long-term development can therefore turn a manageable modification into a substantially different study.
Changing the setting may or may not create a different study
Moving recruitment from one campus to another does not necessarily change the scientific question.
But setting can be theoretically constitutive of the phenomenon.
A study of how students navigate a university with a permissive AI policy may not be equivalent to a study conducted in an institution that prohibits generative AI entirely. If policy environment is central to the phenomenon, changing the setting can change what the question means.
The relevant test is whether setting is merely where the study occurs or part of what the study is about.
Changing the data source can sometimes preserve the study
Suppose you intended to obtain AI-use records from a platform, but access is withdrawn. You replace them with institutional system logs that capture the same relevant behavior with adequate validity.
The data source has changed, but the research question may remain intact.
Now suppose you replace behavioral logs with students' perceptions of how frequently they use AI.
The study may have shifted from observed behavior to self-reported behavior. That can still be valuable, but the evidentiary meaning is different.
Whether the study remains the same depends on whether the new source can support the interpretation implied by the original question.
Changing the method does not automatically mean a different study
Methodological changes can occur while the scientific question remains stable.
Suppose you planned individual interviews but discover that participants can be recruited more effectively for focus groups. If the research question concerns shared interpretations and the group format remains methodologically appropriate, the study may preserve its central inquiry.
Likewise, a quantitative analysis may need to change because the original model assumptions are untenable while the target estimand remains unchanged.
Methods serve questions. Replacing one method with a better method for answering the same question does not necessarily create a new study.
But a method change can reveal that the question has changed
Suppose you began with:
“How do students experience institutional AI policies?”
You planned interviews.
You later replace the entire design with an experiment manipulating policy wording and measuring compliance behavior.
The method has not simply changed. The scientific purpose has shifted from understanding experience to estimating a behavioral effect of an intervention.
The methodological change is evidence of a deeper question change.
A useful diagnostic is: Would a reasonable researcher have chosen the original method if the revised question had been the question from the beginning?
If the answer is clearly no, the revised project may be a different study.
The “design-from-scratch” test is particularly useful
Imagine temporarily forgetting everything you have already collected.
Take the revised research question and ask:
“If I were starting today with no sunk costs, what study would I design to answer this question?”
Then compare that ideal revised design with the study you are currently conducting.
If you would choose essentially the same population, sampling strategy, evidence, measurements, time frame, and analysis, you may still be refining the same study.
If you would design something fundamentally different, continuing to describe the project as an amendment to the original study deserves scrutiny.
The Design-From-Scratch Test
If the revised question had been your original question, would you have designed approximately the study you are conducting now? If not, the question may have moved beyond the study that was built to answer it.
The “same evidence” test provides another clue
Ask whether the data already collected under the original question are still directly relevant to the revised one.
If nearly all existing data remain appropriate, that supports continuity.
If most existing data become irrelevant and entirely new data must be collected, that suggests discontinuity.
This is not an absolute rule. A new analysis of existing data can sometimes address a genuinely new research question. Secondary-data studies do this routinely.
The issue is whether the ongoing project still represents the study that originally generated those data.
The “same conclusion” test can reveal a changed inferential target
Imagine the final sentence of the study.
Original intended conclusion:
“Students commonly experience inconsistent expectations about acceptable generative AI use across courses.”
Revised intended conclusion:
“Providing explicit AI-use guidance increases students' compliance with institutional policy.”
The conclusions make different kinds of claims. One describes experience; the other attributes a behavioral change to an intervention.
If the kind of conclusion the study is designed to support changes substantially, the scientific identity of the study may have changed as well.
The “same literature” test is useful, but not decisive
If changing the question requires replacing most of the theoretical and empirical literature supporting the study, that is another signal.
A shift from students' policy experiences to AI effects on writing performance would require substantially different scholarship.
However, literature overlap alone is not enough to determine study identity. Related questions can draw on much of the same literature while requiring different designs, and one question may draw on several literatures without becoming several studies.
Use this test alongside methodological ones rather than on its own.
The “same participants” test is also insufficient by itself
You can ask entirely different questions using the same participants.
A cohort of students might provide data about academic achievement, mental health, technology use, social relationships, and career intentions.
Shared participants do not turn all possible analyses into one study.
Likewise, changing participants does not necessarily create a new study if the target population remains conceptually the same and recruitment merely expands to another site.
Study identity is determined by the combination of scientific purpose and methodological structure, not by one element alone.
Several small changes can cumulatively create a different study
No individual revision may look dramatic:
You broaden the population slightly. Then you change the primary outcome. Then you extend the follow-up period. Then you add a comparison group. Then you replace the original analysis.
Each amendment may appear defensible in isolation.
Collectively, the final study may bear little resemblance to the original protocol.
This is why protocol-amendment guidance emphasizes documenting important changes rather than considering each one without reference to the study as a whole. SPIRIT 2025 includes explicit attention to protocol amendments and their rationale.
Periodically compare the current study with the original one rather than evaluating amendments only one at a time.
Administrative definitions of a “new study” may differ from scientific ones
There are at least two questions here:
Scientifically, has the inquiry become a different study?
Administratively or ethically, does the change require a new protocol, registration, amendment, or approval?
Those are related but not identical.
An ethics committee may permit a substantial modification as an amendment rather than require a new application. Conversely, an institution may require a new submission for changes that a researcher considers conceptually modest.
Researchers should therefore follow the requirements of the relevant institution, ethics committee, funder, registry, sponsor, or journal rather than assuming that a methodological judgment automatically determines the administrative classification.
Ethical continuity matters
If the research purpose changes substantially, ask whether participants consented to what the study has become.
Suppose participants agreed to interviews about educational technology experiences. The revised study now investigates highly sensitive mental-health histories.
The fact that the same participants are available does not mean the original consent automatically covers the new inquiry.
Changes affecting risks, privacy, data use, procedures, or the nature of participation may require amendment, additional consent, or another institutional process.
The appropriate response depends on the study and applicable ethical requirements.
Preregistration makes the boundary more visible
A preregistration creates a timestamped record of what the study intended to test or analyze before outcomes were known.
If the question changes later, the original plan remains visible.
This does not prohibit change. The Center for Open Science explicitly distinguishes preregistered confirmatory analyses from exploratory analyses and encourages transparent reporting of deviations from preregistered plans.
A substantial new question can therefore be studied using the existing data, but it should be identified according to how and when it arose rather than being retroactively inserted into the preregistered study.
A different study can emerge from the same dataset
This distinction is especially important.
One dataset can support several studies.
A longitudinal educational dataset might be used for one study of dropout prediction, another study of socioeconomic inequalities, and another study of academic trajectories.
The studies may share participants, variables, and data collection history while asking different scientific questions.
Therefore, concluding that a revised question constitutes a different study does not necessarily mean you must discard the existing data.
It means the new analysis should be conceptualized, justified, and reported as a distinct inquiry when appropriate.
A new study does not always require new data
Secondary analysis is a perfectly legitimate research strategy.
You may discover a new question after the original study ends and find that the existing dataset contains appropriate evidence to investigate it.
The question can become the basis of another study using the same data.
The important distinction is provenance: the data were originally collected under one study, while the new analysis addresses another question. Ethical permissions, data-use agreements, preregistration, multiplicity, and other considerations may still apply.
“New study” should not be confused with “new data collection.”
Qualitative studies require a more interpretive boundary
In iterative qualitative research, question refinement can be part of the intended methodology.
Early interviews may challenge the researcher's assumptions. Emerging analysis may shift attention toward dimensions of the phenomenon that were not anticipated. Theoretical sampling may deliberately pursue emerging concepts.
As discussed in changing a research question after data collection has begun, such evolution can be methodologically appropriate.
The boundary is crossed when the revised question no longer develops understanding of the original central phenomenon but instead substitutes another phenomenon or purpose.
For example:
Original: “How do doctoral students experience uncertainty while developing their dissertation research?”
Refined: “How do doctoral students manage uncertainty in relationships with supervisors during dissertation development?”
This could plausibly be a focused development of the original inquiry.
Changed: “How do supervisors evaluate doctoral students' methodological competence?”
The participants, perspective, phenomenon, and purpose have shifted enough that a separate study may be more appropriate.
Mixed-methods studies can contain distinct component studies without losing coherence
A mixed-methods project may intentionally contain a quantitative study and a qualitative study that ask different but connected questions.
For example, one phase estimates patterns of AI use and another investigates how students explain those patterns.
Each component can have its own question and method while remaining part of a larger mixed-methods investigation because their integration addresses an overarching problem.
This complicates the idea of “a different study.” A revised question may constitute a new component study without becoming an unrelated research project.
The relevant issue is whether the new component was deliberately integrated into the larger design or whether it represents an independent inquiry added because interesting data became available.
Changing the question after seeing results does not make the new study invalid
Suppose an unexpected pattern appears in your data and inspires a new research question.
That new question may be scientifically important.
The appropriate response is not to pretend you never saw the result. You can conduct an exploratory analysis, label the question as data-generated, develop a new study using independent data, or preregister a confirmatory analysis in another dataset.
Discovery and confirmation are both legitimate parts of science. The problem arises when the distinction between them is erased.
Sometimes starting a new study is the cleaner methodological choice
Researchers can become reluctant to call something a new study because they have already invested time in recruitment, data collection, ethics applications, or analysis.
Those sunk costs should not determine scientific classification.
If the revised question genuinely requires different participants, evidence, measurements, methods, or inference, designing a new study around it may produce stronger research than repeatedly modifying an existing protocol that was built for another purpose.
A question worth studying is often worth studying with a design actually constructed to answer it.
A practical continuity audit can help
When you are unsure whether the project remains the same study, compare the original and revised versions across several dimensions.
| Dimension |
Ask |
Stronger signal of a different study |
| Purpose |
What is the study fundamentally trying to learn? |
The principal scientific purpose changes |
| Population |
Whom or what will the conclusions concern? |
A substantially different target population is introduced |
| Phenomenon or exposure |
What central thing is being investigated? |
The central phenomenon or exposure changes |
| Outcome |
What result or phenomenon constitutes the answer? |
A different primary construct or outcome becomes central |
| Comparison |
What conditions, groups, or alternatives are being contrasted? |
The scientific contrast changes materially |
| Time |
What temporal process or horizon matters? |
The revised question requires a substantially different follow-up or temporal structure |
| Evidence |
What data would answer the question? |
Most original data are inadequate and new evidence is required |
| Method |
What design would you choose from scratch? |
You would choose a fundamentally different design |
| Inference |
What kind of conclusion is intended? |
The study shifts among description, association, prediction, explanation, or causation |
| Ethics |
What did participants consent to and what risks are involved? |
The revised study introduces materially different participation, risks, or data uses |
No single row functions as an automatic verdict. The pattern across rows is what matters.
The question is ultimately about continuity of scientific purpose
A study can survive many procedural changes while preserving its scientific purpose.
It can also retain the same participants, questionnaire, and topic while becoming a different study because the question and intended inference have changed.
The central test is therefore not administrative similarity. It is whether the revised project is still trying to answer essentially the same scientific question using a design appropriate to that question.
