01 · The Question
A new rule changes the environment. Is that a research opportunity?
A government introduces a new regulation. A university changes its admissions policy. A health authority modifies clinical requirements. A professional body issues new standards. An organization adopts rules governing artificial intelligence, privacy, workplace arrangements, assessment, or another area of practice.
For researchers working near that field, the change can immediately suggest possibilities. You might compare outcomes before and after implementation, study how affected groups respond, examine whether the policy achieves its objectives, or investigate consequences that policymakers did not anticipate.
Those possibilities can support strong research, but the existence of a new policy is not itself the research problem. The opportunity arises because a policy or regulation changes conditions in ways that create uncertainty about implementation, behavior, outcomes, distributional effects, mechanisms, or unintended consequences.
03 · What You Need to Know
The policy creates the change; your research must identify the uncertainty
Begin with what the policy actually changes
Policies and regulations can alter behavior through many routes. They may prohibit or require particular actions, change financial incentives, redistribute resources, establish eligibility, create reporting requirements, expand or restrict rights, change institutional responsibilities, or establish standards that organizations must follow.
Your first task is therefore to understand the intervention itself. Read the authoritative policy or regulatory text rather than relying solely on media coverage, organizational summaries, or commentary. Identify who is covered, what is required or permitted, when implementation begins, whether exemptions exist, which institutions are responsible, and how compliance is expected to occur.
Then ask:
Because this policy changes ________, we do not yet know ________.
The second blank is the potential research problem.
A new policy can generate several different kinds of questions
Policy evaluation is broader than asking whether a policy “worked.” Evaluation can examine the design, implementation, outputs, outcomes, impacts, efficiency, relevance, and sustainability of public interventions. The appropriate question depends partly on how mature the policy is and what decision the evidence is meant to inform.
| Research direction |
Illustrative question |
| Implementation |
Was the policy implemented as intended across the organizations or populations it covers? |
| Adoption or compliance |
How consistently did affected individuals or organizations change their behavior in response? |
| Effectiveness |
Did the policy produce the intended outcome? |
| Mechanism |
Through what behavioral or institutional process did the policy produce its effects? |
| Heterogeneous effects |
Did the policy affect some populations, organizations, or locations differently from others? |
| Equity |
How were benefits, burdens, opportunities, or risks distributed? |
| Unintended consequences |
Did the policy produce important outcomes that were not part of its stated objectives? |
| Sustainability |
Did effects or implementation persist after the initial transition period? |
You do not need to study all of these. A focused study usually becomes stronger when it identifies the uncertainty most relevant to the policy and its stage of implementation.
Implementation and effectiveness are different questions
Suppose a new regulation requires organizations to adopt a particular procedure. Six months later, the intended outcome has not improved. It would be premature to conclude that the policy itself is ineffective without asking whether the procedure was actually implemented.
Organizations may interpret the requirement differently. Staff may receive inadequate training. Resources may be insufficient. Compliance may be superficial. Local adaptations may alter the intervention substantially.
Policy evaluation therefore benefits from separating questions about what was implemented from questions about what effects followed. Otherwise, implementation failure can be mistaken for failure of the underlying policy logic.
Before-and-after differences do not automatically establish policy effects
A policy begins in January. An outcome improves by June. Did the policy cause the improvement?
Possibly, but time alone does not establish causation. Other events may have occurred simultaneously. Long-term trends may already have been moving in the same direction. The composition of the population may have changed. Organizations may have anticipated the policy and altered behavior before formal implementation.
Credible policy-effect research therefore requires a design appropriate to the causal question. Depending on the circumstances, researchers may use comparison groups, interrupted time-series designs, difference-in-differences approaches, regression discontinuity designs, or other quasi-experimental and experimental strategies. Each design rests on assumptions that must be considered rather than invoked simply because policy implementation produced a convenient date on the calendar.
Watch Out
“The outcome changed after the policy” and “the outcome changed because of the policy” are different claims. If your question is causal, your design must address plausible alternative explanations for the observed change.
Policy changes can create natural or quasi-experimental opportunities
Some policy changes introduce variation that researchers did not create. A regulation may apply only above an eligibility threshold. Different jurisdictions may adopt similar policies at different times. One population may be covered while another comparable population is not.
Such circumstances can sometimes support quasi-experimental research because the policy creates contrasts that help researchers estimate effects. But the presence of a comparison does not automatically make the design credible. Researchers still need to examine assumptions such as comparability, pre-existing trends, concurrent interventions, spillovers, anticipation, and how exposure to the policy is defined.
The policy change provides an opportunity. Research design determines how much can reasonably be learned from it.
The intended outcome is only part of the policy story
Policies can change behavior in ways their designers did not anticipate. Organizations may reorganize activities to comply formally while avoiding the intended substantive change. Costs may shift to another group. A regulation designed to increase safety may increase administrative burden. A policy intended to expand access may benefit some populations more than others.
Ex-post regulatory evaluation is valuable partly because actual outcomes and unintended consequences become visible only after implementation. These effects can provide research questions that were difficult to formulate before the policy operated in practice.
An unexpected consequence can also become the kind of observation that generates a separate research idea, provided the phenomenon survives appropriate scrutiny.
Policy effects can vary across contexts
The same formal policy may operate differently across schools, hospitals, municipalities, firms, universities, or other organizations. Implementation capacity, local leadership, resources, infrastructure, population characteristics, enforcement, and institutional culture may shape what happens after adoption.
Variation should not automatically be dismissed as noise. If policy outcomes differ systematically, the stronger research question may concern the conditions under which the policy succeeds or fails.
This is particularly important when findings from early evaluations conflict. Rather than asking which evaluation is correct, you may need to investigate why studies of apparently similar policies produce different results.
A policy can change professional practice before its ultimate outcomes are measurable
Some policy effects unfold slowly. A regulation may immediately change documentation requirements, decision processes, staffing responsibilities, or professional judgment while its intended population-level outcomes take years to become observable.
Those intermediate changes can themselves be legitimate objects of research when they matter to understanding implementation or mechanisms. If your central interest shifts toward how practitioners adapt their work, the more specific opportunity concerns a change in professional practice.
The policy's stated objective should not dictate your conclusion
Policies are often introduced with explicit rationales: improve quality, reduce harm, increase access, encourage innovation, protect privacy, lower costs, or change behavior. Those objectives help define what should be evaluated, but researchers should not assume that implementation will produce the intended causal pathway.
Policy evaluation is useful precisely because stated objectives and actual outcomes can diverge. The researcher's task is to evaluate evidence, not to reproduce the policy's theory of success as though it were already established.
Timing matters when choosing the research question
A policy announced yesterday may be suitable for research on expectations, preparedness, implementation planning, or early responses. It may be far too early for credible conclusions about long-term effects.
Later, different questions become possible. Researchers can examine implementation fidelity, behavioral adaptation, intermediate outcomes, longer-term effects, sustainability, and unintended consequences.
A good question therefore fits not only the policy but also where the policy is in its lifecycle.
04 · A Practical Example
From a new university policy to an evaluable research question
Hypothetical Example
A university introduces mandatory disclosure of generative AI use
A university adopts a policy requiring students to disclose specified uses of generative AI in assessed work. The stated objective is to promote transparency and responsible use while allowing appropriate applications of AI. A researcher initially proposes studying students' attitudes toward the new policy.
Identify the change Students are now required to make aspects of their AI use visible that previously may not have been formally reported.
Identify possible mechanisms Disclosure requirements could influence students' willingness to use AI, the ways they use it, how they document use, their understanding of acceptable practice, or their willingness to report use accurately.
Identify implementation uncertainty Different instructors may interpret acceptable disclosure differently, potentially changing how students experience the same formal policy.
Check existing evidence The researcher reviews scholarship on academic integrity, disclosure, AI use, compliance, assessment practice, and policy implementation rather than assuming that a new institutional policy has no relevant evidence base.
Refine the question The researcher asks how consistency in instructors' implementation of AI-disclosure requirements is associated with students' understanding of acceptable AI use and their reported disclosure behavior.
Choose appropriate evidence The study combines policy implementation information with evidence about student understanding and behavior rather than treating attitude toward the policy as a proxy for whether it works.
The policy created the changing environment. The research opportunity emerged from uncertainty about how implementation might shape the behavior the policy was intended to influence.
07 · A Quick Checklist
Before building a study around a new policy or regulation
Before committing to the research idea, check:
Read the authoritative policy or regulatory document and identify exactly what changed, for whom, and when.
Identify the policy's stated objectives without assuming that its intended causal mechanism is correct.
Search for evidence on comparable policies, predecessor regulations, and the mechanisms underlying the intervention.
Decide whether your question concerns implementation, compliance, effectiveness, mechanisms, equity, sustainability, or unintended consequences.
Check whether enough time has passed for the outcome you want to study reasonably to occur.
If making a causal claim, identify plausible alternative explanations and choose a design capable of addressing them.
Examine whether implementation varies across organizations, jurisdictions, or affected populations.
Consider important unintended and distributional effects rather than studying only the policy's intended average outcome.
State what decision or understanding would improve if your research resolved the identified uncertainty.