03 · What You Need to Know
Where Promising Research Ideas Commonly Break Down
A useful way to evaluate research idea feasibility is to stop asking only, “Why should I do this study?” and temporarily argue the opposite case. What would have to go wrong for this project to become unworkable, uninformative, or no longer worth the investment?
This is not pessimism. It is a form of pre-study scrutiny. Research funders and methodological guidance routinely place considerable weight on the rigor of the evidence supporting a proposed project, the appropriateness of its design, and the management of foreseeable risks. Feasibility research goes further by explicitly investigating uncertainties that could determine whether a larger study can or should proceed.
The Foundation of the Idea May Be Weaker Than It Appears
Most research ideas depend on previous evidence. That evidence may suggest that a problem exists, that two variables might be related, that an intervention could work, or that an unresolved question deserves further investigation.
But previous research is not equally dependable. An apparently persuasive rationale may rest on small studies, inconsistent findings, weak measurements, inappropriate comparisons, selective populations, uncertain analytical choices, or conclusions stronger than the underlying evidence warrants.
The National Institutes of Health, for example, asks researchers to assess the strengths and weaknesses in the rigor of prior research that provides key support for a proposed project. The principle extends beyond biomedical research: the existence of previous studies does not automatically mean that they provide a sufficiently strong foundation for the next one.
Ask a more demanding question than “Can I cite studies supporting this idea?” Ask whether the evidence you are relying on is strong enough to support the decisions your proposed study requires.
Your Idea May Depend on Assumptions That Have Never Been Tested
Every proposed study contains assumptions. Some are theoretical. Others are methodological or practical.
You might assume that participants understand a construct in roughly the way your instrument represents it. You might assume that institutional records contain the variables you need. You might assume that an intervention can be implemented consistently. You might assume that two groups can be meaningfully compared. You might assume that enough eligible participants will agree to participate.
None of these assumptions is necessarily unreasonable. The problem arises when an assumption is both uncertain and essential to the study's success.
A useful stress test is to identify the assumptions on which the proposed study depends and classify them by consequence. If an assumption turns out to be false, does the study merely become more difficult, require redesign, or become incapable of answering its question?
The Population May Exist but Still Be Inaccessible
A population being theoretically available does not mean you can recruit an adequate sample from it.
Recruitment can fail because eligibility criteria are restrictive, the population is difficult to contact, organizations will not provide access, potential participants have little reason to participate, the burden is too high, or recruitment takes much longer than anticipated. Attrition can create a second problem after enrollment begins.
This matters particularly when the design requires a specific sample size, multiple comparison groups, repeated measurements, hard-to-reach populations, or recruitment through gatekeepers.
Instead of treating recruitment as an administrative task that begins after approval, estimate what substantially lower recruitment would do to the study. If obtaining only half the expected sample would make the central analysis untenable, recruitment is not a peripheral issue. It is a design risk.
The Data May Not Be Good Enough to Answer the Question
Researchers sometimes confirm that data exist without establishing that those data are fit for the intended analysis.
An administrative database may contain thousands of records but have extensive missingness in the variable central to your question. Historical records may use inconsistent definitions across years. Sensor data may have substantial measurement error. Survey responses may exhibit severe ceiling effects. An existing dataset may contain convenient proxies rather than valid measurements of the constructs you actually want to study.
Quantity cannot rescue data that do not adequately represent what the research question requires.
Before committing to secondary or routinely collected data, investigate their provenance, definitions, completeness, measurement quality, coverage, and accessibility. Where uncertainty remains substantial, decide in advance what you would do if the available data prove poorer than expected.
The Method May Be Appropriate but Not Feasible
A method can be methodologically defensible and still be impractical in your particular setting.
Perhaps your preferred design requires random assignment that an institution will not permit. Maybe longitudinal follow-up extends beyond the available project period. Laboratory equipment is unavailable. Proprietary data are unaffordable. The analysis requires information that cannot ethically be collected. A qualitative design may require access to participants who cannot safely discuss the phenomenon being studied.
The relevant question is therefore not simply, “What is the best method for this question?” It is also, “Can I actually execute this method with sufficient rigor under the conditions I have?”
When a critical method is uncertain, determine whether a defensible alternative remains available if the preferred method cannot be used. If every substitute changes the research question or produces evidence incapable of addressing it, the methodological dependency deserves attention before the project advances.
The Study May Be Too Ambitious for the Available Resources
Feasibility includes more than money. Time, expertise, personnel, equipment, software, access, administrative support, and researcher workload can all constrain what can reasonably be done.
A doctoral student may propose three participant groups, interviews, surveys, classroom observations, longitudinal follow-up, and multilevel analysis. Each element might be defensible individually. Collectively, however, they may create a project whose execution exceeds the available time and capacity.
Scope becomes a scientific issue when insufficient resources cause shortcuts in recruitment, measurement, analysis, or follow-up. A smaller study executed rigorously may provide more credible evidence than an ambitious design implemented incompletely.
Ethical or Governance Requirements May Change What Is Possible
Some research ideas become substantially different once ethical and governance constraints are considered.
A proposed study may involve sensitive information that cannot be collected as originally envisioned. Data-sharing restrictions may prevent linkage between datasets. Researchers may discover that consent procedures alter the recruitment strategy. Vulnerable populations may require additional protections. Institutional or legal requirements may limit access to records, sites, or interventions.
Ethics should not be treated as a hurdle to clear after the scientific design is complete. Ethical acceptability and methodological feasibility can interact. If the study only works when protections are weakened, the problem lies with the design, not with the protections.
The Expected Relationship or Effect May Not Exist
Many studies are motivated by an expected association, difference, mechanism, or intervention effect. Researchers can become so focused on explaining why that result should appear that they spend less time considering what the project means if it does not.
A null or unexpected finding is not automatically a failed study. Under a rigorous design, it may challenge a theoretical expectation, narrow the range of plausible explanations, provide evidence against an assumed effect, or inform future research.
The more important pre-study question is whether the project remains interpretable when the relationship you expect does not appear. If the entire value of the project depends on obtaining one favored result, the rationale may be too fragile.
The Study May Work Perfectly and Still Add Too Little
Feasibility is not only about whether research can be completed. A study can recruit successfully, collect excellent data, use an appropriate method, and produce a technically sound manuscript while contributing little that was not already reasonably established.
This risk is especially important in mature research areas. A small replication in a nearly identical population using the same measures and design may add relatively little unless there is a meaningful reason to expect that the new setting, population, time period, or methodological improvement matters.
Before investing further, examine whether the existing evidence may already be sufficient for the question you intend to ask. Novelty alone is not the standard, because replication and confirmation can be valuable. The more useful question is what uncertainty will be reduced by adding this particular study.
A Simpler Study May Answer the Question Better
Complexity can sometimes masquerade as sophistication.
A researcher might design a large mixed-methods project when a carefully constructed analysis of existing data could answer the central question. Another might plan an experiment when the actual question is descriptive. Conversely, a survey may be proposed for a question that requires observation, experimentation, longitudinal evidence, or qualitative inquiry.
Before defending your chosen design, ask whether the research question could be answered more convincingly or efficiently through a different study. The goal is not to preserve the original design. It is to obtain evidence capable of answering the question.
Some Risks Are Fatal; Others Are Repairable
Not every weakness should make you abandon an idea. This distinction is essential.
| Potential problem |
Possible response |
When it becomes serious |
| Recruitment may be slower than expected |
Extend recruitment, add sites, revise feasible eligibility criteria, or reduce unnecessary participant burden |
The required population cannot realistically be reached in sufficient numbers |
| Some variables have missing data |
Assess the missingness, improve collection, revise measures, or use appropriate analytical approaches |
The missing information undermines measurement of a central construct or introduces unmanageable bias |
| Preferred method is unavailable |
Use a defensible alternative design or method |
Available alternatives cannot answer the research question credibly |
| Expected effect may be small or absent |
Design the study so that null or unexpected findings remain interpretable |
The project's claimed value depends entirely on obtaining a preferred result |
| Similar studies already exist |
Clarify the unresolved uncertainty or value of replication |
The new study is unlikely to change understanding, estimates, decisions, or confidence in existing evidence |
| Project is too large |
Narrow the question, population, measures, or design |
Reducing scope removes the study's ability to answer its central question |
The purpose of stress-testing is therefore not to eliminate every risk. That would eliminate most research along with it. The objective is to distinguish manageable uncertainty from dependencies that could invalidate the project.
04 · A Practical Example
Stress-Testing a Promising Study Before Committing to It
Hypothetical Example
A Digital Learning Intervention With a Strong Rationale
Suppose a researcher wants to evaluate whether an AI-supported feedback system improves university students' academic writing. Previous literature suggests that timely formative feedback can support writing development, and the university is interested in experimenting with AI-supported learning tools. The idea appears timely, theoretically plausible, and practically relevant.
The researcher initially plans a semester-long comparison between students using the system and students receiving conventional feedback.
Stress test 1: Evidence The researcher examines the literature more critically and finds that previous studies support formative feedback generally, but evidence about this particular type of AI-supported feedback is inconsistent. This does not kill the idea, but it changes how confidently the expected effect should be framed.
Stress test 2: Access Only two instructors are willing to participate, and one teaches substantially fewer students than anticipated. The planned sample may therefore be unrealistic.
Stress test 3: Implementation The university permits use of the system but does not allow students to upload identifiable coursework to an external service. The intervention must be redesigned to satisfy institutional data requirements.
Stress test 4: Measurement The researcher intended to compare final course grades. Further examination shows that instructors use somewhat different grading procedures. A common independently scored writing task may provide a more defensible outcome measure.
Stress test 5: Results The researcher asks what the study would contribute if the AI-supported group showed no meaningful improvement. The answer is still potentially useful if the study can estimate implementation feasibility, document how students actually use the feedback, and provide credible evidence about the proposed effect under clearly defined conditions.
Decision The original design is not ready, but the underlying research idea remains viable. The researcher narrows the study, revises the measurement strategy, resolves the data-governance problem, and treats uncertain recruitment as a major design constraint rather than hoping it will disappear.
The important outcome of this exercise is not that the initial proposal was “bad.” Stress-testing exposed the difference between a promising question and a study design ready to answer it.
06 · What This Means for You
Try to Break the Study Before You Commit to It
Once you become attached to a research idea, evaluation can quietly turn into advocacy. You search for evidence supporting it, explain why the question matters, and solve problems in ways that preserve the project you already want to conduct.
A more useful exercise is to temporarily reverse the burden of proof. Instead of asking how to make the idea work, identify the strongest reasons it might not. In particular, ask what the strongest argument against conducting the proposed study would be.
A simple decision framework
If a problem is unlikely and would have little effect on the study
Acknowledge it, but do not redesign the entire project around a remote possibility.
If a problem is plausible but manageable
Create a contingency plan before data collection begins.
If a critical uncertainty can be investigated in advance
Obtain preliminary information or conduct appropriate feasibility work before committing to the full study.
If a problem requires substantial redesign but the question remains worthwhile
Change the design rather than protecting the original plan.
If the study cannot answer the question under realistic conditions
Reframe or abandon the proposed study before investing further resources.
Separate Warning Signs From Stop Conditions
It may help to decide in advance what evidence would merely concern you and what evidence would cause you to stop.
For example, slower-than-expected recruitment may be a warning sign. Evidence that the entire accessible population is smaller than the minimum sample your design requires could be a stop condition. Some missing values may be manageable. Discovering that the database never collected the primary variable is different.
Making these distinctions before substantial investment can reduce the temptation to continually redefine what counts as acceptable. This becomes especially important once time, identity, funding, or supervisory effort has accumulated around a project. At that stage, the decision can become less about whether the study is still good and more about whether you are willing to let go of what you have already invested.
For high-risk ideas, explicitly define what evidence would make you abandon or substantially redesign the project. A stop condition is not an admission of weak commitment. Sometimes it is evidence that the research decision itself has been designed carefully.
Watch Out
Do not turn stress-testing into a requirement that a study must be guaranteed to succeed. A project whose result is genuinely uncertain may be scientifically valuable. The risks that deserve particular attention are those that threaten your ability to conduct the study rigorously, interpret the evidence, answer the research question, or make a meaningful contribution.