03 · What You Need to Know
Why Realistic Research Design Is Not the Same as Settling for Weak Research
There is almost always a better study than the one you can conduct
Imagine that you have designed a well-powered longitudinal study across five universities. Someone could reasonably suggest ten universities. If you plan twelve months of follow-up, three years might reveal more. If you measure five outcomes, another theoretically relevant construct could be added. If you conduct an experiment in one country, replication elsewhere could strengthen the evidence.
Research design therefore has no obvious point at which every possible improvement has been exhausted.
The concept of a single "best possible study" is consequently misleading unless the constraints are specified. Study design involves choices among competing objectives such as internal validity, external validity, measurement quality, precision, breadth, duration, cost, participant burden, and practical implementation.
The real question is not whether the design could be stronger. It almost certainly could. The question is whether the proposed design is sufficiently strong for the specific research question and claims.
Feasibility is part of a good research question
Research-question frameworks such as FINER explicitly include feasibility alongside interest, novelty, ethics, and relevance. This reflects a practical truth: a question that cannot be investigated with the available population, time, resources, expertise, and access may be intellectually worthwhile but unsuitable for the present project.
Feasibility should therefore be considered while developing the question, not after an ideal methodology has already been fixed.
A useful starting point is to ask what makes a research question feasible rather than merely interesting. The research question and methodology can then be developed together under realistic constraints.
Do not confuse rigor with complexity
A complex study is not automatically a rigorous study.
Rigor concerns whether the design, measurement, sampling, data collection, analysis, and interpretation are appropriate to the research question and conducted carefully enough to support credible conclusions.
Complexity concerns how many components the study contains.
Methodological rigor
The study uses methods appropriate to the research question, implements them competently, manages important sources of bias and uncertainty, and limits conclusions to what the evidence supports.
Methodological complexity
The study contains multiple methods, variables, sites, groups, measurements, analyses, technologies, or other components.
A carefully designed single-method study can be more rigorous than a poorly integrated mixed-methods project. One well-defined primary outcome may be preferable to fifteen weakly justified outcomes. A focused single-site investigation can be stronger than a multisite study whose coordination exceeds the research team's capacity.
More research components create more opportunities for evidence, but also more opportunities for inconsistency, missingness, error, delay, and analytical confusion.
Start by identifying the minimum evidentiary requirements of the question
Before adding desirable features, ask what the research question fundamentally requires.
If the question concerns change over time, repeated observations may be essential. If it concerns causal effects, the design must address causal inference rather than merely measure two variables together. If it concerns experiences and meaning, the data-generation approach must provide appropriate depth. If it concerns prevalence in a population, sampling and population coverage become central.
These are not optional enhancements. They arise from what the question asks.
Write down the design features without which the research question could no longer be answered credibly. Those features form the methodological core you should protect when feasibility pressures appear.
Then separate essential features from desirable improvements
Once the methodological core is clear, classify the remaining features.
| Design feature |
Question to ask |
Typical response |
| Essential |
Would removing this prevent the study from answering the central question credibly? |
Protect it or revise the question. |
| Strengthening |
Would this materially improve the evidence without defining whether the question can be answered at all? |
Retain when resources permit. |
| Scope-expanding |
Does this mainly allow the study to address additional populations, outcomes, settings, or secondary questions? |
Consider removing or deferring when feasibility is tight. |
| Convenience-enhancing |
Does this primarily reduce researcher workload or improve efficiency? |
Compare the time and financial trade-offs. |
| Decorative complexity |
Would the study lose little substantive value without it? |
Remove it. |
The last category is worth taking seriously. Methodological sophistication can occasionally become performative. A technique does not become necessary merely because it looks impressive in the methods section.
The ideal design depends on the purpose of the study
A pilot study, master's thesis, doctoral dissertation, national evaluation, exploratory qualitative project, randomized trial, and secondary-data analysis are not expected to accomplish the same thing.
A doctoral thesis should make a meaningful contribution, but it does not need to answer every question generated by the topic. A pilot or feasibility study may appropriately focus on whether procedures can work rather than on definitive effectiveness. An exploratory study may establish patterns or concepts that later research investigates more rigorously.
Evaluate design adequacy against what the study is intended to accomplish.
A modest study that answers its stated question well is not methodologically inferior merely because a much larger future project could extend it.
Your degree project is not the entire research program
Students can place unnecessary pressure on a thesis by treating it as though it must resolve the whole topic.
Perhaps your broader research agenda concerns how artificial intelligence changes teaching and learning across universities. One dissertation does not need to examine every technology, stakeholder group, discipline, institution, outcome, and country.
Your thesis can investigate one theoretically meaningful component of that larger problem. Later studies can extend, replicate, challenge, or generalize the findings.
Thinking in terms of a research program makes narrowing easier because excluded questions are not necessarily being abandoned. They are being sequenced.
Scope is one of the safest places to reduce ambition
When a study is infeasible, reducing scope can sometimes preserve methodological quality better than weakening the design itself.
You might study fewer populations, focus on one primary outcome, examine one setting rather than several, reduce secondary research questions, or concentrate on one theoretically central mechanism.
The research question and claims should change accordingly.
A study of one university should not retain language implying all universities. A study of one outcome should not claim to evaluate every consequence of an intervention. A study of one subgroup should not quietly become evidence about the whole population.
Narrower evidence can still be strong evidence when the scope of inference is equally disciplined.
Population breadth and methodological quality can trade off
Suppose you can afford either a carefully implemented study of 250 participants from one institution or a poorly supported study spread across six institutions with inconsistent procedures and insufficient coordination.
The multisite design may appear more generalizable on paper. In practice, inconsistent implementation, missing data, delayed permissions, and weak site monitoring can undermine the supposed advantage.
Broader coverage is useful only when the project can maintain the quality required across that breadth.
This does not mean single-site studies are preferable by default. It means external validity should not be pursued by sacrificing the integrity of the evidence being generalized.
Protect measurement before adding more variables
Researchers often expand studies horizontally by adding variables because collecting "just a few more questions" seems inexpensive.
Every variable creates conceptual, participant, data-management, analytical, and interpretive demands. Poorly justified measures can also distract from the constructs that matter most.
When resources are limited, prioritize strong measurement of the central constructs before accumulating secondary variables.
One well-validated measure of the primary outcome can be more useful than several weak proxies selected because they were easy to collect.
Protect an adequate sample before adding secondary ambitions
If a quantitative study requires a certain sample structure or size for its central analysis, do not preserve numerous secondary outcomes while reducing the primary study below what its design requires.
Likewise, qualitative studies should not add several participant groups when the researcher lacks enough time to engage meaningfully with the resulting data.
The number of participants should follow the logic of the design and analysis rather than serving as the first place to cut whenever the project becomes difficult.
If the required sample itself is infeasible, revise the question or design rather than preserving the same claims with inadequate evidence.
Do not collect data merely because they might be useful later
Researchers sometimes justify additional measures with the phrase, "We might as well collect it while we have the participants."
Sometimes that is efficient. Sometimes it produces unnecessary participant burden, longer instruments, more missing data, additional ethics and data-management obligations, and a sprawling dataset with no clear analytical purpose.
For every measure, ask what research question or necessary adjustment it serves.
If you cannot explain why the information is needed, collecting it may increase complexity without strengthening the study.
Mixed methods should solve a research problem, not decorate the methodology
Combining quantitative and qualitative methods can provide insights that neither component could produce alone. It also creates two data-collection workflows, multiple forms of analysis, integration requirements, additional expertise, and usually more time.
A mixed-methods design is justified when integration of the components is necessary or genuinely useful for answering the research question.
Adding interviews because "mixed methods looks stronger" is not sufficient justification.
If one method can answer the central question credibly within your resources, a focused single-method study may be the stronger project.
Longer follow-up is valuable only when the question requires it
Longitudinal research can reveal change, persistence, delayed effects, or trajectories that cross-sectional designs cannot observe. Longer follow-up can therefore strengthen some questions substantially.
It also increases attrition, participant management, cost, and the time before the dataset becomes complete.
Ask what minimum follow-up period is scientifically meaningful for the outcome in question. Do not choose six months simply because twelve months is infeasible if the phenomenon cannot reasonably change within six months.
When the necessary follow-up exceeds the project timeline, a different question may be more appropriate than an artificially compressed longitudinal study.
The most advanced analysis is not necessarily the best analysis
Use the analysis appropriate to the question, design, and data structure.
A more complicated model may offer genuine advantages when the data require it. It may also introduce additional assumptions, interpretation challenges, skill requirements, and opportunities for error without materially improving the answer.
Choose the simplest analysis that appropriately addresses the research problem, not the simplest analysis you happen to know and not the most complicated analysis the software can produce.
If the appropriate analysis exceeds your current skills, that does not automatically mean you should abandon the research question. Instead, determine whether the required competence can realistically be learned, supervised, or supplied through appropriate collaboration.
Ambition should be evaluated against execution quality
Every additional study component consumes attention.
More sites require coordination. More outcomes require measurement and analysis. More methods require expertise. More follow-ups require retention. More equipment creates resource dependencies. More collaborators require communication. More data require management.
There is therefore a point at which increasing ambition reduces the research team's ability to implement each component well.
A useful question is: Can we execute every part of this design to the standard necessary for the resulting evidence to be credible?
If the answer is no, the design is too ambitious regardless of how impressive it appears in the proposal.
Your personal capacity is part of the research system
Student researchers sometimes plan as though they are an infinitely scalable resource.
You may be the person recruiting participants, conducting interviews, managing the dataset, learning new software, running analyses, writing the thesis, coordinating permissions, and responding to supervisors. Each additional component competes for the same limited attention.
A design that would be manageable for a team of five may not be manageable for one doctoral student simply because the methodological diagram looks identical.
Research feasibility should therefore include the actual people available to perform the work.
Resources interact rather than operating independently
A project may appear feasible when each constraint is considered separately.
You can recruit the sample. You can afford the software. You can learn the analysis. The laboratory is available. The timeline seems possible.
But perhaps recruitment requires evenings, the laboratory is available only during those same evenings, learning the analysis requires the period allocated for data collection, and doing your own transcription to save money consumes the time reserved for writing.
Feasibility is therefore not simply a checklist of individual yes-or-no conditions. The resources must coexist within one workable project.
Test the whole study, not isolated components
Once the design is developed, map the complete project from preparation to final submission.
Ask whether the participants, access, data, skills, specialist support, costs, facilities, equipment, and timeline work together. Identify which dependencies occur simultaneously and which tasks compete for the same resource.
A useful final check is to ask whether the study is actually feasible before committing to it. A project can look feasible when each component is considered separately yet become unworkable when all of those components must coexist within one schedule.
Use constraints to sharpen the question
Constraints are often treated only as obstacles. They can also force conceptual discipline.
If you cannot examine ten outcomes, which outcome most directly represents the phenomenon you care about? If you cannot study five populations, which population provides the most theoretically informative setting? If you cannot conduct three methods, which form of evidence most directly answers the central question?
These decisions can produce a clearer study.
The purpose is not to romanticize limited resources. Resource inequality genuinely shapes who can conduct particular forms of research and what questions receive attention. But within a given project, explicit constraints can sometimes prevent methodological accumulation from substituting for conceptual focus.
There is a difference between pragmatic compromise and methodological compromise
Not every compromise threatens the study equally.
Pragmatic compromise
The study becomes narrower, less convenient, less broad, or less ambitious while remaining capable of answering the revised research question credibly.
Methodological compromise
A change removes something necessary for the design to produce evidence appropriate to the research question, while the original claim is retained.
Reducing a national study to one institution and explicitly narrowing the question may be a pragmatic compromise. Keeping the national claim after collecting data from one convenient institution is a methodological compromise.
Removing exploratory outcomes may be pragmatic. Removing the primary outcome because it is expensive while continuing to claim that you measured it is not.
Make the question follow the feasible design when necessary
Researchers sometimes become attached to the wording of a research question even after feasibility constraints have substantially changed the study.
If the design changes, revisit the question.
Perhaps you can no longer examine causal effects but can examine associations. Perhaps you cannot study long-term outcomes but can study short-term responses. Perhaps national coverage becomes one institutional context. Perhaps the full intervention study becomes a feasibility investigation.
These can all remain worthwhile projects when the question is rewritten to match what the evidence can support.
The problem is not narrowing. The problem is pretending that narrowing did not happen.
Do not confuse feasibility with convenience
There is a danger at the opposite extreme.
If "realistic" becomes synonymous with "whatever is easiest," feasibility can be used to justify weak research. Convenience sampling may replace appropriate recruitment without sufficient justification. Important measurements may be omitted because they are difficult. Researchers may choose questions based entirely on datasets already sitting on their computers.
A feasible study still needs to be worth doing.
The objective is not to minimize difficulty. It is to remove demands that are unnecessary for answering a meaningful question while retaining those that are necessary.
Watch Out
"Do the study you can realistically complete" does not mean "choose the easiest study available." Feasibility should constrain the scope of a worthwhile question, not become an excuse for inadequate sampling, weak measurement, inappropriate analysis, or claims that exceed the evidence.
Leave some margin between feasible and barely possible
A study that can be completed only if every approval arrives immediately, every participant attends, no equipment fails, every analysis works on the first attempt, and the first draft requires no substantial revision is not comfortably feasible.
It is theoretically possible under unusually favorable conditions.
A strong research plan should have some resilience. This does not require excessive padding or planning for every catastrophe. It means leaving enough room that ordinary research delays do not automatically destroy the schedule.
You should therefore estimate whether the complete study can actually be finished within the deadline and test the schedule against plausible delays rather than relying only on the best-case timeline.
Sometimes a smaller study produces a larger contribution
Research contribution does not scale mechanically with sample size, number of methods, geographical coverage, or budget.
A focused study can make a strong contribution by asking a precise question, using appropriate evidence, documenting its methods carefully, and interpreting findings within defensible boundaries.
Conversely, an oversized project may generate large amounts of data while struggling to articulate what the study actually contributes.
Scope and significance are not synonyms.
Know when simplification has gone too far
There is, however, a lower boundary.
You can continue narrowing the population, reducing measurements, shortening follow-up, simplifying analysis, and removing procedures until the project becomes very easy to conduct. At some point, the study may no longer answer an important question or may no longer generate evidence adequate to answer it.
The challenge is finding the boundary between useful simplification and loss of scientific value.
You therefore need to ask when simplifying a study makes it more feasible and when simplification makes the study no longer worth doing.
