Manuel B. Garcia

Manuel B. Garcia serves as the Senior Director for Educational Technology and Digital Learning at FEU Institute of Technology, Manila, Philippines. Read More

Contact Info

1607, FEU Tech Building,
P. Paredes St, Sampaloc,
Manila, Philippines
mbgarcia@feutech.edu.ph

Follow Me

Part 04 · Design

Designing a Study You Can Defend

Design a credible research study by choosing appropriate methods, sampling strategies, measures, and procedures that fit your research question and produce evidence capable of supporting defensible conclusions.

Why This Stage Matters

Your question should determine how you study it.

An interesting research question becomes meaningful evidence only when the study is designed to answer it appropriately. Choices about approach, participants, measurements, procedures, and comparisons determine what researchers will be able to observe and what conclusions the resulting evidence can reasonably support.

Research design therefore involves a series of connected methodological decisions. Researchers must consider quantitative, qualitative, or mixed approaches, select suitable study designs and sampling strategies, determine appropriate sample sizes, and address validity, reliability, bias, feasibility, and other threats to credible findings.

More sophisticated methods do not automatically produce stronger research. A defensible design is one in which each methodological choice has a clear reason, important limitations have been anticipated, and the evidence being collected is genuinely capable of addressing the question the study set out to investigate.

Start Here

Start with the first question in this stage.

If you're working through this part from the beginning, this is the first guide in the sequence.

01
First Guide

How to Choose the Right Research Design for Your Research Question

The right research design is not simply the most rigorous design you know. It is the design that can generate credible evidence for the question you are actually asking while remaining ethical and realistically executable.

Start the Guide →
Part 04 · 138 Sections · 217 Guides

From research question to study design

Work through the methodological decisions that determine how your research will produce credible evidence.

01
how to choose a research design How to Choose the Right Research Design for Your Research Question
→
02
what is research design What Is Research Design, and What Is It Actually Supposed to Accomplish?
→
03
research design vs research method Research Design vs. Research Method: What’s the Difference?
→
04
methodology vs methods Methodology vs. Methods: Why Does the Difference Matter?
→
05
quantitative vs qualitative vs mixed methods Quantitative vs. Qualitative vs. Mixed Methods: Which Approach Should You Use?
→
06
exploratory descriptive explanatory evaluative research What Makes a Study Exploratory, Descriptive, Explanatory, or Evaluative?
→
07
exploratory descriptive explanatory research designs Are Exploratory, Descriptive, Explanatory, and Evaluative Studies Actually Research Designs?
→
08
descriptive and explanatory research Can a Study Be Both Descriptive and Explanatory?
→
09
multiple research designs in one study Can One Study Have More Than One Research Design?
→
10
how to name a research design How Specific Should You Be When Naming Your Research Design?
→
11
do you have to name your research design Do You Have to Name Your Research Design at All?
→
12
appropriate research design What Makes a Research Design Appropriate for a Research Question?
→
13
different research designs same research question Can Two Different Research Designs Answer the Same Research Question?
→
14
best research design for a research question Is There Always One “Best” Research Design for a Research Question?
→
15
ideal vs feasible research design Should You Choose the Ideal Design or the Best Design You Can Realistically Conduct?
→

16
experimental vs quasi-experimental vs observational studies Experimental, Quasi-Experimental, and Observational Studies: What’s the Difference?
→
17
cross-sectional vs longitudinal research Cross-Sectional vs. Longitudinal Research: Which Design Fits Your Study?
→
18
prospective vs retrospective research Prospective vs. Retrospective Research: What’s the Difference and Why Does It Matter?
→
19
cohort vs case-control studies Cohort vs. Case-Control Studies: What’s the Difference and When Should You Use Each?
→
20
case study research What Is a Case Study, and Why Is Studying One Case Not Automatically a Case Study?
→
21
phenomenology vs ethnography vs grounded theory When Should You Use Phenomenology, Ethnography, Grounded Theory, or Narrative Research?
→
22
mixed methods research When Is Mixed Methods Actually Mixed Methods, and When Are You Just Collecting Two Types of Data?
→
23
sequential vs concurrent mixed methods Sequential vs. Concurrent Mixed Methods: Which Should Come First and Why?
→
24
single-site vs multisite research Single-Site vs. Multisite Research: When Does Studying More Settings Actually Help?
→
25
timing of data collection in research How Does the Timing of Data Collection Affect What a Study Can Establish?
→
26
cross-sectional or longitudinal research design Can the Same Research Question Be Studied Using Cross-Sectional or Longitudinal Designs?
→
27
time points in research design When Does Adding Another Time Point Fundamentally Change the Study Design?
→
28
how to choose a research design How Do You Choose Between a Simpler Design and a More Informative but More Complex One?
→

29
correlation vs causation Correlation, Association, Prediction, and Causation: What Is Your Study Actually Designed to Establish?
→
30
comparative research question Does Your Research Question Actually Require a Comparison?
→
31
do you need a control group Do You Really Need a Control Group? When Comparisons Are and Aren’t Necessary
→
32
control group vs comparison group Control Group vs. Comparison Group: What’s the Difference?
→
33
appropriate comparison group What Makes a Comparison Group Appropriate?
→
34
baseline differences between groups What Happens When the Groups You Compare Are Different Before the Study Begins?
→
35
randomization vs random sampling Randomization vs. Random Sampling: Why Do Researchers Keep Confusing Them?
→
36
random assignment in research What Does Random Assignment Actually Do, and What Doesn’t It Do?
→
37
between-subjects vs within-subjects Between-Subjects vs. Within-Subjects Designs: Which Comparison Makes More Sense?
→
38
baseline in research What Is a Baseline, and Does Every Study Need One?
→
39
blinding in research What Is Blinding, and When Does a Study Actually Need It?
→
40
single double triple blinding Single-, Double-, and Triple-Blinding: Are These Labels Actually Useful?
→
41
contamination between groups What Is Contamination Between Groups, and How Can Study Design Reduce It?
→
42
counterfactual in causal research What Is a Counterfactual, and Why Does Causal Research Depend on One?
→
43
temporal order in causal research Why Is Temporal Order Important for Making Causal Claims?
→
44
observational study causal inference When Can an Observational Study Support a Causal Claim?
→
45
association vs causation research design How Do You Know When Your Design Supports Association but Not Causation?
→

46
stakeholder engagement in research What Is Stakeholder Engagement in Research?
→
47
stakeholders in research Who Counts as a Stakeholder in a Research Study?
→
48
research participants vs research partners Research Participants vs. Research Partners: What’s the Difference?
→
49
patient and public involvement in research What Is Patient and Public Involvement (PPI) in Research?
→
50
community-based participatory research What Is Community-Based Participatory Research?
→
51
participatory research vs traditional research Participatory Research vs. Traditional Research: What Changes?
→
52
co-production in research What Does Co-Production Mean in Research?
→
53
stakeholder involvement in research stages At What Stages of Research Can Stakeholders Be Involved?
→
54
community involvement in research questions Can Communities Help Decide the Research Question?
→
55
stakeholder involvement in research design Can Stakeholders Help Design Research Methods and Data Collection?
→
56
stakeholder engagement research relevance How Can Stakeholders Improve Research Relevance Without Controlling the Findings?
→
57
research partner influence study decisions How Much Influence Should Research Partners Have Over Study Decisions?
→
58
tokenism in research When Does Consultation Become Tokenism?
→
59
research partner compensation How Should Community or Patient Research Partners Be Compensated?
→
60
research partners authorship Should Research Partners Qualify for Authorship?
→
61
stakeholder disagreement research design What Should Researchers Do When Stakeholders Disagree About How the Study Should Be Designed?
→

62
how to choose a data collection method How Do You Choose a Data Collection Method That Actually Fits Your Research Question?
→
63
surveys vs interviews vs focus groups vs observation Surveys, Interviews, Focus Groups, and Observation: Choosing a Data Collection Method
→
64
evidence for a research question What Kind of Evidence Does Your Research Question Actually Require?
→
65
primary vs secondary data Primary vs. Secondary Data: Does It Change Your Research Design?
→
66
self-report vs objective measures Self-Report vs. Objective Measures: Which Kind of Evidence Should You Use?
→
67
direct vs indirect evidence in research Direct vs. Indirect Evidence: When Are You Measuring the Thing You Actually Care About?
→
68
multiple data collection methods Can You Use More Than One Data Collection Method Without Doing Mixed-Methods Research?
→
69
what makes a study mixed methods Why Doesn’t Using Both Numbers and Words Automatically Make a Study Mixed Methods?
→
70
multiple data sources in research Should You Collect the Same Information From More Than One Source?
→
71
triangulation in research What Is Triangulation, and What Is It Actually Supposed to Accomplish?
→
72
data triangulation vs methodological triangulation Data Triangulation vs. Methodological Triangulation: What’s the Difference?
→
73
multiple methods research evidence Does Using More Sources or Methods Automatically Make the Evidence Stronger?
→
74
data collection trade-offs What Are the Trade-Offs Between Depth, Breadth, Standardization, and Flexibility in Data Collection?
→
75
participant burden in research How Do Participant Burden and Practical Constraints Affect the Choice of Data Collection Method?
→
76
when research data cannot be collected What Should You Do When the Best Evidence for Your Question Cannot Realistically Be Collected?
→

77
research population and sample How to Define Your Population and Choose the Right Sample
→
78
target population vs accessible population Target Population vs. Accessible Population: Who Are You Actually Studying?
→
79
sample vs population in research Sample vs. Population: What Exactly Are You Trying to Learn About?
→
80
inclusion and exclusion criteria What Are Inclusion and Exclusion Criteria, and How Restrictive Should They Be?
→
81
eligibility criteria and generalizability How Do Inclusion and Exclusion Criteria Change the Population Your Findings Apply To?
→
82
sampling frame What Is a Sampling Frame, and What Happens When It Doesn’t Match Your Population?
→
83
probability vs non-probability sampling Probability vs. Non-Probability Sampling: Which Sampling Method Should You Use?
→
84
probability sampling methods Simple Random, Stratified, Cluster, and Systematic Sampling: Which Probability Method Fits Your Study?
→
85
non-probability sampling methods Convenience, Purposive, Quota, and Snowball Sampling: When Is Non-Probability Sampling Defensible?
→
86
convenience sampling Is Convenience Sampling Always Bad? When Can a Practical Sample Still Be Useful?
→
87
sampling in qualitative research How Do You Sample Participants in Qualitative Research?
→
88
sample size and statistical power How Many Participants Do I Need? Understanding Sample Size and Statistical Power
→
89
qualitative sample size How Many Participants Does Qualitative Research Need? Saturation, Information Power, and the Myth of a Magic Number
→
90
does a bigger sample make a study better Does a Bigger Sample Automatically Make a Study Better?
→
91
representative sample What Is a Representative Sample, and Does Your Study Actually Need One?
→
92
sampling bias What Is Sampling Bias, and How Can the Sampling Process Distort Your Findings?
→
93
nonresponse bias What Is Nonresponse Bias, and When Does It Threaten Your Sample?
→
94
large sample bias Can a Sample Be Large but Still Seriously Biased?
→
95
generalizability in research Can Your Findings Be Generalized? External Validity, Transferability, and the Limits of Your Sample
→

96
inclusion and exclusion in research Who Gets Included in Research, and Who Gets Left Out?
→
97
diversity in research samples Why Does Diversity in Research Samples Matter?
→
98
representation vs representativeness Representation vs. Representativeness: What’s the Difference?
→
99
sex vs gender in research Sex vs. Gender in Research: Why Does the Difference Matter?
→
100
analyze results by sex or gender When Should Research Analyze Results by Sex or Gender?
→
101
race and ethnicity in research How Should Researchers Think About Race and Ethnicity as Research Variables?
→
102
demographic categories in research When Can Demographic Categories Oversimplify the People Being Studied?
→
103
language requirements in research How Can Language Requirements Exclude Important Populations From Research?
→
104
inclusion and exclusion criteria When Do Inclusion and Exclusion Criteria Create an Unrepresentative Sample?
→
105
underrepresentation in research What Is Underrepresentation in Research, and Why Does It Matter?
→
106
accessible research participation How Can Researchers Make Participation More Accessible?
→
107
study procedures participant exclusion How Can Study Procedures Unintentionally Exclude Certain Participants?
→
108
diverse sample representative Does Making a Sample More Diverse Automatically Make It Representative?
→
109
oversampling underrepresented groups When Is It Appropriate to Deliberately Oversample an Underrepresented Group?
→
110
inclusive study population How Should Researchers Balance Inclusion With the Need for a Clearly Defined Study Population?
→

111
how to operationalize a construct How Do You Turn a Construct Into Something You Can Actually Measure?
→
112
conceptual and operational definitions How Do Your Conceptual and Operational Definitions Guide Measurement?
→
113
measure vs instrument vs indicator Measure vs. Instrument vs. Indicator: What’s the Difference?
→
114
direct vs indirect measures Direct vs. Indirect Measures: When Can’t You Measure the Thing You Actually Care About?
→
115
objective vs subjective measures Objective vs. Subjective Measures: Is One Really Better Than the Other?
→
116
self-report vs observed measures Self-Report vs. Observed or Recorded Measures: What Changes?
→
117
single-item vs multi-item measures Single-Item vs. Multi-Item Measures: How Much Measurement Is Enough?
→
118
levels of measurement What Are Levels of Measurement, and Do Nominal, Ordinal, Interval, and Ratio Still Matter?
→
119
measurement error in research What Is Measurement Error, and How Can It Affect Your Study Before Analysis Begins?
→
120
systematic vs random measurement error Systematic vs. Random Measurement Error: What’s the Difference?
→
121
multiple ways to measure a construct Can the Same Construct Be Measured in Several Valid Ways?
→
122
incomplete construct measurement What Happens When Your Measure Captures Only Part of the Construct?
→
123
existing vs new measure Should You Prefer an Existing Measure or Create a New One?
→
124
measure appropriate for population and context How Do You Know Whether a Measure Is Appropriate for Your Population and Context?
→
125
measurement change research question Can the Way You Measure Something Change the Research Question You Are Actually Answering?
→

126
valid research design What Makes a Research Design Valid and Defensible?
→
127
reliability vs validity Reliability vs. Validity: How Do You Know Your Research Measures What It Should?
→
128
internal vs external validity Internal vs. External Validity: What Kind of Validity Does Your Study Actually Need?
→
129
types of validity Construct, Content, Criterion, and Face Validity: What’s the Difference?
→
130
reliable but not valid Can a Measure Be Reliable but Not Valid?
→
131
validated instrument study validity Does Using a Validated Instrument Make Your Study Valid?
→
132
validity evidence across populations Validated for Whom? Why Validity Evidence May Not Automatically Transfer Across Populations
→
133
threats to validity in research Bias, Confounding, and Other Threats to Valid Research
→
134
selection information measurement bias Selection Bias, Information Bias, and Measurement Bias: Where Can Bias Enter Your Study?
→
135
confounding in research What Is Confounding, and Why Can It Distort a Relationship?
→
136
statistical adjustment confounding Why Doesn’t Statistical Adjustment Automatically Fix Confounding?
→
137
effect modifiers moderators confounders What Are Effect Modifiers and Moderators, and Why Aren’t They the Same as Confounders?
→
138
internally valid but not generalizable Can a Study Be Internally Valid but Not Generalizable?
→
139
internal validity vs generalizability Can Improving Internal Validity Reduce Generalizability?
→
140
rigor in qualitative research What Does Rigor Mean in Qualitative Research? Credibility, Dependability, Confirmability, and Transferability
→
141
reliability and validity in qualitative research Does Qualitative Research Need Reliability and Validity?
→
142
trustworthiness in qualitative research How Do Researchers Establish Trustworthiness in Qualitative Research?
→
143
rigor in mixed-methods research What Does Rigor Mean in Mixed-Methods Research?
→
144
research limitation design flaw trade-off How Do You Know Whether a Limitation Is a Design Flaw or Simply a Trade-Off?
→
145
research limitations useful evidence Can a Study Have Important Limitations and Still Produce Useful Evidence?
→

146
plan data analysis before data collection Should You Decide How You’ll Analyze the Data Before You Collect It?
→
147
analysis plan before data collection What Should an Analysis Plan Decide Before Data Collection Begins?
→
148
analysis research question design alignment How Do You Know Whether Your Planned Analysis Actually Matches Your Research Question and Design?
→
149
research question planned analysis Should Every Research Question Have a Planned Analysis?
→
150
hypothesis corresponding analysis Should Every Hypothesis Have a Corresponding Analysis?
→
151
variables in analysis plan How Should Your Variables and Their Roles Shape the Analysis Plan?
→
152
statistical test vs study design Should Your Statistical Test Influence Your Study Design, or Should the Design Come First?
→
153
choose statistical test before data collection Should You Choose the Statistical Test Before Collecting the Data?
→
154
don't know how to analyze data What Happens If You Collect Data You Don’t Know How to Analyze?
→
155
multiple analyses same research question What Should You Do When More Than One Analysis Could Answer the Same Research Question?
→
156
qualitative analysis plan before data collection Do Qualitative Studies Need an Analysis Plan Before Data Collection?
→
157
qualitative analysis plan What Should a Qualitative Analysis Plan Specify in Advance?
→
158
mixed methods integration plan Does Mixed-Methods Research Need an Integration Plan Before Data Collection?
→
159
analysis plan flexibility What Parts of an Analysis Plan Can Legitimately Remain Flexible?
→
160
when to involve a statistician in research When Should a Statistician or Methodologist Be Involved in Study Design?
→

161
test research design before main study Should You Test Your Research Design Before Starting the Main Study?
→
162
pilot study vs feasibility study Pilot Study vs. Feasibility Study: What’s the Difference?
→
163
do I need a pilot study Do You Need a Pilot or Feasibility Study Before Your Main Research?
→
164
what should a feasibility study tell you What Exactly Should a Feasibility Study Tell You?
→
165
what should a pilot study test What Exactly Should a Pilot Study Test?
→
166
pilot study conclusions What Shouldn’t You Conclude From a Pilot Study?
→
167
pretest vs pilot study vs feasibility study Pretest vs. Pilot Study vs. Feasibility Study: What’s the Difference?
→
168
parts of a study to test before full study Which Parts of a Study Can Be Tested Before the Full Study Begins?
→
169
recruitment feasibility How Do You Decide Whether Recruitment Is Feasible Before Committing to the Main Study?
→
170
test participant understanding study procedures How Can You Test Whether Participants Understand the Study Procedures?
→
171
data collection procedures practicality How Do You Know Whether Data Collection Procedures Are Practical?
→
172
pilot study sample size Can a Pilot Study Be Too Small to Be Useful?
→
173
test planned analysis on pilot data Should You Test Your Planned Analysis on Pilot Data?
→
174
include pilot data in main study Can You Include Pilot Study Participants or Data in the Main Study?
→
175
pilot study design not feasible What If Your Pilot Study Shows That Your Original Design Won’t Work?
→
176
study changes after pilot How Much Should a Study Change After a Pilot Before It Becomes a Different Study?
→

188
preregistration and registered reports Preregistration and Registered Reports: Should You Commit to Your Study Plan Before Seeing the Results?
→
189
what is preregistration What Is Preregistration, and What Problem Is It Supposed to Solve?
→
190
what to preregister What Goes Into a Research Preregistration? A Guide to Hypotheses, Sampling, Variables, Exclusions, and Analyses
→
191
can a preregistered study change Does Preregistration Mean That Nothing in the Study Can Change?
→
192
what can change after preregistration What Can Still Change After a Study Has Been Preregistered?
→
193
protocol vs preregistration Protocol vs. Preregistration: What’s the Difference?
→
194
preregistration vs registered report Preregistration vs. Registered Report: What’s the Difference?
→
195
does every study benefit from preregistration Does Every Kind of Research Benefit From Preregistration?
→
196
qualitative research preregistration Can Qualitative Research Be Preregistered?
→
197
exploratory research preregistration Can Exploratory Research Be Preregistered?
→
198
exploratory vs confirmatory research Exploratory vs. Confirmatory Research: Do You Have to Decide Before Collecting the Data?
→
199
preregistration deviations Can You Deviate From a Preregistered Study Without Doing Something Wrong?
→
200
document preregistration deviations How Should Deviations From a Preregistered Plan Be Documented?
→
201
does preregistration make research more rigorous Does Preregistration Automatically Make a Study More Rigorous?
→

202
change research design after data collection Can You Change Your Research Design After Data Collection Has Started?
→
203
participant recruitment problems What Should You Do When Participant Recruitment Doesn’t Go According to Plan?
→
204
cannot reach planned sample size What Should You Do If You Cannot Reach Your Planned Sample Size?
→
205
control group problems in research What If Your Control or Comparison Group Does Not Work as Planned?
→
206
recruited population differs from planned population What If the Population You Can Recruit Differs From Your Planned Study Population?
→
207
research instrument not working as expected What Should You Do When a Measure or Instrument Does Not Work as Expected?
→
208
research method does not fit research question What Should You Do When Your Planned Method Turns Out Not to Fit Your Research Question?
→
209
research design assumption wrong What If an Important Assumption Behind Your Research Design Turns Out to Be Wrong?
→
210
changing research method different study When Does Changing Your Research Method Turn It Into a Different Study?
→
211
changing research sample different study When Does Changing Your Research Sample Turn It Into a Different Study?
→
212
report methodological deviations When Does a Methodological Deviation Need to Be Reported?
→
213
research changes requiring ethics approval Which Changes Require Revisiting Ethics Approval, the Protocol, or Preregistration?
→
214
document unplanned research changes How Should Unplanned Changes to a Research Study Be Documented?
→
215
adapting research design When Is Adapting the Design Methodologically Defensible?
→
216
methodological compromise in research How Much Methodological Compromise Can a Research Study Tolerate?
→
217
when to stop or redesign a research study When Should You Stop or Redesign a Study Because the Research Design No Longer Works?
→
When You're Ready to Move On

A defensible design also has ethical responsibilities.

Before data collection begins, researchers need to consider participants, consent, privacy, integrity, and the ethical obligations attached to the study.

Next · Part 05

Doing Research Without Compromising Ethics

Understand research ethics and integrity, protect participants and research data, address informed consent and conflicts of interest, and meet the ethical responsibilities that apply from study planning through publication.

Continue to Part 05 →