03 · What You Need to Know
Where Stakeholder Knowledge Can Improve Research Design
Research methods contain assumptions about real people and settings
Methodological decisions are technical, but they are rarely purely technical.
A sampling strategy assumes that eligible people can be reached. A survey assumes respondents understand the questions as intended. An intervention assumes participants can follow its procedures. An outcome measure assumes that what it measures matters. A data-collection schedule assumes participants can realistically comply.
Stakeholders can help test these assumptions before they become expensive problems.
Evidence from PCORI-funded stakeholder-engaged projects has documented stakeholder contributions to study design, data collection, refinement of research questions, selection of interventions, choice and measurement of outcomes, recruitment strategies, and interpretation.
Stakeholders can help choose outcomes that matter
Researchers often choose outcomes because they are established in the literature, measurable with validated instruments, required by convention, or expected by reviewers. Those considerations matter, but they do not guarantee that the outcome captures what affected people consider consequential.
A clinical outcome may improve without changing daily functioning. An educational intervention may raise test scores while substantially increasing workload. A public service may increase overall uptake while remaining inaccessible to particular groups.
Stakeholders can identify outcomes researchers have overlooked or help distinguish technically measurable outcomes from outcomes that matter to decisions and lived experience.
PCORI's methodology standards explicitly require appropriate engagement of people representing the population of interest and other relevant stakeholders in patient-centered comparative effectiveness research contexts. Its engagement research has also documented stakeholder influence on outcome selection and measurement.
Stakeholders can help researchers select or refine measures
Identifying an important outcome does not automatically determine how it should be measured.
Stakeholders may review questionnaires, rating scales, interview questions, observation procedures, or other instruments for relevance and comprehensibility. They may identify terminology that is ambiguous, culturally inappropriate, stigmatizing, or simply unfamiliar to the intended population.
They may also point out that an instrument is too long, repetitive, inaccessible, or burdensome under real-world conditions.
That input does not mean researchers should alter validated instruments casually. Changing wording, response options, scoring, administration, or item content can affect measurement properties. When standardized measures are required, researchers need to determine what modifications are permissible and what validation evidence applies.
Stakeholder relevance and measurement validity both matter.
Stakeholders can improve recruitment strategies
Recruitment is an area where technically plausible plans frequently encounter practical reality.
Researchers may not know which organizations are trusted, which communication channels people actually use, why particular language discourages enrollment, or why study visits are impossible for some potential participants.
Stakeholders can identify these barriers and suggest alternatives. PCORI evaluations have reported research partners contributing to recruitment strategies, while empirical studies of stakeholder engagement have identified changes to study methods and measurement tools attributed to partner involvement.
There is nevertheless an important boundary. Recruitment strategies still need to comply with ethics approval, privacy requirements, inclusion criteria, and protections against coercion or undue influence.
Stakeholders can help assess participant burden
A protocol can appear reasonable to researchers while being exhausting for participants.
Consider the cumulative burden of travel, waiting time, repeated questionnaires, invasive measurements, digital tasks, childcare arrangements, lost wages, or frequent follow-up. Researchers may evaluate each procedure individually and miss how the full study feels to someone completing all of them.
People with relevant lived experience can help identify where burden may undermine recruitment, retention, data quality, or fairness.
This does not imply that every inconvenient procedure should be removed. Some procedures may be essential. But researchers can examine whether the same information could be collected less burdensomely or whether support can reduce unnecessary barriers.
Stakeholders can contribute to intervention design
When research evaluates an intervention, stakeholders can help determine whether that intervention is understandable, acceptable, accessible, and plausible in the setting where it will operate.
Patients may identify treatment burdens. Practitioners may identify workflow conflicts. Students may notice that an educational technology assumes access to devices they do not have. Community organizations may recognize that an intervention clashes with local routines or priorities.
Stakeholder involvement can therefore affect both what intervention is studied and how it is delivered.
This should not become an excuse to redesign an intervention until it produces the result stakeholders want. Intervention development and outcome evaluation are different tasks, and changes should be documented transparently.
Stakeholders can help design qualitative data collection
Interview and focus-group guides are particularly amenable to stakeholder input.
Partners can identify questions that sound judgmental, assumptions embedded in wording, important topics researchers omitted, or sequences that make sensitive questions unnecessarily abrupt. They may also advise on who should conduct interviews, where conversations should occur, and what conditions may help participants speak openly.
Stakeholder involvement can therefore improve the relevance and acceptability of qualitative data collection without determining in advance what participants should say.
Stakeholders can contribute to quantitative research too
Stakeholder involvement is sometimes associated primarily with interviews and participatory qualitative methods. That is unnecessarily restrictive.
Partners can contribute to quantitative studies by helping define outcomes, reviewing survey instruments, assessing intervention feasibility, developing recruitment and retention strategies, discussing acceptable follow-up schedules, interpreting quantitative findings, and identifying subgroup questions that matter to users of the research.
Stakeholder involvement describes who contributes to research decisions, not whether the final dataset contains numbers or words.
Research partners can sometimes collect data themselves
In some participatory projects, community, patient, practitioner, or other research partners may take direct roles in data collection.
A community researcher might conduct interviews because participants feel more comfortable speaking with someone who understands the local context. Patient partners might contribute to qualitative interviewing. Community organizations might participate in structured data-collection activities.
This can offer advantages, but it introduces additional considerations.
| Issue |
Why it matters |
What to plan |
| Training |
Data need to be collected consistently and appropriately |
Method-specific preparation, practice, and ongoing support |
| Confidentiality |
Partners may know participants personally or encounter sensitive information |
Clear confidentiality procedures and data-access rules |
| Role boundaries |
Community members may simultaneously occupy personal, professional, and research roles |
Explicit expectations about which role applies during data collection |
| Ethics and governance |
Data collectors may have responsibilities specified by ethics or institutional requirements |
Verify approvals, delegation, consent procedures, and institutional policies |
| Consistency |
Differences in interviewing or measurement can affect the data |
Protocols, supervision, documentation, and quality checks appropriate to the method |
| Compensation |
Collecting data is substantive research work |
Plan fair payment and reimbursement where appropriate |
Direct partner participation in data collection should therefore be designed as research work rather than added informally because someone has community access.
Stakeholder involvement can improve feasibility without automatically improving validity
A method becoming easier or more acceptable does not necessarily make it more scientifically valid.
Suppose stakeholders strongly prefer removing several sensitive questions from a survey. Doing so might improve acceptability, but those items may measure a construct essential to answering the research question. Alternatively, researchers might insist on a technically sophisticated procedure that causes so much attrition that the resulting data become less useful.
Neither methodological purity nor stakeholder preference resolves the issue automatically.
The task is to understand the trade-off and determine whether the design can satisfy both scientific and practical requirements.
Researchers remain accountable for methodological integrity
Stakeholder involvement does not transfer scientific accountability away from investigators.
Researchers remain responsible for ensuring that sampling, measurement, comparison groups, intervention procedures, analyses, and other methodological choices are appropriate to the question. They also remain responsible for complying with applicable ethics, regulatory, privacy, and institutional requirements.
At the same time, “methodological rigor” should not become a convenient phrase for rejecting stakeholder suggestions without examination. A procedure that cannot be implemented appropriately in the intended population may be methodologically elegant on paper and methodologically unsuccessful in practice.
The better question is whether stakeholder knowledge can improve relevance and feasibility without compromising what the evidence can legitimately support.
Stakeholder involvement should be matched to expertise and purpose
Not every stakeholder needs to comment on every methodological detail.
A patient partner may be well positioned to discuss participant burden without needing to choose a multilevel statistical model. A practitioner may understand implementation constraints but have little reason to determine psychometric thresholds. A statistician may be responsible for the analytical strategy while needing community partners to explain whether the outcome being modeled has practical meaning.
PCORI guidance recommends tailoring engagement to the project's goals and to the level at which partners wish to engage, while preparing both researchers and stakeholders for their roles.
Good partnership therefore does not distribute every decision equally. It brings the right forms of knowledge into the decisions where they are relevant.
