How Can You Test Whether Participants Understand the Study Procedures?

How Do You Know Participants Understand What Your Study Asks Them to Do?

Research instructions often appear obvious to the people who wrote them. Participants encounter them without the months of discussion, protocol development, and disciplinary knowledge that made those instructions seem obvious to the research team.

A participant may misunderstand when to complete a task, what a question means, which activities are required, how often something should be recorded, or what counts as completing a procedure correctly. They may even complete the study without mentioning the misunderstanding.

That creates an important methodological problem. If participants interpret the procedure differently from what researchers intended, the resulting variation may be mistaken for variation in the phenomenon being studied.

Testing participant understanding therefore requires more than asking, “Is everything clear?” You need evidence about what participants think the instructions mean and what they actually do when following them.

Ask Participants to Show You What They Understood

Test Comprehension as Part of the Research Procedure

“Do You Understand?” Is a Weak Comprehension Test

Researchers sometimes finish an explanation by asking participants whether everything is clear. A participant says yes, and the procedure begins.

That exchange tells you very little about comprehension. Participants may believe they understand while holding an interpretation different from yours. They may also hesitate to admit confusion, particularly when instructions sound simple or when the researcher appears to expect an affirmative answer.

Instead of asking participants to judge their own comprehension globally, ask questions or observe behaviors that reveal what they actually understood.

First Identify What Participants Must Understand Correctly

Not every sentence in a study document deserves the same amount of testing. Begin by identifying instructions whose misunderstanding could alter participation, data quality, adherence, safety, or interpretation.

Depending on the study, participants may need to understand eligibility questions, consent information, questionnaire wording, task instructions, intervention activities, device use, diary entries, specimen collection, follow-up schedules, or rules about when and how an activity should be completed.

For each critical procedure, write down the intended interpretation. If you cannot specify what correct understanding would look like, it will be difficult to test whether participants have achieved it.

Ask Participants to Explain Instructions in Their Own Words

One useful approach is to ask participants to explain what they believe an instruction means. This is sometimes described as teach-back when participants are asked to restate important information in their own words.

Suppose an instruction says, “Record your activity at the end of each study day.” Rather than asking whether that instruction is clear, you might ask when the participant would complete the record and what they would do if they forgot until the following morning.

The response can reveal interpretations that a yes-or-no comprehension question would miss.

This approach should be used as a test of the communication, not as an examination the participant is expected to pass. If several participants misunderstand the same instruction, the more productive conclusion may be that the instruction needs revision.

Use Cognitive Probing When Wording or Interpretation Matters

Cognitive interviewing and related probing approaches can help researchers investigate how participants understand questions and instructions.

Researchers might ask what a particular phrase means to the participant, how they arrived at an answer, what time period they considered, whether any words were difficult, or whether response options matched the answer they wanted to give.

This can be particularly useful for questionnaires, screening questions, consent materials, instructions, and participant-facing documents. It shifts the focus from whether participants can produce an answer to how they interpreted the material while producing it.

If only a particular instrument or instruction is uncertain, this kind of focused work may be more appropriate than a complete pilot. The distinction between a pretest and broader pilot or feasibility study depends on what needs to be learned.

Observe What Participants Actually Do

Understanding is behavioral as well as verbal. Participants may correctly paraphrase an instruction but encounter difficulty when carrying it out.

Observation can reveal where participants hesitate, skip steps, perform tasks in the wrong sequence, seek assistance, use equipment incorrectly, or improvise because the instructions do not cover the situation they encounter.

Researchers can document the number and types of clarification requests, procedural errors, deviations, incomplete tasks, and points at which researcher assistance becomes necessary.

These observations can be more informative than a general satisfaction rating because they identify exactly where the procedure breaks down.

Let Participants Try the Procedure Under Realistic Conditions

A procedure may appear understandable during an explanation but become difficult when participants attempt it independently.

If the main study expects participants to complete an online assessment at home, for example, testing it only while a researcher sits beside them may conceal problems. If participants will use a device independently, follow instructions across several days, or complete activities in a busy school, clinic, workplace, or household setting, preliminary testing should approximate those conditions where practical.

Pilot and feasibility guidance generally recommends that measurement and data-collection procedures resemble those intended for the later full-scale study when those procedures are being tested. This makes the findings more informative about what participants will actually experience.

Test the Sequence, Not Just Individual Instructions

Each instruction can be understandable on its own while the overall sequence remains confusing.

A participant may know how to complete a questionnaire, collect a sample, and upload a record, yet be uncertain which happens first, whether they occur on the same day, or what to do when one step cannot be completed.

Testing the complete participant journey can expose these interactions. This is particularly important for multistep procedures, repeated assessments, interventions, longitudinal studies, and research involving several platforms or locations.

When the interaction among study components is the uncertainty, it may be worth testing the connected study workflow rather than isolated components.

Pay Attention to Requests for Help

Researcher assistance can conceal a poorly designed procedure.

During preliminary testing, staff may naturally help when a participant becomes confused. That assistance can make the session appear successful even though the main study expects participants to complete the same procedure independently.

Record when assistance is requested, what type of help is needed, and whether the same problem appears across participants. If repeated explanation is necessary, revise the procedure rather than simply training researchers to rescue participants every time.

Look for Patterns Across Participants

One participant misunderstanding an instruction may reflect an individual difficulty. Several participants making the same interpretation is stronger evidence that the communication itself may be problematic.

Keep structured notes about questions, errors, hesitations, alternative interpretations, requests for clarification, and suggested wording. These patterns can guide revision more systematically than relying on the researcher's general impression that the session “went well.”

Qualitative and quantitative evidence can complement each other. Error frequencies may show where problems occur, while participant explanations can reveal why.

Check Whether Different Participants Interpret Procedures Differently

Understanding may vary across literacy levels, languages, age groups, educational backgrounds, accessibility needs, cultural contexts, or prior familiarity with the task or technology.

Preliminary testing should therefore include people sufficiently similar to those who will encounter the procedure in the main study. When meaningful diversity exists within the intended population, testing only the participants most comfortable with the procedure may give an overly optimistic picture.

This does not mean a small pretest can establish comprehension for every possible subgroup. It means participant selection for preliminary testing should reflect the types of comprehension difficulty the researchers need to detect.

Separate Comprehension From Acceptability

A participant can understand a procedure perfectly and still dislike it. Conversely, a participant can find a procedure pleasant while misunderstanding what is required.

Both may matter for feasibility, but they answer different questions. If participants fail to complete a procedure, determine whether they did not understand it, understood but found it burdensome, encountered a technical barrier, or simply chose not to comply.

Revise and Test Again When the Change Is Consequential

Pretesting is often iterative. Researchers identify a problem, revise the instruction, and assume the new version fixes it. Sometimes it does. Sometimes the revision creates a different ambiguity.

When a change affects an important procedure, test the revised version again if practical. This is especially valuable when the original misunderstanding was common or could materially alter data collection.

How a Simple Probe Can Reveal an Invisible Misunderstanding

The original instruction looked simple and generated no complaints. The misunderstanding became visible only when researchers examined what participants thought the instruction actually meant.

Common Mistakes When Testing Participant Understanding

Make Participant Understanding Observable Before the Main Study

Identify the points at which misunderstanding could change participation or the data. Then choose a test capable of revealing the relevant interpretation or behavior.

Also distinguish understanding from the broader question of whether participants can complete the data-collection procedures practically. A procedure can be understandable but too burdensome, time-consuming, inconvenient, or technically unreliable for the main study.

Before Assuming Participants Understand the Procedure, Check

Frequently Asked Questions About Testing Participant Understanding

Do Not Ask Whether Instructions Are Clear; Find Out What Participants Think They Mean

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.