How Do You Know Whether Data Collection Procedures Are Practical?

A Procedure Can Collect the Right Data and Still Be Impractical

Your protocol specifies exactly what data you need. The measures are appropriate. The equipment is available. The questionnaire works. On paper, the data collection plan appears complete.

Then actual participants arrive.

An assessment expected to take 30 minutes takes an hour. Participants become tired before reaching the final measures. Equipment must be moved between rooms. Internet connectivity interrupts an online task. Research staff spend substantial time resolving identifiers. Follow-up appointments become difficult to schedule. Missing data accumulate in one particular part of the procedure.

Data collection is practical only when the planned process can operate consistently under the conditions of the study. That requires testing more than whether the individual instrument technically works.

Test the Complete Data Collection Workflow Under Realistic Conditions

Practicality Is About What Happens When the Protocol Meets Reality

Start With the Data the Research Question Actually Requires

Practicality should not become an excuse to remove difficult measurements simply because they require effort. First establish what information the research design genuinely needs. Then ask whether the proposed way of collecting that information can be implemented adequately.

Some measurements may be scientifically indispensable even when they are demanding. In that case, preliminary testing can help researchers redesign scheduling, staffing, participant support, or equipment arrangements rather than abandoning the measure.

Conversely, a burdensome procedure collecting information that is merely “nice to have” may not justify the costs it imposes on participants and the study.

Measure How Long the Procedure Actually Takes

Estimated completion time is often based on the instrument in isolation. Real data collection includes instructions, setup, consent or confirmation procedures, transitions between measures, equipment preparation, questions from participants, technical problems, breaks, and data checks.

Measure the full time required under realistic conditions. If several activities occur during the same visit, examine the duration of the complete visit rather than adding idealized estimates for each component.

Pilot and feasibility research has shown why this matters: data collection that appears reasonable as a stand-alone activity may become burdensome when embedded within the participant's complete study visit.

Assess Participant Burden, Not Just Completion

A participant who manages to finish a procedure may still have found it excessively demanding.

Burden can arise from duration, repetition, cognitive effort, travel, invasive procedures, scheduling, technology, privacy concerns, repeated reminders, or the cumulative demands of several study activities.

Evidence can come from completion behavior, withdrawal, skipped items, requests for breaks, participant feedback, interviews, observations, or other measures appropriate to the study.

This matters because burdensome data collection can contribute to nonresponse and attrition, thereby affecting both feasibility and data quality.

Look at Missing Data as a Procedural Signal

Missing data are not always merely a statistical problem to be handled during analysis. They can indicate that something about data collection is not working.

If participants consistently skip one questionnaire section, fail to complete late-session measures, miss a particular follow-up, or leave fields blank on one device, examine the procedure that generates the missingness.

The problem might involve confusing wording, participant fatigue, sensitive content, software design, scheduling, staff instructions, equipment failure, or an error in how data are saved.

During preliminary testing, inspect both the amount and pattern of missing data. A single overall percentage can conceal a concentrated problem in one essential variable or time point.

Test Whether Participants Can Follow the Procedure Independently Where Required

Some data are collected while researchers supervise participants closely. Others depend on participants completing diaries, surveys, device measurements, logs, specimen collection, or follow-up tasks independently.

If independent completion is part of the main study, test it. A procedure that succeeds only because a researcher repeatedly explains what to do is not operating under the conditions for which it was designed.

If errors appear, determine whether the difficulty is practical or whether participants do not understand the instructions. The latter may require specific testing of how participants understand the study procedures.

Assess Researcher Workload and Staffing Requirements

Data collection can also be impractical for the research team.

Record how much staff time is needed for preparation, administration, troubleshooting, data checking, follow-up, travel, specimen handling, equipment setup, and documentation. Determine which tasks require specialized personnel and whether several activities compete for the same staff member.

A procedure that works because the principal investigator personally supervises every participant may not scale to hundreds of participants or multiple sites.

Ask whether the staffing model used during testing can realistically be reproduced in the main study.

Test Equipment, Technology, and Connectivity Where They Will Be Used

Technology that performs reliably in the research office may behave differently in schools, homes, clinics, field sites, or institutions with restrictive networks.

Preliminary testing can reveal problems with devices, software compatibility, browser behavior, battery life, internet connectivity, login procedures, permissions, synchronization, storage, or transfer between systems.

Where equipment is shared, also test scheduling, cleaning, calibration, transport, charging, maintenance, and replacement procedures if these could affect data collection.

Test Whether Data Are Recorded Correctly

A participant can complete every procedure while the resulting data are still unusable.

Researchers should verify that electronic forms save responses correctly, paper forms can be entered without ambiguity, identifiers are assigned consistently, units are recorded correctly, dates and timestamps behave as expected, response options are coded appropriately, and missing values can be distinguished from legitimate values.

Range checks, required fields, branching logic, and automated validation can help, but they should themselves be tested. An incorrectly programmed validation rule can prevent valid data from being entered just as easily as it can prevent errors.

Follow the Data Beyond the Moment of Collection

Practicality does not end when the participant leaves. Data may need to be transferred, uploaded, transcribed, linked, scored, cleaned, de-identified, backed up, or combined with information from other sources.

A pilot should therefore allow enough time for preliminary data to move through the intended management process. Field-trial guidance specifically recommends testing data entry, cleaning, analysis systems, and, where applicable, specimen processing and transport as part of pilot work.

This is particularly important when multiple platforms or identifiers are involved. Problems discovered after hundreds of participants have completed the study may be difficult or impossible to reconstruct.

Test Repeated and Follow-Up Collection, Not Just Baseline

Baseline data collection often receives the most attention because every participant begins there. Yet later assessments may be more difficult.

Participants may be less motivated, schedules may change, contact information may become outdated, repeated questionnaires may feel burdensome, or follow-up procedures may require additional travel.

If the main research depends on repeated observations, preliminary testing should examine the relevant follow-up process whenever the pilot duration permits. Successful baseline collection does not establish that longitudinal data collection is practical.

Consider Whether the Procedure Scales

Suppose one researcher can collect data from five participants per day. Can the study recruit and assess 500 participants within its planned period? Can the equipment support simultaneous sessions? Can the data-management system handle the volume? Can multiple staff members apply the procedure consistently?

Practicality at small scale does not guarantee practicality at large scale.

When assessing feasibility, compare the resources and throughput observed during preliminary testing with what the main study will require. Pilot and feasibility guidance emphasizes considering whether processes demonstrated at small scale can realistically be reproduced when the research expands.

Do Not Confuse Practicality With Measurement Quality

A measure can be easy to administer and scientifically inappropriate. Another can have strong measurement properties but be difficult to collect in your setting.

Both matter. Choosing the easiest measure without considering validity can weaken the study, while choosing an excellent measure that participants cannot realistically complete can produce a different kind of failure.

When a Reasonable Assessment Becomes an Unreasonable Study Visit

Every individual measurement could technically be collected. The pilot revealed that the combined data-collection procedure, rather than any single instrument, was the real feasibility problem.

Common Mistakes When Assessing Data Collection Practicality

Evaluate the Procedure From the Participant’s First Task to the Analysis-Ready Dataset

Do not evaluate data collection as a collection of instruments. Evaluate it as a workflow involving participants, researchers, equipment, locations, time, and data systems.

If the practical problems involve several connected processes rather than data collection alone, consider whether the broader study workflow should be examined in a pilot.

And where data structures or analytical implementation remain uncertain, test whether the planned analysis can operate on the type of data your procedures actually produce before the definitive dataset is collected.

Before Calling Your Data Collection Procedures Practical, Check

Frequently Asked Questions About Data Collection Feasibility

Practical Data Collection Must Work Beyond the Protocol

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.