03 · What You Need to Know
Every data collection design privileges some forms of evidence over others
Depth asks how much you can understand about each case
Depth refers broadly to the richness, detail, contextual information, elaboration, or interpretive material collected about a participant, case, event, setting, or phenomenon.
In-depth interviews, prolonged observation, diaries, detailed case materials, and other intensive approaches can allow researchers to examine experiences or processes that would be difficult to capture using a few fixed-response items.
If a participant says that a new institutional policy made teaching “more difficult,” a structured questionnaire may record the rating. A semi-structured interview can ask what became difficult, when the problem emerged, how the participant responded, whether the difficulty varied across courses, and what the participant believes caused it.
That elaboration is valuable when the research question requires explanation, interpretation, process, context, or meaning. It is less useful if the study needs a standardized estimate across a large population and the detail cannot be analyzed in a way that supports that purpose.
Breadth asks how widely the evidence needs to extend
Breadth can refer to several dimensions of coverage. A study may seek evidence from more participants, more institutions, more settings, more demographic groups, more time points, or a wider range of variables.
Structured questionnaires can often be administered to substantially more participants than hour-long interviews using the same resources. Administrative datasets may provide coverage across entire institutions or long periods. Large-scale observation, by contrast, can become resource intensive quickly.
Broad coverage can be important when the research question concerns prevalence, distribution, variation across groups, or patterns that require a sufficiently large and appropriate sample. Yet breadth should not be confused with representativeness. Collecting responses from thousands of conveniently recruited participants does not automatically produce evidence that represents the intended population.
Standardization asks how consistently evidence is collected
Standardization means keeping important aspects of data collection consistent across participants, cases, or observations. Highly standardized questionnaires present the same items and response options. Standardized interviews may require interviewers to use the same wording, order, and neutral probing procedures.
Consistency can improve comparability and reduce variation introduced by differences in administration. It can also simplify coding and analysis when researchers need to compare responses across many participants.
Standardization, however, requires researchers to decide much of the data structure in advance. If an important issue was omitted from a closed questionnaire, respondents generally cannot make the instrument ask the missing question for you.
Flexibility allows the collection process to respond to what emerges
Flexible data collection permits researchers to adapt questions, probes, sampling, observations, or lines of inquiry as relevant information emerges. This feature is particularly prominent in many qualitative approaches, where data collection and analysis may develop iteratively.
In a semi-structured interview, researchers can ask all participants about central topics while allowing follow-up questions to respond to individual answers. If participants repeatedly introduce an unexpected issue, later interviews may explore it more deliberately when doing so is consistent with the methodological design.
Qualitative methodological guidance commonly emphasizes that data collection may remain open and responsive during fieldwork, with decisions about whom to speak with, what to observe, and what to ask shaped partly by emerging understanding.
Flexibility is not the absence of rigor. It should be purposeful and consistent with the research approach. Changing procedures haphazardly because the researcher forgot to plan something is not methodological flexibility.
Depth and breadth often compete for finite resources
Imagine you have 40 researcher-hours available for participant data collection. You might conduct 20 two-hour sessions, 40 one-hour sessions, or administer a short questionnaire to far more participants.
The first design can generate much more information about each participant. The questionnaire can extend much further across the population. Neither is inherently superior because they support different forms of inference.
This trade-off is one reason surveys, interviews, focus groups, and observation should be chosen according to what the question requires rather than according to a simple hierarchy of methods.
Standardization and flexibility also create tension
Highly standardized collection improves consistency partly by limiting variation in how evidence is elicited. Flexible approaches create room to clarify meanings, pursue unexpected responses, and respond to contextual differences.
Consider the question “What barriers prevent faculty members from using a new technology?” A closed questionnaire can present the same predetermined list of barriers to every respondent. That makes frequencies and comparisons relatively straightforward. Yet participants can report only the barriers the researchers anticipated unless an open-response option is provided.
A semi-structured interview can uncover barriers the research team did not anticipate and explore how several barriers interact. But different participants may discuss different issues in different levels of detail, making simple item-by-item comparison less straightforward.
The choice therefore depends on whether comparability across respondents or responsiveness to individual and contextual variation is more important for the research purpose.
Semi-structured approaches deliberately occupy the middle ground
The trade-offs are not binary. Data collection methods can be designed along a continuum.
Semi-structured interviews illustrate this well. Researchers usually begin with a set of topics or guiding questions, which creates some consistency across interviews. At the same time, interviewers can probe responses, alter the order when appropriate, and follow relevant ideas introduced by participants.
This provides more flexibility than a fully standardized interview and more structure than an unstructured conversation. The balance should be deliberate rather than accidental.
Open-ended questions can add flexibility without producing interview-level depth
Researchers sometimes add open-ended questions to structured surveys to capture responses that predefined categories may miss. This can broaden the kinds of information participants can provide.
Yet a free-text box is not equivalent to an interview. Researchers cannot immediately ask what a vague response means, request an example, explore a contradiction, or pursue an unexpected idea. Participants also vary considerably in how much they are willing to type.
Open-ended survey items can therefore provide useful qualitative material while preserving much of the logistical breadth of a survey, but researchers should not assume that they reproduce the depth or interaction of an interview.
Flexibility can reduce comparability if it is not managed carefully
If every participant receives completely different questions, researchers may later discover that crucial topics were explored with some participants but never discussed with others. This may be appropriate in some highly emergent designs, but it can be problematic when the study requires comparison across cases.
Researchers using flexible methods therefore often maintain common areas of inquiry while allowing the route through them to vary. Interview guides, observational frameworks, fieldwork protocols, reflexive documentation, and iterative analytic decisions can provide structure without eliminating responsiveness.
Standardization can create an illusion of equivalence
Asking everyone exactly the same question does not guarantee that everyone interprets it in exactly the same way.
Terms such as “regularly,” “engagement,” “effective,” or “AI use” may carry different meanings for different participants. Standardized administration can ensure that wording is consistent while leaving variation in interpretation unresolved.
This is why instrument development, piloting, cognitive testing where appropriate, construct definition, and validity evidence matter. Standardization reduces some sources of variation; it does not make meaning automatically uniform.
Methodological fit should determine the balance
The right combination depends on which data collection approach actually fits the research question.
A question asking how common a clearly defined attitude is across a population may prioritize standardization and breadth. A question asking how researchers experience rejection during peer review may require greater depth and flexibility. A comparative qualitative study may need enough consistency across interviews to examine common issues while retaining flexibility to pursue participant-specific experiences.
Researchers should therefore avoid treating depth, breadth, standardization, or flexibility as universally desirable. Their value depends on what the study is trying to know.
