03 · What You Need to Know
Why Methodological Consultation Often Matters Most Before Data Collection
A Statistician Does More Than Choose Statistical Tests
Statistical consultation is sometimes imagined as a final-stage service: researchers collect data, send a spreadsheet, and ask which test to run.
That substantially understates the potential contribution of statistical expertise. Statisticians may contribute to study design, randomization, sampling, outcome definition, sample-size and precision planning, measurement schedules, analysis plans, missing-data strategies, data-quality procedures, model selection, sensitivity analysis, interpretation, and reporting.
NIH's Biostatistics and Clinical Epidemiology Service, for example, describes consultation across study design, protocol development, sample-size calculations, statistical analysis plans, data analysis, interpretation, and manuscript preparation. University biostatistics units similarly encourage investigators to seek consultation during proposal and protocol development rather than only after data collection.
The underlying reason is simple: statistical analysis begins conceptually long before statistical software is opened.
The Best Time Is Before Irreversible Design Decisions
The most useful rule is not “consult exactly six months before data collection” or any other fixed timetable. It is to involve the relevant expertise before decisions that depend on that expertise become difficult or impossible to change.
If the primary outcome is unclear, seek input before finalizing the instrument. If clustering affects sample size, seek input before recruitment targets are fixed. If randomization is required, involve a statistician before allocation begins. If a longitudinal design may need additional measurement occasions, discuss it before participants have completed the study.
This is why analysis planning should occur before data collection. Consultation is most powerful while analytical reasoning can still improve the design.
Involve a Statistician When Sample Size Is Not Straightforward
Sample-size calculations are often treated as formulas into which researchers insert an effect size, alpha level, and desired power. Real studies can be considerably more complicated.
Clustered designs, repeated measurements, unequal allocation, multiple primary outcomes, noninferiority or equivalence questions, survival outcomes, attrition, complex sampling, rare events, and specialized models can require assumptions that interact with the analysis.
A statistician can help identify which quantity the sample-size calculation should target, which assumptions are needed, how uncertainty in those assumptions affects feasibility, and whether the proposed primary analysis is consistent with the calculation.
This work needs to occur before recruitment. A retrospective calculation cannot increase the sample that has already been collected.
Seek Input When the Design Creates Dependency or Multiple Levels
Studies involving repeated measurements, students within classrooms, patients within hospitals, participants within communities, matched observations, family members, or other dependent data structures often require analytical and sample-size considerations beyond methods that assume independent observations.
The relevant expertise can help determine the unit of analysis, identify clustering or repeated structure, evaluate whether the number of higher-level units is adequate, and select methods consistent with how the data will be generated.
Ignoring these issues at the design stage can leave researchers with a large number of individual observations but too few independent clusters to support the intended analysis.
Seek Input When You Are Unsure Whether the Design Can Answer the Question
A methodological consultation is particularly valuable when the uncertainty is not “Which test should I use?” but “Can this study actually answer what I want to know?”
A methodologist may help distinguish descriptive, associational, predictive, and causal questions; evaluate whether the proposed comparison is defensible; identify sources of bias; clarify temporal ordering; or suggest a stronger feasible design.
The aim is not to make the study statistically elaborate. It is to ensure that the planned analysis, research question, and design are aligned.
Seek Input Before Choosing or Constructing Important Outcome Measures
Measurement decisions can determine the quality of everything that follows.
If a study relies on a new scale, composite outcome, diagnostic threshold, repeated measure, latent construct, or complex scoring procedure, expertise in psychometrics, measurement, epidemiology, statistics, or the substantive field may be needed before the instrument is finalized.
A statistician cannot later recover reliability, validity, temporal coverage, or construct information that the measurement process failed to capture. Nor is every measurement problem primarily statistical. The appropriate specialist may be a psychometrician, qualitative researcher, measurement expert, or domain specialist.
Seek Input When Randomization or Allocation Is Involved
Randomized studies require decisions about allocation ratio, randomization procedures, stratification, blocking, concealment, and sometimes clustering or adaptive features.
ICH E9 describes statistical principles as relevant to trial design and analysis, including randomization, sample size, multiplicity, and the specification of principal analyses. Formal clinical-trial requirements do not apply wholesale to every experiment, but the broader lesson is useful: allocation is a design feature with analytical consequences.
Randomization should therefore be planned before participants are assigned, not reconstructed after researchers have already allocated them.
Seek Input When There Are Several Plausible Analyses
Sometimes the difficulty is not finding an analysis but choosing among several defensible ones.
Methods may differ in their analytical target, assumptions, handling of repeated observations, treatment of missingness, efficiency, or interpretability. A statistician can help determine which should be primary and which might serve as sensitivity analyses.
That conversation is especially valuable before the results are known. Once researchers can see which analysis produces the preferred conclusion, methodological judgment can become entangled with outcome preference.
If multiple analyses could answer the same research question, advance consultation can help establish their roles before comparison of their outputs becomes possible.
Seek Input When Missing Data Could Be Consequential
Missing data are partly an analytical issue and partly a design problem.
Before collection, methodological input can help identify why observations might be lost, improve follow-up procedures, determine which auxiliary information should be retained, and plan analyses and sensitivity analyses appropriate to plausible missing-data mechanisms.
After collection, statistical expertise can help evaluate the observed missingness and implement suitable methods. But preventing avoidable missing data is usually preferable to searching for a sophisticated correction afterward.
Seek Input for Complex Surveys and Sampling Designs
Studies using stratification, clustering, unequal selection probabilities, survey weights, multistage sampling, or finite-population designs can require specialized survey methodology.
Consultation before sampling can influence how units are selected, how sample sizes are allocated across strata, what information must be retained for weighting, and how variance will eventually be estimated.
If sampling probabilities or cluster identifiers were never recorded, some design-based analyses may become difficult or impossible to reconstruct later.
Qualitative Studies May Need a Different Kind of Methodologist
Not every methodological problem belongs to a statistician.
A qualitative study may benefit from consultation about methodological coherence, sampling, interviewing, observation, reflexivity, coding, interpretive analysis, or the fit between epistemological assumptions and analytical approach. Asking a quantitative statistician to resolve those issues merely because the project contains “data” may not provide the expertise the study needs.
Similarly, qualitative analysis should be considered before data collection, particularly when collection and analysis are intended to proceed iteratively.
Mixed-Methods Research May Require Expertise in Integration
A mixed-methods study may have competent quantitative and qualitative components yet remain methodologically weak if nobody has planned how they will be integrated.
Mixed-methods expertise can help determine the design, sequencing, priority of strands, participant linkage, sampling connections, joint displays, and development of integrated conclusions.
The relevant question is not simply whether the team contains both a statistician and a qualitative researcher. Someone also needs to understand how the two forms of evidence will be brought together through a coherent mixed-methods integration plan.
Consultation Is Particularly Valuable When the Method Is New to the Team
A method can be well established in the literature and still be inappropriate for a team to implement without support if nobody has sufficient experience with it.
This may apply to multilevel models, causal inference methods, structural equation models, Bayesian analyses, survival methods, machine-learning workflows, complex qualitative approaches, psychometric models, or mixed-methods designs.
Methodological expertise is not simply knowing which menu item to select in software. It includes understanding assumptions, diagnostics, limitations, interpretation, and what the method can and cannot establish.
Consultation After Data Collection Is Still Worthwhile
Early consultation is preferable when design decisions remain open, but that does not mean researchers should avoid asking for help once the data exist.
If you have already collected data that you do not know how to analyze, consultation can help determine what questions the dataset can support, identify appropriate methods, diagnose limitations, and prevent inappropriate analysis.
Bring more than the spreadsheet. Provide the research question, protocol, design, sampling procedures, instruments, variable definitions, timing of measurements, data-management decisions, and any original analysis plan.
A consultant needs to understand how the data came into existence before deciding what can reasonably be done with them.
Not Every Study Needs a Statistician
Recommending early consultation does not mean every research project requires a professional statistician.
A research team may already have sufficient methodological competence for a straightforward design and analysis. Some studies do not use statistical inference at all. Others need expertise from a different methodological discipline.
The practical criterion is whether the team can competently justify and execute the consequential methodological decisions the study requires. If not, seek the relevant expertise before those decisions become irreversible.
Watch Out
Do not add a statistician's name to a project merely to signal methodological credibility. Consultation should involve substantive intellectual contribution, and authorship or acknowledgment should follow the applicable contribution and publication policies rather than professional title alone.
