Research data analysis guide
How to choose a statistical analysis for a research question
A decision framework that starts with the research question and data-generating process—not a menu of tests chosen after seeing significant results.
AcademicFix
A defensible analysis links the estimand, design, variables, assumptions and reporting plan before software produces a p-value.
AcademicFix
Decision check
- 01
The target question and estimand are stated in plain language
- 02
Study design and sampling structure are reflected in the model
- 03
Outcome and predictor measurement levels are correctly identified
- 04
Assumptions and missing-data decisions are documented
- 05
Effect sizes, uncertainty and reproducible outputs are planned
1 / 5
Define the question and the quantity you want to estimate
Decide whether the objective is description, group comparison, association, prediction, causal estimation, scale evaluation or time-to-event analysis. Then state the target quantity: a mean difference, odds ratio, correlation, change over time, predictive error or another estimand.
The same variables can support different analyses when the question changes. Choosing a test from variable names alone skips the scientific decision that gives the result meaning.
- Analysis objective is stated before naming a test
- Target population and quantity are explicit
- Confirmatory and exploratory questions are distinguished
2 / 5
Map the design, dependence and sampling process
Record whether observations are independent, paired, repeated, clustered, longitudinal or spatially related. Note randomisation, stratification, weighting and unequal follow-up where applicable.
A method that assumes independent observations can understate uncertainty when participants contribute repeated measurements or are nested within sites. Design information must therefore enter the model, not remain only in the methods narrative.
- Unit of analysis matches the sampling unit
- Repeated or clustered observations are identified
- Design weights or strata are considered when relevant
3 / 5
Classify variables and inspect data quality
Identify the outcome type and how predictors were measured: continuous, count, binary, ordinal, nominal, time-to-event or repeated response. Preserve meaningful coding and record transformations rather than forcing every variable into a familiar test.
Before modelling, inspect ranges, impossible values, duplicates, missingness patterns and distributional shape. Data cleaning decisions should be reproducible and should not be driven by whether they make a result statistically significant.
- Outcome type matches the proposed model family
- Coding, missing values and exclusions are documented
- Outliers are investigated rather than automatically deleted
4 / 5
Choose the simplest model that answers the question
Use the design and outcome to define a defensible candidate model, then evaluate its assumptions. A two-group mean comparison, generalised linear model, mixed model, survival model or non-parametric procedure is appropriate only when its structure matches the question and data.
Advanced labels do not make an analysis stronger. Mediation, moderation, structural equation models, multilevel models and machine-learning workflows require additional assumptions, sample-size considerations and validation plans that must be priced and scoped separately in professional work.
- Model complexity is justified by the question and design
- Assumptions have planned diagnostics or robust alternatives
- Multiple testing and model-selection decisions are addressed
5 / 5
Plan transparent reporting and reproducibility
Report descriptive context, effect estimates, uncertainty intervals, exact sample sizes, missing-data handling, assumption checks and software details alongside p-values. A non-significant result is not proof of no effect, and statistical significance is not practical importance.
Keep an analysis script or syntax file, a data dictionary and an output log tied to the reported tables. The manuscript should be traceable back to genuine author-supplied data without hidden manual edits to results.
- Effect sizes and uncertainty are planned
- Tables can be reproduced from saved code or syntax
- Exploratory changes are labelled rather than concealed
Research data analysis guide
Frequently asked questions
Can variable types alone determine the correct statistical test?
No. Variable types narrow the options, but the research question, design, dependence, sampling process and intended interpretation are also required.
Is a non-parametric test always safer when data are non-normal?
No. The relevant assumptions concern the model and estimand, not only a histogram. Sample size, design, robustness and the exact hypothesis should guide the decision.
Why are advanced analyses quoted separately?
They require additional design checks, assumptions, diagnostics, reporting and often iterative model comparison. A file and data review is needed before defining a responsible scope.