Welch’s t-test for two independent means
See why Welch’s and Student’s t-tests can give different confidence intervals for the same mean difference. Work through a fictional example with groups of 12 and 30.
AcademicFix Editorial Team · Sources checked
Bu rehberi Türkçe okuyun- Mean difference (A − B)
- +5.50 points
- Welch 95% confidence interval
- [−0.34, 11.34]
- Pooled 95% confidence interval
- [1.48, 9.52]
Fictional example. Group A has 12 observations and group B has 30. The last row shows what the common-variance assumption does to this calculation.
Compare the two calculationsThe group of 12 has more variable scores than the group of 30. Both calculations put the mean difference at 5.50 points. Welch’s confidence interval includes zero, while the pooled interval lies above zero. The two methods use group variability differently when they estimate the standard error. If your study compares two independent means and you cannot justify a common population variance, plan to use Welch’s t-test. NIST, two-sample t-test · Delacre et al., 2017
This guide is for researchers comparing the same continuous outcome in two independent groups. The worked example estimates a two-sided mean difference on the original score scale.
On this page
Are the observations independent?
Start with the unit of observation. A design can have two independent groups when each student belongs to one group and the individual observations are independent. Before-and-after scores from the same student, or one-to-one matches, call for a paired analysis. Clustered observations, such as students in the same classroom, fall outside this guide’s scope. NIST, two-sample t-test · Rasch et al., 2011
Welch’s method estimates each group’s variance separately. The classical pooled Student test combines the sample variances under a common population variance assumption. When small groups show strong skew or contain outliers, inspect the measurements and distributions. Assess whether the standard Welch inference is suitable for that design and those distributions. Delacre et al., 2017 and the 2022 correction · Rasch et al., 2011
Use the guide to normality checks and Q-Q plots to examine the group distributions.
Why avoid choosing the test from Levene’s result?
Levene’s test assesses whether the population variances are equal. A large p-value leaves uncertainty about that equality. If you run Levene’s test and then choose between Student’s and Welch’s tests on the same observations, the combined decision rule can change the error rates for inference about the means. Justify your primary method before examining the results. Delacre et al., 2017 · Rasch et al., 2011
Make the analysis plan specific. For example, state that you will compare the two independent mean scores using a two-sided Welch t-test and report A − B with a 95% confidence interval.
Why do the intervals differ for the same 5.50 points?
Fictional example. A research team plans to use the same score scale after two types of practice. These invented summaries illustrate the calculation.
| Group | n | Mean | SD |
|---|---|---|---|
| A | 12 | 74.50 | 9.00 |
| B | 30 | 69.00 | 4.00 |
Both methods estimate 74.50 − 69.00 = 5.50 points. In the Welch calculation, group A contributes 9² / 12 = 6.75 to the squared standard error and group B contributes 4² / 30 ≈ 0.53. The standard error is the square root of their sum. The smaller, more variable group contributes about 93% of that sum in this example.
The Welch–Satterthwaite degrees of freedom are (a + b)² / [a² / (nA − 1) + b² / (nB − 1)], where a = sA² / nA and b = sB² / nB. For the null hypothesis of zero difference, divide the estimated difference by the standard error to obtain t. To build the confidence interval, multiply the standard error by the appropriate two-sided t critical value and use that margin on either side of the estimate. NIST Technical Note 2106, §3.1 · R stats, t.test source
| Measure | Welch | Pooled Student |
|---|---|---|
| Difference | 5.50 | 5.50 |
| Standard error | 2.70 | 1.99 |
| Degrees of freedom | 12.78 | 40 |
| t statistic | 2.04 | 2.77 |
| Two-sided p | 0.0628 | 0.0085 |
| 95% confidence interval | [−0.34, 11.34] | [1.48, 9.52] |
The pooled variance gives greater weight to the larger, less variable group B (29/40 versus 11/40). In this example, the pooled calculation gives a smaller standard error and a narrower interval. Use the comparison to understand the consequence of the common-variance assumption. Choose the primary analysis from your design and analysis plan. NIST, two-sample t-test · Delacre et al., 2017
How do you report a Welch result?
Fictional results sentence. The mean score was 74.50 in group A (SD = 9.00, n = 12) and 69.00 in group B (SD = 4.00, n = 30). Welch’s t-test estimated a mean difference (A − B) of 5.50 points (95% CI [−0.34, 11.34]), t(12.78) = 2.04, p = 0.0628 (two-sided).
At the prespecified 5% level, the test did not reject a difference of zero. The interval spans a small negative difference and positive differences of more than 11 points. Report that uncertainty and explain what difference would matter on your scale. See the guide to p-values, effect sizes and confidence intervals.
Four decisions to record in the analysis plan
- Record the unit of observation, group membership and any matching or clustering.
- Specify the mean difference, its direction and the confidence level in advance.
- Inspect each group’s distribution and outliers, then document why your chosen method suits the design.
- Report the group sizes, means, standard deviations and uncertainty together.
Before collecting data, plan sample size and power. Check that the calculation’s design and assumptions match your analysis plan.
Sources
- NIST/SEMATECH · Two-Sample t-Test for Equal Means
- NIST Technical Note 2106 · Comparing Instruments, §3.1
- Delacre, Lakens and Leys (2017) · Welch’s and Student’s t-tests
- Delacre, Lakens and Leys (2022) · Published correction
- Rasch, Kubinger and Moder (2011) · Pre-testing the two-sample t-test
- R stats · t.test implementation