# Very basic question about confidence intervals

Suppose that I estimate a (frequentist) confidence interval for the sample mean of a variable $$X$$, say at the 95% level. Suppose I also estimate a 95% confidence for the sample mean of a variable $$Y$$.

Claim: we cannot reject the hypothesis that $$X$$ and $$Y$$ have the same mean at the 5% level if and only if these confidence intervals overlap.

Is this true? I have a vague recollection that the answer is 'no'.

Yes your vague recollection is correct. You can verify it with a trivial example.

Lets take $$\bar{X}=10$$, with $$\sigma(X)=2$$ and $$n=30$$. So the $$95\%$$ confidence interval (using student's t distribution) is approximately given by $$10 ± 0.75$$.

Now suppose $$\bar{Y}=11$$ with $$\sigma(Y)=1$$ and $$n=30$$ so the $$95\%$$ confidence interval is given by approximately $$11 ± 0.37$$.

Confidence intervals above clearly overlap since for $$\bar{X}$$ the upper bound is 10.75, but for $$\bar{Y}$$ the lower bound is $$10.63$$ but when we actually do the t-test for difference in their means we get:

$$t=\frac{10-11}{\sqrt{\left(\frac{2^2}{30}+\frac{1^2}{30}\right)}} \\ \approx -2.5$$

Hence in this case we clearly can reject the null hypothesis that the means are same even at $$5\%$$ (in fact the $$t$$-stat is very close to being rejected at $$1\%$$ even) despite that their confidence levels happen to overlap.

• Thanks, very helpful! But I remain a bit confused. Say $\bar{Y} > \bar{X}$ as in your example. Then the intervals overlap (using the CLT approximation) iff $\bar{Y} - 1.96SE(\bar{Y}) > \bar{X} + 1.96SE(\bar{X})$, or $\bar{Y} - \bar{X} > 1.96[SE(\bar{X}) + SE(\bar{Y})]$. Meanwhile, the means are `statistically different' iff $\bar{Y} - \bar{X} - 1.96SE(\bar{X} - \bar{Y}) = \bar{Y} - \bar{X} - 1.96[SE(\bar{X}) + SE(\bar{Y})] > 0$: same inequality! So what am I missing? Feb 11 at 13:38
• Perhaps I am wrong that the standard error of the difference in means is the sum of the standard errors of the means? Feb 11 at 13:40
• @afreelunch yes that does not hold since generally (save for special cases) $se(X)-se(Y)\neq se(Y-X)$ (also inside se() you will not add bar - bar $\bar{X}$ stands for mean of random variable $X$ whether se(X) is standard error of random variable X).
– 1muflon1
Feb 11 at 13:51
• By 'standard error', I mean standard deviation of the sample mean; not the standard deviation of the random variable (I think this is the usual terminology?) Feb 11 at 16:16
• But I think I have found my mistake: $$SE[\bar{X} - \bar{Y}] = Var[\bar{X} - \bar{Y}]^{0.5} = [Var[\bar{X}] + Var[\bar{Y}]]^{0.5} = [SE[\bar{X}]^2 + SE[\bar{Y}]^2]^{0.5}$$ which does not generally equal $$SE[\bar{X}] + SE[\bar{Y}]$$ So the standard error of the difference in means is not equal to the sum of standard errors. Feb 11 at 16:22