3
$\begingroup$

The adjusted R^2 formula is :

$$ \overline{R}^{2}=1-\left( \left( 1-R^{2}\right) \cdot > \dfrac{n-1}{n-k}\right) $$

In case of k > 1 , I continue like that;

$$ \overline{R}^{2}=1-\left( \dfrac{n-1}{n-k}-\dfrac{n-1}{n-k}R^{2}\right) $$

then

$$ \left( n-k\right) \cdot \overline{R}^{2}=n-k-\left( n-1\right) +\left( n-1\right) R^{2} $$

so $$ \left( n-k\right) \cdot \overline{R}^{2}-\left( n-1\right) R^{2}=1-k $$

but after that I don't know how to proceed, is there someone who has an idea?

$\endgroup$

3 Answers 3

8
$\begingroup$

$$ SSRes=\sum_{i=1}^n\left( y_i-\hat y_i \right)^2\\ SSTotal=\sum_{i=1}^n\left( y_i-\bar y \right)^2 $$

$$ R^2=1-\dfrac{ SSRes/(n-1) }{ SSTotal/(n-1) } $$

$$ \bar R^2=1-\dfrac{ SSRes/(n-k) }{ SSTotal/(n-1) } $$

When $k>1$, the numerator of the second equation will be larger, meaning that the second equation subtracts a larger number from $1$ than does the first equation.

Thus, $R^2\ge \bar R^2.$ $\square$

A way to think about $R^2$ is that it compares the error variance to the total variance, and adjusted $\bar R^2$ does the same but with an estimate of the error variance that is slightly larger. Since that estimate of the error variance is slightly larger, adjusted $\bar R^2$ must be slightly less favorable (slightly smaller).

$\endgroup$
2
$\begingroup$

That's a great answer from Dave. I have another answer.

There is an alternative form of writting $\bar{R}^2$ in terms of $R^2$, with $p$ being the number of covariates in the model and $n$ the number of observations: $$ \bar{R}^2 = \frac{(n-1)R^2 - p}{n-p-1} $$ Now, subtract $p$ and add $p$ inside $(n-1)$: $$ \bar{R}^2 = \frac{(n-1)R^2 - p}{n-p-1} = \frac{(n-p-1+p)R^2 - p}{n-p-1} = \frac{(n-p-1)R^2 + pR^2 - p}{n-p-1} = \frac{(n-p-1)R^2 - p(1-R^2)}{n-p-1} = R^2 - \frac{p(1-R^2)}{n-p-1} $$

We can see that $\bar{R}^2$ is $R^2$ minus something nonnegative whenever $n > p + 1$, so $\bar{R}^2$ will be less or equal to $R^2$. For the "equal" part, $\bar{R}^2 = 1$ only when $R^2 = 1$ (knowing that $0 \le R^2 \le 1$).

$\endgroup$
1
$\begingroup$

Continuing from \begin{align*} \left( n-k\right) \cdot \overline{R}^{2}-\left( n-1\right) \cdot R^{2} & = 1-k \\ \left( n-k\right) \cdot \overline{R}^{2} + (k-1) \cdot 1 & = \left( n-1\right) \cdot R^{2}. \end{align*} So $R^{2}$ is a weighted average of $\overline{R}^{2}$ and $1$, and since $\overline{R}^{2} \leq 1$: \begin{align*} \left( n-k\right) \cdot \overline{R}^{2} + (k-1) \cdot \overline{R}^{2} & \leq \left( n-1\right) \cdot R^{2} \\ \left(n-1\right) \cdot \overline{R}^{2} & \leq \left( n-1\right) \cdot R^{2} \\ \overline{R}^{2} & \leq R^{2} \\ \end{align*}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.