When is an OLS parameter unchanged on a subsample?

Question

There is a sample of $n$ observations, each element has a numeric $Y$ and $X$ characteristic. There is an OLS regression over the sample $$ Y = b_0 + b_1 X + \textbf{u}, $$ $\textbf{u}$ being the vector of residuals. Suppose we remove observation $i$ from the sample, restricting it to $n-1$ observations, and run an OLS regression again, yielding $$ Y_{-i} = b_0' + b_1' X_{-i} + \textbf{u}'. $$ Subscript $_{-i}$ denotes that $Y$ and $X$ are not the same vectors as before, as observation $i$ is missing.

Seems to me that if the observation $i$ we removed is on the original sample's 'regression line', that is if $$ Y_i = b_0 + b_1 X_i, $$ then $b_0 = b_0'$ and $b_1 = b_1'$.

Example: (in R code)

x = c(5,3,4,5,4,4)
y = c(20,15,14,21,10,25)

plot(x,y)
abline(coef(reg))

plot(x[-1],y[-1])

reg1 = lm(y[-1] ~ x[-1])
abline(coef(reg1))

My questions are:

1) Is this true, and if yes, what is the proof?
[I figured this one out in the meantime, but feel free to give a nicer solution.]

2) Is it possible that after removing an observation (not on the 'regression line') we have

($b_0 = b_0'$ and $b_1 \neq b_1'$) or ($b_0 \neq b_0'$ and $b_1 = b_1'$)?

Answer 1

Since Q1 has been solved, I'll focus on Q2. Yes, it's possible to remove a sample point that is not on the regression line but still yields $b_{1}=b_{1}'$ and $b_{0}\ne b_{0}'$. Such points have the property $x_{j}=\bar{x}$, where $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_{i}$. These points are known as points without leverage.

Suppose $\left(x_{j},y_{j}\right)$ is a point without leverage but not on the regression line, that is, $x_{j}=\overline{x}$ and $y_{j}\ne\overline{y}$. We know

\begin{align*} b_{1} & =\frac{\sum_{i}\left(x_{i}-\bar{x}\right)y_{i}}{\sum_{i}\left(x_{i}-\bar{x}\right)^{2}}\\ & =\frac{\sum_{-j}\left(x_{i}-\bar{x}\right)y_{i}+\left(x_{j}-\overline{x}\right)y_{j}}{\sum_{-j}\left(x_{i}-\bar{x}\right)^{2}+\left(x_{j}-\overline{x}\right)^{2}}\\ & =\frac{\sum_{-j}\left(x_{i}-\bar{x}\right)y_{i}}{\sum_{-j}\left(x_{i}-\bar{x}\right)^{2}}\\ & =\frac{\sum_{-j}\left(x_{i}-\bar{x}_{-j}\right)y_{i}}{\sum_{-j}\left(x_{i}-\bar{x}_{-j}\right)^{2}}\\ & =b_{1}' \end{align*} here we use the conditions $x_{j}-\overline{x}=0$ and $\overline{x}=\overline{x}_{-j}$. Combined with the condition $\overline{y}\ne\overline{y}_{-j}$, we have \begin{align*} b_{0}' & =\overline{y}_{-j}-b_{1}'\overline{x}_{-j}\\ & \ne\overline{y}-b_{1}\overline{x}\\ & =b_{0} \end{align*}

Answer 2

I think I figured out 1). I am still interested in 2).

OLS sets $b_0$ and $b_1$ in such a way that

$$ (b_0 \ \ b_1) = \arg\min_{a_0, a_1} \sum_j \left(Y_j - a_0 - a_1 X_j\right)^2. $$ The first order conditions are $$ (-2)\sum_j \left(Y_j - b_0 - b_1 X_j\right) = 0 $$ and $$ (-2)\sum_j X_j \left(Y_j - b_0 - b_1 X_j\right) = 0. $$ If observation $i$ is on the regression line then $$ \begin{align*} \left(Y_i - b_0 - b_1 X_i\right) & = 0 \\ \\ X_i\left(Y_i - b_0 - b_1 X_i\right) & = 0. \end{align*} $$ It follows from this and the original first order conditions that $$ \begin{align*} (-2)\sum_{j\neq i} \left(Y_j - b_0 - b_1 X_j\right) & = 0 \\ \\ (-2)\sum_{j\neq i} X_j\left(Y_j - b_0 - b_1 X_j\right) & = 0, \end{align*} $$ which are the first order conditions of the subsample's OLS problem.