# When is an OLS parameter unchanged on a subsample?

There is a sample of $n$ observations, each element has a numeric $Y$ and $X$ characteristic. There is an OLS regression over the sample $$Y = b_0 + b_1 X + \textbf{u},$$ $\textbf{u}$ being the vector of residuals. Suppose we remove observation $i$ from the sample, restricting it to $n-1$ observations, and run an OLS regression again, yielding $$Y_{-i} = b_0' + b_1' X_{-i} + \textbf{u}'.$$ Subscript $_{-i}$ denotes that $Y$ and $X$ are not the same vectors as before, as observation $i$ is missing.

Seems to me that if the observation $i$ we removed is on the original sample's 'regression line', that is if $$Y_i = b_0 + b_1 X_i,$$ then $b_0 = b_0'$ and $b_1 = b_1'$.

Example: (in R code)

x = c(5,3,4,5,4,4)
y = c(20,15,14,21,10,25)

plot(x,y)
abline(coef(reg)) plot(x[-1],y[-1])

reg1 = lm(y[-1] ~ x[-1])
abline(coef(reg1)) My questions are:

1) Is this true, and if yes, what is the proof?
[I figured this one out in the meantime, but feel free to give a nicer solution.]

2) Is it possible that after removing an observation (not on the 'regression line') we have

($b_0 = b_0'$ and $b_1 \neq b_1'$) or ($b_0 \neq b_0'$ and $b_1 = b_1'$)?

Since Q1 has been solved, I'll focus on Q2. Yes, it's possible to remove a sample point that is not on the regression line but still yields $b_{1}=b_{1}'$ and $b_{0}\ne b_{0}'$. Such points have the property $x_{j}=\bar{x}$, where $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_{i}$. These points are known as points without leverage.
Suppose $\left(x_{j},y_{j}\right)$ is a point without leverage but not on the regression line, that is, $x_{j}=\overline{x}$ and $y_{j}\ne\overline{y}$. We know
\begin{align*} b_{1} & =\frac{\sum_{i}\left(x_{i}-\bar{x}\right)y_{i}}{\sum_{i}\left(x_{i}-\bar{x}\right)^{2}}\\ & =\frac{\sum_{-j}\left(x_{i}-\bar{x}\right)y_{i}+\left(x_{j}-\overline{x}\right)y_{j}}{\sum_{-j}\left(x_{i}-\bar{x}\right)^{2}+\left(x_{j}-\overline{x}\right)^{2}}\\ & =\frac{\sum_{-j}\left(x_{i}-\bar{x}\right)y_{i}}{\sum_{-j}\left(x_{i}-\bar{x}\right)^{2}}\\ & =\frac{\sum_{-j}\left(x_{i}-\bar{x}_{-j}\right)y_{i}}{\sum_{-j}\left(x_{i}-\bar{x}_{-j}\right)^{2}}\\ & =b_{1}' \end{align*} here we use the conditions $x_{j}-\overline{x}=0$ and $\overline{x}=\overline{x}_{-j}$. Combined with the condition $\overline{y}\ne\overline{y}_{-j}$, we have \begin{align*} b_{0}' & =\overline{y}_{-j}-b_{1}'\overline{x}_{-j}\\ & \ne\overline{y}-b_{1}\overline{x}\\ & =b_{0} \end{align*}
OLS sets $b_0$ and $b_1$ in such a way that
$$(b_0 \ \ b_1) = \arg\min_{a_0, a_1} \sum_j \left(Y_j - a_0 - a_1 X_j\right)^2.$$ The first order conditions are $$(-2)\sum_j \left(Y_j - b_0 - b_1 X_j\right) = 0$$ and $$(-2)\sum_j X_j \left(Y_j - b_0 - b_1 X_j\right) = 0.$$ If observation $i$ is on the regression line then \begin{align*} \left(Y_i - b_0 - b_1 X_i\right) & = 0 \\ \\ X_i\left(Y_i - b_0 - b_1 X_i\right) & = 0. \end{align*} It follows from this and the original first order conditions that \begin{align*} (-2)\sum_{j\neq i} \left(Y_j - b_0 - b_1 X_j\right) & = 0 \\ \\ (-2)\sum_{j\neq i} X_j\left(Y_j - b_0 - b_1 X_j\right) & = 0, \end{align*} which are the first order conditions of the subsample's OLS problem.