Wooldrige says that ‘intuitively, to estimate k+1 parameters, we need at least k+1 observations’. Why is this the case?
2 Answers
$\begingroup$
$\endgroup$
2
The $k+1$ parameters are $k+1$ unknowns. In general, you need at least $k+1$ equations (which are observations in the context of OLS estimation) to uniquely pin down those $k+1$ unknowns.
-
$\begingroup$ So the ‘observations’ are the equations which comprise the OLS first order conditions? $\endgroup$ Nov 18, 2019 at 12:29
-
$\begingroup$ @AliLodhi: I'm not sure what you mean by "OLS first order conditions", but I have in mind the OLS regression model $y_i=\beta_0+\beta_1x_{1i}+\cdots+\beta_k x_{ki}+\epsilon_i$, where each observation $i$ is an instantiation of this equation. $\endgroup$– Herr K.Nov 18, 2019 at 17:26
$\begingroup$
$\endgroup$
2
The intuition is that OLS is a linear model and to estimate any linear model you need at least 2 points in 2D space. The reason for that is that with a single point you can’t uniquely identify any line.
Adding extra parameter increases the dimensions and in each higher dimension you need one more point to estimate linear model. You can think about it in a way that you need at least one point per dimension but in 1D space there is no line since 1D would be just Y axis hence you get minimum k+1 points.
answered Nov 17, 2019 at 15:08
-
1$\begingroup$ "with a single point you can’t crate any line" --- I would beg to differ: through a single point you can create infinitely many lines. $\endgroup$– Herr K.Nov 17, 2019 at 20:19
-
$\begingroup$ @HerrK. You are right it should be identify unique line I changed it $\endgroup$– 1muflon1 ♦Nov 17, 2019 at 20:45