Wooldrige says that ‘intuitively, to estimate k+1 parameters, we need at least k+1 observations’. Why is this the case?


The $k+1$ parameters are $k+1$ unknowns. In general, you need at least $k+1$ equations (which are observations in the context of OLS estimation) to uniquely pin down those $k+1$ unknowns.

  • $\begingroup$ So the ‘observations’ are the equations which comprise the OLS first order conditions? $\endgroup$ – Ali Lodhi Nov 18 '19 at 12:29
  • $\begingroup$ @AliLodhi: I'm not sure what you mean by "OLS first order conditions", but I have in mind the OLS regression model $y_i=\beta_0+\beta_1x_{1i}+\cdots+\beta_k x_{ki}+\epsilon_i$, where each observation $i$ is an instantiation of this equation. $\endgroup$ – Herr K. Nov 18 '19 at 17:26

The intuition is that OLS is a linear model and to estimate any linear model you need at least 2 points in 2D space. The reason for that is that with a single point you can’t uniquely identify any line.

Adding extra parameter increases the dimensions and in each higher dimension you need one more point to estimate linear model. You can think about it in a way that you need at least one point per dimension but in 1D space there is no line since 1D would be just Y axis hence you get minimum k+1 points.

  • 1
    $\begingroup$ "with a single point you can’t crate any line" --- I would beg to differ: through a single point you can create infinitely many lines. $\endgroup$ – Herr K. Nov 17 '19 at 20:19
  • $\begingroup$ @HerrK. You are right it should be identify unique line I changed it $\endgroup$ – 1muflon1 Nov 17 '19 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.