0
$\begingroup$

Some textbooks (for example, Introductory Econometrics: A Modern Approach by Jeffrey Wooldridge) assume that no perfect multicollinearity for the OLS regression, while others (for example, Econometrics by Fumio Hayashi) just assume that no multicollinearity. Are there any differences between these two assumptions? and what are the differences?

Thanks in advanced!

$\endgroup$
3
$\begingroup$

Perfect multicolinearity means that two independent variables are perfectly correlated with their $r^2=1$ multicolinearity technically refers to any non zero correlation between two independent variables but when it is mentioned as a problem it usually implies that the correlation is high. So there is a difference between the two terms perfect multicollinearity is a special case of multicolinearity.

Also they have different implications for OLS. Under perfect multicolinearity OLS cannot be identified. Under high muliticolinearity OLS can be identified but standard errors will be inflated.

However, to make the things confusing, I noticed that sometimes people use the terms interchangeably. I only have the Wooldridge book so I can’t check the text of the other one to compare it but I think they actually refer to the same thing.

| improve this answer | |
$\endgroup$
0
$\begingroup$

So, let us begin with an observation that you are missing. There are three major branches of statistics and multicollinearity is not an intrinsic issue in all of them.

The class of metrics you are generally called "Frequentist." Ordinary least squares is likely the simplest case for regression. In it, you will lack full rank if they are perfectly collinear. Your determinant will be zero. Since you cannot divide by zero, you cannot have a solution under perfect collinearity.

Nonetheless, the solution for perfect collinearity is simple, remove one or more variables until your determinant is non-zero. There cannot be an information loss because they are collinear.

If a regression is nearly collinear then the determinant will be very close to zero quite often. The difficulty is that very small changes in the determinant will have extraordinary consequences for the stability of the estimators.

For example, there is a very small difference between $$\frac{1}{33}\text{ and }\frac{1}{50}$$ is very small, but if the difference between $33x$ and $50x$ can be huge. Because of this intrinsic instability, nearly trivial changes in a sample can create large and unpredictable effects on the location of the estimators.

Least-squares estimators minimize the joint error but not necessarily the error on a variable-by-variable basis.

The reason for the instability is related to the double counting of information. Frequency methods are optimal, on average, over the sample space. Because of this averaging process, the same information can enter the calculation more than one time when variables are not orthogonal. The squares minimizing tool is trying to be optimal over the entire sample space but can be a poor tool on a subset of the sample space even if it would be good overall.

That contrasts with Bayesian methods where the multiplicative nature of the calculation creates the situation where multicollinearity doesn't have to create instability. Instead, the added information and structure causes little improvement in the certainty of the model. Because the posterior has an automatic penalty for the added structure, it may be the case that inference over the model space may recommend removing variables because they add to overall uncertainty while adding little new information. It can be the case, with multicollinearity that a model with two independent variables adds more certainty than a model with ten variables.

In a sense that may be the same as with the Frequentist model. If you required all ten variables then it may be difficult to find the true location of the parameters in a Bayesian space. Instead of "instability", you get model uncertainty.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.