Basic Fundamentals question on regression analysis based on economic theory:

Question

In running any regression on demand based on parameters derived from theory, I'm having some confusion in some order of operations when the results come out inconsistent.

For instant in the case of multi-collinearity and having to drop parameters -- what is the case when parameters of equal importance to theory (like price's) have perfect collinearity? They can't be indexed or made into any ratio, so must be dropped.

My default is to discriminate between the parameters on their explanatory power -- R squared. But what about F-test of significance? If one is higher R than the other but may be nullified by the insignificant F test. Would the 'normal' protocol still be to include the parameter that results with that same higher R-squared despite the F-test being insignificant. -- Or is it even not possible given the parameters are in the same model the F-test is being ran on?

Answer 1

You don't drop parameters - you drop explanatory variables. Perfect colinearity is first and foremost a sample problem, it relates to the actual data series you have, not to the theoretical relation between two regressors, although, if you have a large enough sample, it is valid to think that the colinearity property may hold at population level also.

Assume the model

$$y = b_0 + b_1x_1 + b_2x_1 +u$$

Say that you try to run OLS and you get a message of perfect or near-perfect co-linearity between $x_1$ and $x_2$. This means that the data tell you that an affine relation exists like

$$x_2 =\delta_0 + \delta x_1$$

But this means that the regression specification is

$$y = b_0 + b_1x_1 + b_2(\delta_0 + \delta x_1) +u$$

$$\implies y = \gamma_0 + \gamma_1x_1 +u$$

$$\gamma_0 = b_0 + b_2\delta_0,\;\;\; \gamma_1=b_1 +b_2\delta_1$$

This does not negate or contradict the theory. You just found out that the two variables are in such a relation between them, that their effect on the dependent variable cannot be separately estimated using linear methods like least-squares. You didn't find out that one of them has no effect.

Answer 2

You never consider either $r^2$ or significance in choosing a model. $r^2$ is non-decreasing in the number of parameters. You can improve $r^2$ quite often by adding a set of unrelated variables, such as the rainfall in Quebec to estimates of the price of diamonds in Singapore. Significance assumes you believe the null is true. If you have multiple models, then you obviously do not believe one to be true.

When using Frequentist methods, model selection should be done with something similar to the Akaike Information Criterion or the Bayesian Information Criterion. Both are summary point estimates of a stylized Bayesian posterior density over the model space. Only use one, don't use both. There are also a few other similar methods. The information criterion are essentially Bayesian solutions with slightly different prior mass functions.

You should write software to go through each combination of variable, excluding variables that are collinear, and run either the AIC or the BIC. The AIC and the BIC measure how close a model is to the data generating function. This is not like significance an doesn't imply significance.