0
$\begingroup$

I'm cross-posting this question on both Economics and Cross Validated to get answers from a different perspective on each field. It is generally accepted to cross-post if the question is tailored to fit better in each community. See: https://meta.stackexchange.com/a/64069/510233

Cross-posted in Cross Validated: https://stats.stackexchange.com/q/403213/243829


I'll try to be as detailed as possible to best deliver what I want to know in this question.

In economic data, there are some variables that are composed of many other variables. Mostly indices or multiples. I can think of some examples like CPI, Consumer Sentiment Index, or even EV/EBITDA, PER.

Better and more general example would be something like Wind chill Temperature Index which is

enter image description here

where:

  • WCI = wind chill index, kcal/m2/h
  • v = wind velocity, m/s
  • Ta = air temperature, °C

What I want to know is whether a variable like WCI can have more meanings(effects) than v and Ta seperately combined.

Based on what I learned in regression analysis from Ecnonometrics class, a simple interaction term like enter image description here would be interpreted as the difference in marginal effect between a control group (D=0) and a treatment group (D=1), when expressed like: enter image description here

Also, more general interaction term like VariableA * VariableB would be interpreted as an impact of VariableA on the coefficient of VariableB, in some examples like

PollutionLevel = B0 + B1*Population + B2*NumberOfCars + B3*Population*NumberOfCars
= B0 + B1*Population + (B2 + B3*Population)*NumberOfCars 

(originally from a question posted on Cross Validated)

But I think these interpretations are too simple to capture the whole meaning(effect) of a new variable derived from other variables, which leads to other questions like:

  1. What would be the interpretation of a super complex interaction term like WCI in regression perspective? (and other data science methods like Random Forest or Deep Neural Network, etc if possible.)

  2. Would it still be valid and preferable to include an explanatory variable like WCI in the model when I already have v and Ta? Would it make the model more accurate?

  3. Intuitively, I feel like people would care more about important indices but not necessarily the variables it is composed of. Can a complex interaction term derived from other variables have more effect than a simple combination of the other variables?

The third one is related to my original question of whether one variable that is composed of other variables in a dataset can have more effect on the dependent variable.

One example I can think of is the steepness of the yield curve that is considered very important in the bond market, which is a mere slope of the interest rates of bonds with different YTM.

People seem to care about the slope of the yield curve more than individual rates of each bond so I think it is a legit explanatory variable but it is unclear whether it is justified to introduce this new variable from existing variables or how it should be interpreted.

$\endgroup$
  • $\begingroup$ What do you mean by "v and Ta seperately combined". Their product (i.e. a quadratic term)? The short answer is that "more meaning" depends on having some theoretical justification for using a specific formula to combine any given variables. $\endgroup$ – Fizz Apr 15 at 20:01
  • $\begingroup$ @Fizz I was being somewhat unclear since I wasn't even sure how to describe this problem. What I meant by "v and Ta separately combined" was whether including WCI in regression would make the model better - in terms of its efficiency (less variance in OLS) and accuracy (unbiasedness), larger predictive power, etc. $\endgroup$ – user8491363 Apr 16 at 1:08
3
$\begingroup$

What you describe as "complex interaction term" is typically referred to as composite variable. Interaction terms are products of two or more variables. You also use terms like "effect" and "meaning", which in more technical sense, seem to refer to issues of unbiasedness, accuracy, and interpretation.

If a composite variable is a linear combination of other variables (e.g., their sum), then including that variable over its components will not change the predictive power of a linear regression model, though it will change the interpretation of regression coefficient(s). If it is a non-linear combination, such as the case with your WCI variable, then you will obtain different predictions in a linear regression model. If you are interested in interpreting the effect of WCI on your dependent/outcome variable (or if you believe it lies in the causal pathway and affects the interpretation of another variable of interest), then you should include WCI in the model. The interpretation will be standard for regression models, i.e., by how much the value of dependent variable is expected to change for a one-unit increase in the value of WCI, holding other variables constant. If, however, you are interested in the effect of wind velocity or temperature on the outcome, then you will have a hard time interpreting the results with WCI used as explanatory variable. Including interaction terms or composite variables can also lessen the precision of your coefficient estimates, just like it does when you have multicollinearity.

In purely predictive (machine learning) models, you are most interested in the predictive power of your model, not in the interpretation of variables. Those models readily accept "engineered" variables (hence, "feature engineering" is super important for models like random forests, though not for deep learning). Accuracy will depend on a lot of factors, but using composite variables can certainly help.

$\endgroup$
  • $\begingroup$ So to summarize what you just said, either y = B0 + B1*v + B2*Ta or y = B0 + B1*WCI is fine but NOT y = B0 + B1*v + B2*Ta + B3*WCI? $\endgroup$ – user8491363 Apr 16 at 1:38
  • $\begingroup$ How a model is specified really depends on your research question and theoretical justification, as @Fizz stated in their comment. My point above is that a model with all three of these variables used will make it pretty much impossible to interpret estimated coefficients (betas) and will increase variance, though it may help with predictions (I don't know what your dependent variable is). $\endgroup$ – AlexK Apr 16 at 1:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.