1
$\begingroup$

The conditional mean of $Y$ given $X$ is:$$E[Y|X]=\int yf(y|x)dy$$ What is the relation to the linear model? I have read somewhere that when X and $Y$ are normal (their marginal distributions), then the conditional mean of $Y$ becomes a linear function of $X$. How can I see this?

$\endgroup$
  • $\begingroup$ If $X$ and $Y$ are independent then your claim is clearly false so I am guessing you omitted some conditions. $\endgroup$ – Giskard Jan 19 '16 at 19:57
  • $\begingroup$ If X and Y are independent, Y would become a constant function (simply the mean of Y)...I am thinking of non degenerate cases..but thanks $\endgroup$ – ChinG Jan 19 '16 at 20:03
  • 1
    $\begingroup$ I was confused because you denote the variable $Y$ and the conditional mean of $Y$ with the same letter in your question. (I think you meant to write "$E[Y|X]$ becomes a linear") Please consider editing this so that the question is easier to understand for future readers. $\endgroup$ – Giskard Jan 19 '16 at 22:05
2
$\begingroup$

If X and Y are normal, then the distribution of X conditional on Y is: $$ X | (Y = y) = N(\mu_x + \rho \frac{\sigma_x}{\sigma_y}(y - \mu_y), (1-\rho)^2 \sigma^2_x )$$

Therefore, $E[X|Y = y] = \mu_x + \rho \frac{\sigma_x}{\sigma_y}(y - \mu_y) = (\mu_x - \mu_y (\rho \frac{\sigma_x}{\sigma_y})) + (\rho \frac{\sigma_x}{\sigma_y})y $ which is linear in $y$. Symmetrically: $$E[Y|X = x] = (\mu_y - \mu_x (\rho \frac{\sigma_y}{\sigma_x})) + (\rho \frac{\sigma_y}{\sigma_x})x $$ which is also linear in $x$

$\endgroup$
  • $\begingroup$ Thanks a lot..how did you get the first step? Is it a property of normal distributions? $\endgroup$ – ChinG Jan 19 '16 at 20:05
  • $\begingroup$ In fact from your equation, one can clearly see that the coefficient is simply the cov(x,y)/var(x). Thanks! $\endgroup$ – ChinG Jan 19 '16 at 20:08
  • 1
    $\begingroup$ It is a well known expression but I checked it against the Wikipedia page en.wikipedia.org/wiki/Multivariate_normal_distribution $\endgroup$ – BKay Jan 19 '16 at 20:24
1
$\begingroup$

A pair of random variables $(Y,X)$ that has a joint bivariate distribution that belongs to the Elliptical Symmetric Family and to the Pearson Family (they overlap), have the property that the associated conditional expectation functions (of $Y$ given $X$ but also of $X$ given $Y$) are linear (more generally, affine) functions.

Examples include the Normal distribution and Student's $t$-distribution. Other bivariate distributions that have affine conditional expectation functions are the Pareto, Beta, Gamma, F-, Binomial, Poisson, and Negative Binomial.

$\endgroup$
  • $\begingroup$ Thanks a lot for your answer. Why is it then, that even when we have absolutely no reason to believe that Y and X have joint normals or marginal normals that we stick to linear regression? $\endgroup$ – ChinG Jan 21 '16 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.