The conditional mean of $Y$ given $X$ is:$$E[Y|X]=\int yf(y|x)dy$$ What is the relation to the linear model? I have read somewhere that when X and $Y$ are normal (their marginal distributions), then the conditional mean of $Y$ becomes a linear function of $X$. How can I see this?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ If $X$ and $Y$ are independent then your claim is clearly false so I am guessing you omitted some conditions. $\endgroup$– GiskardJan 19, 2016 at 19:57
-
$\begingroup$ If X and Y are independent, Y would become a constant function (simply the mean of Y)...I am thinking of non degenerate cases..but thanks $\endgroup$– ChinGJan 19, 2016 at 20:03
-
1$\begingroup$ I was confused because you denote the variable $Y$ and the conditional mean of $Y$ with the same letter in your question. (I think you meant to write "$E[Y|X]$ becomes a linear") Please consider editing this so that the question is easier to understand for future readers. $\endgroup$– GiskardJan 19, 2016 at 22:05
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
If X and Y are normal, then the distribution of X conditional on Y is: $$ X | (Y = y) = N(\mu_x + \rho \frac{\sigma_x}{\sigma_y}(y - \mu_y), (1-\rho)^2 \sigma^2_x )$$
Therefore, $E[X|Y = y] = \mu_x + \rho \frac{\sigma_x}{\sigma_y}(y - \mu_y) = (\mu_x - \mu_y (\rho \frac{\sigma_x}{\sigma_y})) + (\rho \frac{\sigma_x}{\sigma_y})y $ which is linear in $y$. Symmetrically: $$E[Y|X = x] = (\mu_y - \mu_x (\rho \frac{\sigma_y}{\sigma_x})) + (\rho \frac{\sigma_y}{\sigma_x})x $$ which is also linear in $x$
-
$\begingroup$ Thanks a lot..how did you get the first step? Is it a property of normal distributions? $\endgroup$– ChinGJan 19, 2016 at 20:05
-
$\begingroup$ In fact from your equation, one can clearly see that the coefficient is simply the cov(x,y)/var(x). Thanks! $\endgroup$– ChinGJan 19, 2016 at 20:08
-
1$\begingroup$ It is a well known expression but I checked it against the Wikipedia page en.wikipedia.org/wiki/Multivariate_normal_distribution $\endgroup$– BKayJan 19, 2016 at 20:24
$\begingroup$
$\endgroup$
1
A pair of random variables $(Y,X)$ that has a joint bivariate distribution that belongs to the Elliptical Symmetric Family and to the Pearson Family (they overlap), have the property that the associated conditional expectation functions (of $Y$ given $X$ but also of $X$ given $Y$) are linear (more generally, affine) functions.
Examples include the Normal distribution and Student's $t$-distribution. Other bivariate distributions that have affine conditional expectation functions are the Pareto, Beta, Gamma, F-, Binomial, Poisson, and Negative Binomial.
answered Jan 19, 2016 at 22:44
-
$\begingroup$ Thanks a lot for your answer. Why is it then, that even when we have absolutely no reason to believe that Y and X have joint normals or marginal normals that we stick to linear regression? $\endgroup$– ChinGJan 21, 2016 at 17:17