Confusion over homoskedacity assumption

Question

In my introductory econometrics course, we have been discussing the GM-assumptions and homoskedasticity. Unfortunately, I have some confusions and interlinked questions, so I am wondering if somebody could help me with my understanding please?

model: $y_i = \beta_0 + \beta_1 x_i + u_i $

What is the difference between $E[u] = 0$ and $E[u|x] = 0$? Does the first imply the latter?

I understand that the latter marks that the conditional mean, ie that the expected value of $u$ is $\sigma^2$ irrespective of $x$, but isn't this implied by the unconditional mean? Is one of them a stronger assumption?

Do the above imply $cov(u,x) = 0$ and how? What is the difference between $cov(y,x)$ and $cov(u_i,x_i)$?

We have stated that the homoskedasticity assumption implies zero covariance between $u$ and $x$, but how does it come? And why is it labelled a "weaker assumption" (at least by Woolridge)? What really confuses me is that we often use the notation $cov(u_i,x_i)$ instead of $cov(u,x)$ - is there any difference? If the first one relates to a specific $i$, does it really have a meaning at all?

Apologies if these questions are rather trivial. And thank you very much for your help!

Answer 1

Intuitive explanation might help.

(i) $E[u]=0$ vs $E[u|x]=0$: Imagine partitioning the population by the value of $x$ so that each slice of population has the same value of $x$ in it. You can then get the mean of $u$ for each slice. "$E[u|x]=0$", which is a shortcut notation of "$E[u|x=c]=0$ for (almost) all $c$", means that the average of $u$ in every partition is zero. You can also get the mean of $u$ for the whole population. "$E[u]=0$" means that the average of $u$ for the whole population is zero.

For example, let $x$ be years of education. $E[u|x]=0$ means that the average of $u$ for those with 9 years of education is zero, the average of $u$ for those with 10 years of education is zero, etc., while $E[u]=0$ means that the average of $u$ for the whole population is zero.

You see that "$E[u|x]=0$" implies some sort of "$u$ being not related with $x$", and it's called mean independence. On the other hand, $E[u]=0$ does not say anything about the relationship between $u$ and $x$. $E[u]=0$ makes sense without $x$ ever being involved, while $E[u|x]=0$ makes sense only in relation to $x$.

(ii) $var(u)=\sigma^2$ vs $var(u|x)=\sigma^2$: Imagine partitioning the population by the value of $x$. You can get the variance of $u$ for every slice. '$var(u|x)=\sigma^2$' means the variance of $u$ for every partition is $\sigma^2$. Here, the key point is that $\sigma^2$ is a constant and is not involved with $x$. It means that all the slices have the same variance. $var(u|x)=\sigma^2$ is very informative. Next, you can also get the variance of $u$ for the whole population, and $var(u)=\sigma^2$ means that that variance (of $u$ for the whole population) is denoted $\sigma^2$, where $\sigma^2$ is just a notation.

$var(u|x)$ can depend on $x$ (heteroskedasticity), but it is totally nonsense to question whether $var(u)$ depends on $x$ because $var(u)$ has nothing to do with $x$ from the beginning (unless you mean $var(u|x)$ by $var(u)$).

(iii) $cov(x,u)=0$: This just means that $cov(x,u)=0$, where $cov(a,b)$ is defined as $E[(a-Ea)(b-Eb)]$. In your case, $cov(x,u)=E[xu]$ because $E[u]=0$. If you need intuition about the meaning of $cov(x,u)=0$, imagine you have $(x,u)$ values plotted on the XY plane for the whole population, with $x$ on the horizontal axis and $u$ on the vertical axis. You draw a nice straight line (where being straight is important). $cov(x,u)=0$ means that the straight line is horizontal. It says something about $u$ and $x$ being not related, and when it happens, we say that "$x$ and $u$ are uncorrelated."

$E[u|x]=0$ implies $cov(x,u)=0$ but not vice versa, which is the reason why $cov(x,u)$ is weaker than $E[u|x]=0$. You can prove it using the law of iterated expectations: $E[xu] = E[xE(u|x)] = E[x\cdot 0]=E[0]=0$. For a counter example for the other way around, suppose that $x\sim N(0,1)$ and $u=x^2-1$. Then $E[u|x]=x^2-1$, which is not zero except $x=\pm 1$ but $cov(x,u) = E[x(x^2-1)] = E[x^3]-E[x]=0-0=0$.

(iv) What is the difference between $cov(y,x)$ and $cov(u,x)$? If $y=\beta_0 + \beta_1 x +u$, $cov(y,x) = cov(\beta_0+\beta_1 x+u,x) = \beta_1 cov(x,x) + cov(u,x)$, where $cov(x,x) = var(x)$ by definition. That's the difference.

ADD:

I've just noticed the OP's confusion about $cov(x_i,u_i)$ and $cov(x,u)$. First, $cov(x,u)$ can be intuitively understood as the population property explained above in (iii). $cov(x,u)$ is about the population, and does not say anything about the sample. The notation $cov(x_i,u_i)$ is on the other hand about the sample, and should in fact mean something like "$cov(x_i,u_i)$, $i=1,\ldots,n$", i.e., $cov(x_1,u_1)$, $cov(x_2,u_2)$, ..., and $cov(x_n,u_n)$. They are all the same if the first person, the second person, ..., the $n$th person are independent random draws from the same population (which is the meaning of $iid$). If they are drawn from different populations, $cov(x_i, u_i)$ may be different for different $i$, but I am pretty sure you are assuming $iid$.

Now, what is $cov(x_1,u_1)$ then? This is the hard part. To understand its meaning, you should understand that $(x_1,u_1)$ is a random vector, whose value can change when you repeat sampling in your thought experiments. (This will keep confusing you until you understand this point.) The "first person" in the sample will keep changing when you repeat drawing the sample over and over again in your mind, and thus the $(x_1,u_1)$ value will keep changing over repeated samples. So, as you repeat sampling indefinitely, the values of $x_1$ and $u_1$ will change and make a (joint) distribution. $cov(x_1,u_1)$ is the covariance of that joint distribution. Likewise, you can understand $cov(x_2,u_2)$ as the covariance of $x_2$ and $u_2$ over the repeated samples. If you keep drawing the first observation independently from the same population as you talk about $cov(x,u)$, then $cov(x_1,u_1) = cov(x,u)$.

So when we talk about the population property, we use the notation $cov(x,u)$. When we talk about the covariance of $x_1$ and $u_1$ (over repeated samples), we use the notation $cov(x_1,u_1)$. The notation $cov(x_i,u_i)$ is a (sloppy) shortcut notation of $cov(x_1,u_1), \ldots, cov(x_n,u_n)$. "$cov(x_i,u_i)$" itself does not make much sense until you say what $i$ is, like in "$cov(x_i,u_i)$ is zero for every $i=1,\ldots,n$", or "$cov(x_i,u_i)$ is nonzero for every $i=1,\ldots,n$ but they are the same for all $i$." The statement "$cov(x_i,u_i)=0$" is usually a lazy (or clumsy) writing of "$cov(x_i,u_i)=0$ for all $i=1,\ldots,n$", which means $cov(x_1,u_1)=0$, $cov(x_2,u_2)=0$, ..., $cov(x_n,u_n)=0$.

Answer 2

Just to add on to the previous answer, since I do not have enough reputation to comment:

In general, for random variables $X$ and $Y$, $\mathop{{}\mathbb{E}}[Y|X = x]$ indicates what the expected value of $Y$ is, given that $X = x$. Note that this is a real number! Further, if $X$ and $Y$ are independent, then $\mathop{{}\mathbb{E}}[Y|X = x] = \mathop{{}\mathbb{E}}[Y]$.

However, we can use this to define the conditional expectation of $Y$ given $X$ as a random variable, defined on the range of X: $$\mathop{{}\mathbb{E}}[Y|X](x) = \mathop{{}\mathbb{E}}[Y|X = x]$$ This is a random variable because we do not know apriori the value of $X$. $x$ is the value the random variable $X$ takes depending on the outcome. Thus, the 'source' of randomness is $X$.

An important relation is: $$\mathop{{}\mathbb{E}}[Y] = \mathop{{}\mathbb{E}}[\mathop{{}\mathbb{E}}[Y|X]]$$ where the outermost expectation is with respect to the value $X$ takes.

We can now answer your question.

$\mathop{{}\mathbb{E}}[u|x] = 0$ for all $x$ implies $\mathop{{}\mathbb{E}}[u] = 0$ using the above relation, and noting that the expectation of a constant random variable is the same constant (here, $0$).