Smaller variance of estimators in matrix sense

When comparing asymptotic variances, you usually conclude that a covariance matrix A is more efficient whenever the difference B - A ≥ 0 (where ≥ 0 means that the difference is a positive semidefinite matrix).

However, this comparison involves all elements in the matrices - both the diagonal elements (asymptotic variances of each beta estimator) and the off-diagonal ones (covariances between estimators) -, while we are usually interested in estimators for a particular beta coefficient having small standard errors.

Could anyone explain why is this method preferred to a comparison that entails only the diagonal elements? Could it be the case that some diagonal element is smaller in B while B - A ≥ 0?

Thank you!

• Hi: the covariance of a random vector, $X$, is its covariance matrix along with the diagonal elements. But the scalar variance of a random vector, $X$, is $X^\prime A X$ which is a scalar. So, this is why they compare them that way. If one matrix is "greater", then the variance of a given random vector, $X$ is always less if it has $A$ as its covariance matrix rather than $B$ as its covariance matrix. – mark leeds Jan 21 at 15:59
• An estimator can be more efficient than another, not clear what it means to write "a covariance matrix A is more efficient" – Bertrand Jan 21 at 16:41
• Note that efficiency denotes whether the variance of some estimator reaches it's CR lower bound so I don't think the term "efficient" applies here, atleast the way you explained the notion of $B-A$ being greater than or equal to zero. – mark leeds Jan 22 at 2:39
• Thank you! Just to clarify, 1) Bertrand: indeed "a covariance matrix A is more efficient" makes little sense. I meant an estimator with covariance matrix A being more efficient than another estimator. 2) Mark, indeed I should have used the term "relative efficiency" . – econ86 Jan 23 at 0:26

Relative efficiency between two unbiased estimators $$\widehat{\theta }_{A}$$ and $$\widehat{\theta }_{B}$$ of an unknown parameter vector $$\theta _{0}\in \mathbb{R}^{K}$$ is usually defined as follow (see for instance Ruud, 2001). The estimator $$\widehat{\theta }_{A}$$ is said to be efficient relative to $$\widehat{\theta }_{B}$$ if we have: $$\mathrm{V}\left[ \widehat{\theta }_{A}\right] \equiv \Omega _{A}<<\Omega _{B}\equiv \mathrm{V}\left[ \widehat{\theta }_{B}\right] .$$

If $$\Omega _{B}-\Omega _{A}$$ is positive definite, then the diagonal terms of $$\Omega _{B}$$ and $$\Omega _{A}$$ are necessarily such that $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}.$$ Indeed if $$v^{T}\left( \Omega _{B}-\Omega _{A}\right) v>0$$ for any $$v\neq 0$$ then for $$v=e_{i}$$ we catch the diagonal terms out of $$\Omega _{B}-\Omega _{A}$$ and find that $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}.$$ The converse, however, it is not true: the mere conditions $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}$$ do not garantee that $$\Omega _{B}-\Omega _{A}$$ be positive definite. Covariances matter.

• The answer to your second question is: no it never happens that $$\sigma _{Bii}^{2}<\sigma _{Aii}^{2}$$ when $$\Omega _{B}>>\Omega _{A}$$.

• The answer to your first question is: it is important to consider all covariance terms. If we want that the confidence ellipse (at any threshold) of $$\widehat{\theta }_{A}$$ be nested within the confidence ellipse of $$\widehat{\theta }_{B}$$ then we need $$\Omega _{A}<<\Omega _{B}$$ and not just $$\sigma _{Aii}^{2}<\sigma _{Bii}^{2}$$.

See Ruud, (2001, chapter 9) for a proof and detailed explanations. An example is provided here, illustrating that the confidence ellipses are nested when $$\Omega _{B}-\Omega _{A}$$ positive definite, and not nested if $$\Omega _{B}-\Omega _{A}$$ is not positive definite.

Example: $$\begin{eqnarray*} \widehat{\theta }_{A}\left( a\right) &\sim &\mathcal{N}\left( 0,\Omega _{A}\left( a\right) \right) ,\qquad \widehat{\theta }_{B}\sim \mathcal{N}% \left( 0,\Omega _{B}\right) \\ \Omega _{A}\left( a\right) &=&\left( \begin{array}{cc} 4 & a \\ a & 9% \end{array}% \right) ,\qquad \Omega _{B}=\left( \begin{array}{cc} 5 & 0 \\ 0 & 10% \end{array}% \right). \end{eqnarray*}$$

For $$a=0$$ the matrix $$\Omega _{B}-\Omega _{A}\left( 0\right)$$ is positive definite and the $$95\%$$ threshold confidence ellipses are nested. The Figure below (left panel) represents the iso-curves centered on $$\theta _{0}=0$$ and whose equations are given by $$v^{T}\left( \Omega _{A}\left( 0\right) \right) ^{-1}v=5.99$$ and $$x^{T}\Omega _{B}^{-1}x=5.99$$ and illustrates that for $$a=0$$ the later is nested within the former. This is no longer true for $$a=-5$$ (right panel) in which case $$\Omega _{B}-\Omega _{A}\left( -5\right)$$ is no longer positive definite. In this case the probability that $$\widehat{\theta }_{A}$$ is further away than $$\widehat{\theta }_{B}$$ from the true value $$% \theta _{0}=0$$ is positive, and $$\widehat{\theta }_{A}$$ is no longer efficient (case with $$a=-5$$) relatively to $$\widehat{\theta }_{B}$$. Note that the variances always satisfy $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}$$ but this is not sufficient for the ellipses to be nested, the covariances must also satisfy $$\left( \sigma _{B11}^{2}-\sigma _{A11}^{2}\right) \left( \sigma _{B22}^{2}-\sigma _{A22}^{2}\right) -\left( \sigma _{B12}-\sigma _{A12}\right) ^{2}>0$$, which is not the case in this example for $$a=-5$$. • Betrand: That's interesting but I don't think OP is interested in efficiency. I think he's interested in what it means when one says that one covariance matrix $B$ is greater than another covariance matrix $A$, in the sense of $(B-A) > 0$. That's my take but I could be wrong. Thanks for insights. – mark leeds Jan 22 at 20:59
• Mark, I tried in the answer to clarify the question and had to introduce two characters $\widehat{\theta }_{A}$ and $\Omega_A$ instead of using only $A$. Well, I will delete the answer when it is downvoted. – Bertrand Jan 22 at 21:15
• Bertrand, I have found your answer very helpful. Could you please clarify (if relation to my first question) what do you mean by the "confidence ellipse" (at any threshold) of an estimator be nested within the confidence ellipse of another? Thank you! – econ86 Jan 23 at 0:28
• Bertrand: I think your answer was great. I just wasn't sure if that was the question the OP was asking. Sounds like it was so my mistake and thanks for great explanation. – mark leeds Jan 23 at 15:55
• econ86: I have added an example in my last edit, hope that it helps. – Bertrand Jan 23 at 16:13