# Smaller variance of estimators in matrix sense

When comparing asymptotic variances, you usually conclude that a covariance matrix A is more efficient whenever the difference B - A ≥ 0 (where ≥ 0 means that the difference is a positive semidefinite matrix).

However, this comparison involves all elements in the matrices - both the diagonal elements (asymptotic variances of each beta estimator) and the off-diagonal ones (covariances between estimators) -, while we are usually interested in estimators for a particular beta coefficient having small standard errors.

Could anyone explain why is this method preferred to a comparison that entails only the diagonal elements? Could it be the case that some diagonal element is smaller in B while B - A ≥ 0?

Thank you!

• Hi: the covariance of a random vector, $X$, is its covariance matrix along with the diagonal elements. But the scalar variance of a random vector, $X$, is $X^\prime A X$ which is a scalar. So, this is why they compare them that way. If one matrix is "greater", then the variance of a given random vector, $X$ is always less if it has $A$ as its covariance matrix rather than $B$ as its covariance matrix. Jan 21 '19 at 15:59
• An estimator can be more efficient than another, not clear what it means to write "a covariance matrix A is more efficient" Jan 21 '19 at 16:41
• Note that efficiency denotes whether the variance of some estimator reaches it's CR lower bound so I don't think the term "efficient" applies here, atleast the way you explained the notion of $B-A$ being greater than or equal to zero. Jan 22 '19 at 2:39
• Thank you! Just to clarify, 1) Bertrand: indeed "a covariance matrix A is more efficient" makes little sense. I meant an estimator with covariance matrix A being more efficient than another estimator. 2) Mark, indeed I should have used the term "relative efficiency" . Jan 23 '19 at 0:26

Relative efficiency between two unbiased estimators $$\widehat{\theta }_{A}$$ and $$\widehat{\theta }_{B}$$ of an unknown parameter vector $$\theta _{0}\in \mathbb{R}^{K}$$ is usually defined as follow (see for instance Ruud, 2001). The estimator $$\widehat{\theta }_{A}$$ is said to be efficient relative to $$\widehat{\theta }_{B}$$ if we have: $$\mathrm{V}\left[ \widehat{\theta }_{A}\right] \equiv \Omega _{A}<<\Omega _{B}\equiv \mathrm{V}\left[ \widehat{\theta }_{B}\right] .$$

If $$\Omega _{B}-\Omega _{A}$$ is positive definite, then the diagonal terms of $$\Omega _{B}$$ and $$\Omega _{A}$$ are necessarily such that $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}.$$ Indeed if $$v^{T}\left( \Omega _{B}-\Omega _{A}\right) v>0$$ for any $$v\neq 0$$ then for $$v=e_{i}$$ we catch the diagonal terms out of $$\Omega _{B}-\Omega _{A}$$ and find that $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}.$$ The converse, however, it is not true: the mere conditions $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}$$ do not garantee that $$\Omega _{B}-\Omega _{A}$$ be positive definite. Covariances matter.

• The answer to your second question is: no it never happens that $$\sigma _{Bii}^{2}<\sigma _{Aii}^{2}$$ when $$\Omega _{B}>>\Omega _{A}$$.

• The answer to your first question is: it is important to consider all covariance terms. If we want that the confidence ellipse (at any threshold) of $$\widehat{\theta }_{A}$$ be nested within the confidence ellipse of $$\widehat{\theta }_{B}$$ then we need $$\Omega _{A}<<\Omega _{B}$$ and not just $$\sigma _{Aii}^{2}<\sigma _{Bii}^{2}$$.

See Ruud, (2001, chapter 9) for a proof and detailed explanations. An example is provided here, illustrating that the confidence ellipses are nested when $$\Omega _{B}-\Omega _{A}$$ positive definite, and not nested if $$\Omega _{B}-\Omega _{A}$$ is not positive definite.

Example: $$\begin{eqnarray*} \widehat{\theta }_{A}\left( a\right) &\sim &\mathcal{N}\left( 0,\Omega _{A}\left( a\right) \right) ,\qquad \widehat{\theta }_{B}\sim \mathcal{N}% \left( 0,\Omega _{B}\right) \\ \Omega _{A}\left( a\right) &=&\left( \begin{array}{cc} 4 & a \\ a & 9% \end{array}% \right) ,\qquad \Omega _{B}=\left( \begin{array}{cc} 5 & 0 \\ 0 & 10% \end{array}% \right). \end{eqnarray*}$$

For $$a=0$$ the matrix $$\Omega _{B}-\Omega _{A}\left( 0\right)$$ is positive definite and the $$95\%$$ threshold confidence ellipses are nested. The Figure below (left panel) represents the iso-curves centered on $$\theta _{0}=0$$ and whose equations are given by $$v^{T}\left( \Omega _{A}\left( 0\right) \right) ^{-1}v=5.99$$ and $$x^{T}\Omega _{B}^{-1}x=5.99$$ and illustrates that for $$a=0$$ the later is nested within the former. This is no longer true for $$a=-5$$ (right panel) in which case $$\Omega _{B}-\Omega _{A}\left( -5\right)$$ is no longer positive definite. In this case the probability that $$\widehat{\theta }_{A}$$ is further away than $$\widehat{\theta }_{B}$$ from the true value $$% \theta _{0}=0$$ is positive, and $$\widehat{\theta }_{A}$$ is no longer efficient (case with $$a=-5$$) relatively to $$\widehat{\theta }_{B}$$. Note that the variances always satisfy $$\sigma _{Bii}^{2}>\sigma _{Aii}^{2}$$ but this is not sufficient for the ellipses to be nested, the covariances must also satisfy $$\left( \sigma _{B11}^{2}-\sigma _{A11}^{2}\right) \left( \sigma _{B22}^{2}-\sigma _{A22}^{2}\right) -\left( \sigma _{B12}-\sigma _{A12}\right) ^{2}>0$$, which is not the case in this example for $$a=-5$$.

• Betrand: That's interesting but I don't think OP is interested in efficiency. I think he's interested in what it means when one says that one covariance matrix $B$ is greater than another covariance matrix $A$, in the sense of $(B-A) > 0$. That's my take but I could be wrong. Thanks for insights. Jan 22 '19 at 20:59
• Mark, I tried in the answer to clarify the question and had to introduce two characters $\widehat{\theta }_{A}$ and $\Omega_A$ instead of using only $A$. Well, I will delete the answer when it is downvoted. Jan 22 '19 at 21:15
• Bertrand, I have found your answer very helpful. Could you please clarify (if relation to my first question) what do you mean by the "confidence ellipse" (at any threshold) of an estimator be nested within the confidence ellipse of another? Thank you! Jan 23 '19 at 0:28
• Bertrand: I think your answer was great. I just wasn't sure if that was the question the OP was asking. Sounds like it was so my mistake and thanks for great explanation. Jan 23 '19 at 15:55
• econ86: I have added an example in my last edit, hope that it helps. Jan 23 '19 at 16:13