0
$\begingroup$

These questions arose when I was reading Online Appendix D for the paper Missing Events in Event Studies: Identifying the Effects of Partially-Measured News Surprises by R.S. Gurkaynak, B. Kisacikoglu and J.H. Wright.

From what I have learnt, the usual exercise of Expectation-maximization with kalman filter is an iterative repetition of

  1. E-step: based on the estimates of parameters from the last M-step, use kalman filter to update the estimates of the latent variable;

  2. M-step: use the estimates of latent variable from the E-step to get new estimates of parameters.

In the paper cited above, however, the authors seem to use EMKF algorithm to estimate the parameters with one single M-step. Specifically, the authors aim to uncover the latent variable $f_t$ in the equation $$y_t=\beta's_t+\gamma d_tf_t+\varepsilon_t,$$ where $y_t$ is an $n\times1$ vector, $s_t$ is a $K\times1$ vector, which are observed without error, $d_t$ is a dummy that is $1$ if there is an event at time $t$ or $0$ otherwise, $f_t$ is an iid $N(0,\sigma_f^2)$ latent variable, $\beta$ is a $K\times n$ matrix, $\gamma$ is an $n\times1$ vector, and $\varepsilon_t$ is iid with mean zero and diagonal variance-covariance matrix of $\Sigma_\varepsilon.$

The log-likelihood function is as follows: $$\begin{aligned}l(\theta)=-\frac{1}{2}\Sigma_{t=1}^{T}\mathbf{1}(d_{t}=1)\{(y_{t}-\beta^{\prime}s_{t})^{\prime}(\Sigma_{\varepsilon}+\gamma\gamma^{\prime})^{-1}(y_{t}-\beta^{\prime}s_{t})\\&+\log(|\Sigma_{\varepsilon}+\gamma\gamma^{\prime}|)\}\\&+\mathbf{1}(d_{t}=0)\{y_{t}^{\prime}\Sigma_{\varepsilon}^{-1}y_{t}+\log(|\Sigma_{\varepsilon}|)\}\end{aligned}.$$ To identify the parameters, the authors solve the following system of equations, which is obtained by maximize the log-likelihood function: $$\beta=E(\mathbf{1}(d_t=1)s_ts_t^{\prime})^{-1}E(\mathbf{1}(d_t=1)s_ty_t^{\prime}),$$ $$\Sigma_{\varepsilon}=E(y_{t}y_{t}^{\prime}\mathbf{1}(d_{t}=0)),$$ $$\gamma\gamma^{\prime}=E((y_{t}-\beta^{\prime}s_{t})(y_{t}-\beta^{\prime}s_{t})^{\prime}\mathbf{1}(d_{t}=1))-\Sigma_{\varepsilon}.$$

But $f_t$ does not appear in the equations above. Doesn't this mean to solve for $\beta$, $\gamma$ and $\Sigma_{\epsilon}$ one does not actually need the kalman filter? Can anyone explain this to me?

$\endgroup$
10
  • 1
    $\begingroup$ I'm not sure about what you're saying the connection is between the KF and EM. The only connection I've heard of ( that doesn't mean that yours is incorrect ) is Stoffer's famous paper where he used the EM algorithm to estimate the variances in the KF framework. It's possible that this is exactly what you're talking about above. Check out Stoffer's paper to see if that sheds any light. I'll look for it and hopefully send a link after this comment. $\endgroup$
    – mark leeds
    Commented Oct 6 at 2:02
  • 1
    $\begingroup$ It's Shumway and Stoffer, 1982 at this page, near the bottom. I hope it is what you were talking about. dsstoffer.github.io $\endgroup$
    – mark leeds
    Commented Oct 6 at 2:10
  • 1
    $\begingroup$ It says in the appendix on page 10: "as discussed in the text, we uncover $f_{t}$ using the kalman filter ... ". So, you should try to get your hands on the text if that's possible. Also, if what I linked to is what the paper is referring to, then I would think that Shumway and Stoffer, 1982 should be in the references of the paper. $\endgroup$
    – mark leeds
    Commented Oct 6 at 2:25
  • 1
    $\begingroup$ since $f_t$ only occurs when $d_t=1$ then you can get consistent estimates of the parameters as they are showing. You can then use those estimates to plug in as guesses for the Kalman Filter to then obtain $f_t$. Its a quite elegant solution to a special case. $\endgroup$
    – Andrew M
    Commented Oct 6 at 4:29
  • 1
    $\begingroup$ Hi zyy: Okay, I wasn't sure if what I pointed to was distinct from whats in your paper or the same thing. It sounds distinct in which case I don't know what they're doing with the EM part of the algorithm. Chances are, it's somehow viewing the latent variable as missing data and then "uncovering it" using the EM algorithm but that's a total guess. Hopefully, the Andrew M comments are helpful. I don't have time to try and understand the paper. $\endgroup$
    – mark leeds
    Commented Oct 6 at 9:16

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.