Can you provide me with an intuition about what it means to derive a probit or logit model from an underlying latent variable model?
1 Answer
$\begingroup$
$\endgroup$
Logit and probit models can be derived from an underlying latent variable model. To see this, let $y^{\ast}$ be an unobserved (i.e. latent) variable, and suppose that \begin{equation} y^{\ast} = \beta_0 + x\beta + \epsilon, ~y=1[y^{\ast}>\gamma] \label{latent} \end{equation} where $1[\cdot]$ is an indicator function taking value $1$ if $y^{\ast}>\gamma$ and zero otherwise. Here, $\gamma$ is a chosen threshold. If the latent unobserved utility $y^\ast$ is above the threshold, then $y$ takes the value one; otherwise, it takes the value zero.
The idea is to see the binary outcome $y=\{0,1 \}$ as a dichotomization of a latent continuous variable $y^{\ast}$ modeled with a canonical multiple linear regression model. For example, a bank can issue a loan ($y=1$) or choose not to ($y=0$). Think of it as having an underlying continuous latent outcome, $y^{\ast}$ which can be regarded as a measure of the choice utility.
In the model above, we assume that $\epsilon$ is independent of $x$ and that $\epsilon$ either has the standard logistic distribution or the standard normal distribution.
Recall, the standard logistic and standard normal distribution are symmetric about zero. Thus, $1-G(-z)=G(z)$.
From the model above, we can derive the response probability for $y$. First, we assumed that $y_i=1$ if and only if $y^{\ast}_i > \gamma$, with $\gamma$ set arbitrarily to zero. Thus, \begin{align*} \mathbb{P}(y=1|x) =& \mathbb{P}(y^{\ast} >0|x) = \mathbb{P}(\beta_0 + x\beta + \epsilon >0|x) = \\ =&\mathbb{P}[ \epsilon > -(\beta_0 + x\beta ) |x] =1 - \mathbb{P}[ \epsilon \le -(\beta_0 + x\beta ) |x] \\ =& 1 - G[ - (\beta_0 + x\beta)] = G(\beta_0 + x \beta) \end{align*} where $\mathbb{P}[ \epsilon \le -(\beta_0 + x\beta ) |x]$ is the cdf of the error term of the latent outcome.
