1
$\begingroup$

I know what the probability mass function of a multinomial logit model is. However, I do not know what the probability mass function of an individual observation i, f(yi|xi), is for a multinomial logit model. I would really appreciate if someone could explain how to get the answer and knows the correct answer.

$\endgroup$
  • $\begingroup$ I'm voting to close this question as off-topic because this belongs to Cross-Validated. $\endgroup$ – luchonacho Oct 16 '17 at 11:54
  • 2
    $\begingroup$ I disagree, I considered what category my question should be asked to. I am studying econometrics and the topic of multinomial logit models is being discussed in all of my economics textbooks and by my professor. It is correct that this question could also be asked in the cross validated forum, but I would ask you not to delete my question in case anyone in econometrics or quantitative economics can help me. Thanks $\endgroup$ – Niamh45 Oct 16 '17 at 12:05
  • $\begingroup$ I agree with @Amy that this question is on-topic. As explained in the help section, econometrics/statistics is on-topic here, even though there is overlap with cross-validated.se: economics.stackexchange.com/help/on-topic $\endgroup$ – jmbejara Oct 16 '17 at 13:24
  • $\begingroup$ @jmbejara we are just fostering inefficiency. As it stands, the question is not specific about economics, and its likely to benefit more people in CV. With such policy, every single econometric CV question that has not been asked here is on-topic. Clearly not the best policy to pursue. $\endgroup$ – luchonacho Oct 16 '17 at 14:40
  • $\begingroup$ @luchonacho Allowing the overlap is by design and is not unique to economics.SE. As evidenced by this post (meta.stackexchange.com/a/4713/299907), this is something that occurs throughout stack exchange, broadly. I like the current policy. $\endgroup$ – jmbejara Oct 16 '17 at 15:09
1
$\begingroup$

A useful way to view the model is as follows. For each observation $i$, there are $K$ different possible observations (choices). So, there is a random variable $\varepsilon_{i,k}$ for each possible choice. We observe which choice is made and the values of the explanatory variables $\boldsymbol X_i$. Then, for a given distribution of the random variables $\varepsilon_{i,k}$ (given parameter values of the Type-1 EV), we know the conditional probability of picking choice $k$ conditional on $\boldsymbol X_i$.

This is explained in the following part of the relevant Wikipedia article:

It is also possible to formulate multinomial logistic regression as a latent variable model, following the two-way latent variable model described for binary logistic regression. This formulation is common in the theory of discrete choice models, and makes it easier to compare multinomial logistic regression to the related multinomial probit model, as well as to extend it to more complex models.

Imagine that, for each data point $i$ and possible outcome $k$, there is a continuous latent variable $Y_{i,k}^*$ (i.e. an unobserved random variable) that is distributed as follows:

$$\begin{align} Y_{i,1}^{\ast} &= \boldsymbol\beta_1 \cdot \mathbf{X}_i + \varepsilon_1 \, \\ Y_{i,2}^{\ast} &= \boldsymbol\beta_2 \cdot \mathbf{X}_i + \varepsilon_2 \, \\ \cdots & \\ Y_{i,K}^{\ast} &= \boldsymbol\beta_K \cdot \mathbf{X}_i + \varepsilon_K \, \\ \end{align} $$

where $\varepsilon_k \sim \operatorname{EV}_1(0,1),$ i.e. a standard type-1 extreme value distribution.

This latent variable can be thought of as the utility associated with data point $i$ choosing outcome $k$, where there is some randomness in the actual amount of utility obtained, which accounts for other unmodeled factors that go into the choice. The value of the actual variable $Y_i$ is then determined in a non-random fashion from these latent variables (i.e. the randomness has been moved from the observed outcomes into the latent variables), where outcome $k$ is chosen if and only if the associated utility (the value of $Y_{i,k}^{\ast}$) is greater than the utilities of all the other choices, i.e. if the utility associated with outcome $k$ is the maximum of all the utilities. Since the latent variables are continuous, the probability of two having exactly the same value is 0, so we ignore the scenario. That is:

\begin{align} \Pr(Y_i = 1) &= \Pr(Y_{i,1}^{\ast} > Y_{i,2}^{\ast} \text{ and } Y_{i,1}^{\ast} > Y_{i,3}^{\ast}\text{ and } \cdots \text{ and } Y_{i,1}^{\ast} > Y_{i,K}^{\ast}) \\ \Pr(Y_i = 2) &= \Pr(Y_{i,2}^{\ast} > Y_{i,1}^{\ast} \text{ and } Y_{i,2}^{\ast} > Y_{i,3}^{\ast}\text{ and } \cdots \text{ and } Y_{i,2}^{\ast} > Y_{i,K}^{\ast}) \\ \cdots & \\ \Pr(Y_i = K) &= \Pr(Y_{i,K}^{\ast} > Y_{i,1}^{\ast} \text{ and } Y_{i,K}^{\ast} > Y_{i,2}^{\ast}\text{ and } \cdots \text{ and } Y_{i,K}^{\ast} > Y_{i,K-1}^{\ast}) \\ \end{align}

Or equivalently:

\begin{align} \Pr(Y_i = 1) &= \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,1}^{\ast}) \\ \Pr(Y_i = 2) &= \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,2}^{\ast}) \\ \cdots & \\ \Pr(Y_i = K) &= \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,K}^{\ast}) \\ \end{align}

Edit: Sometimes you'll see notation like this: $$ Pr( i \rightarrow k) = \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,k}^{*}), $$ where $i \rightarrow k$ means that individual $i$ chooses choice $k$, is fairly concise once the appropriate notation is defined. If you're looking for a useful way to put this into a likelihood function, you might see something like this: $$ P_i = \prod_{k=1}^K Pr( i \rightarrow k)^{\mathbb 1_{i \rightarrow k}}, $$ where $\mathbb 1_{i \rightarrow k}$ is an indicator function that is equal to 1 when $i$ chooses choice $k$ and zero otherwise and $P_i$ is the PMF of the observation associated with individual $i$.

Edit 2:

For an explicit formula, see the same Wikipedia article. Check out, for instance, the section with the following:

As a result, it is conventional to set $C = -\boldsymbol\beta_K$ (or alternatively, one of the other coefficient vectors). Essentially, we set the constant so that one of the vectors becomes 0, and all of the other vectors get transformed into the difference between those vectors and the vector we chose. This is equivalent to "pivoting" around one of the $K$ choices, and examining how much better or worse all of the other $K-1$ choices are, relative to the choice are pivoting around. Mathematically, we transform the coefficients as follows:

\begin{align} \boldsymbol\beta'_1 &= \boldsymbol\beta_1 - \boldsymbol\beta_K \\ \cdots & \cdots \\ \boldsymbol\beta'_{K-1} &= \boldsymbol\beta_{K-1} - \boldsymbol\beta_K \\ \boldsymbol\beta'_K &= 0 \end{align}

This leads to the following equations:

\begin{align} \Pr(Y_i=1) &= \frac{e^{\boldsymbol\beta'_1 \cdot \mathbf{X}_i}}{1 + \sum_{k=1}^{K-1} e^{\boldsymbol\beta'_k \cdot \mathbf{X}_i}} \, \\ \cdots & \cdots \\ \Pr(Y_i=K-1) &= \frac{e^{\boldsymbol\beta'_{K-1} \cdot \mathbf{X}_i}}{1 + \sum_{k=1}^{K-1} e^{\boldsymbol\beta'_k \cdot \mathbf{X}_i}} \, \\ \Pr(Y_i=K) &= \frac{1}{1 + \sum_{k=1}^{K-1} e^{\boldsymbol\beta'_k \cdot \mathbf{X}_i}} \, \\ \end{align}
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.