Our object is calculating AIC, and we are unsure whether we can use our measure below when calculating AIC. The following are the data from the experiments and our method to calculate the information criterion. Would the method be valid?

  • Experimental data (Selten and Chmura 2008, AER):
    A group consists of four participants in the role of Player 1 and four in the role of Player 2. The players played a two-person game against randomly chosen opponents within their independent group in each period. Player 1 could choose between $u$ and $d$, and Player 2 could choose between $l$ and $r$. The game is repeated for 200 periods.
    No Feedback until the end of the experiment: The decisions in a play were made without any information about the choices of the other players, e.g., the other players’ choice and payoff, period number, and cumulative payoff.
    An independent observation from the experiment consists of $f_{u_j }$ and $f_{l_j }$. $f_{u_j }$ is the ratio of Player 1s in group $j$ choosing $u$ across the whole period. Similarly, $f_{l_j}$ is the ratio of Player 2s in group $j$ choosing $l$ across the whole period. Since there are 108 independent groups, we have 108 independent observations.

  • Theory model selection:
    For each theory model, let $p_u$ and $p_l$ be the predicted values for $f_{u_j }$ and $f_{l_j }$. We calculate the mean squared distances (MSDs) as follows:
    $MSD= \frac{1}{N} ∑_{j=1}^{108}[(f_{u_j} - p_u )^2 + (f_{l_j} - p_l)^2 ]$.
    First, we get the best-fitting values of the parameters for each model by minimizing MSD. Then, to figure out which model explains the behaviors in our games best, we compare the minimized MSD values among the models.

  • Issue:
    Since theory models have different numbers of parameters, MSD may have a limitation. Thus, we need an alternative measure considering the number of parameters, such as AIC and BIC. Can we regard MSD as the sum squared residuals? That is, can we calculate AIC as $AIC=n⋅log⁡(MSD)+2k$, where $k$ is the number of parameters? Here, we have a concern when we calculate the information criteria using MSD. One may worry that $f_{u_j }$ and $f_{l_j }$ in one observation can be correlated. If they are correlated, MSD will be biased. However, since the experiment did not provide any feedback about the others’ decisions, we consider $f_{u_j }$ and $f_{l_j }$ are independent. Then, would it be okay to calculate AIC as above?

  • $\begingroup$ Since you have got no answers here, you may consider moving the question to Cross Validated Stack Exchange. AIC is a relatively more frequent object of questions there than here. $\endgroup$ Commented Nov 1, 2023 at 8:46


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.