# How to verify Value Function in nonzero sum two player Differential Game?

There are two agents $i=1,2$. The state $k$ is governed by $\tau_i\in[0,1]$ where \begin{align} \dot{k} = f(k,\tau_1,\tau_2). \end{align}

Define the value function of player $i$ by \begin{align} v_i(k) := \sup_{\tau_i}\int^\infty_0{e^{-\rho t}F(k,\tau_i,\sigma_{-i})dt}. \end{align} where $\sigma_{-i}(k)$ is the policy function or a markovian stratgey profile of player $-i$. And $\rho>0$ is the time preference rate.

The Hamilton-Jacobi-Bellman equation reads \begin{align} \rho v_i(k) = \sup_{\tau_i}\{F(k,\tau_i,\sigma_{-i}) + v'(k)f(k,\tau_i,\sigma_{-i})\}. \end{align}

Now assume I approximated a function $\hat{v}_i$ by value function iteration. How do I know that it's the true value function?

• Check the distance with the last iteration. If you like it, you can stop. If it's too big, keep iterating. – VicAche Jun 5 '15 at 9:17
• first of all you can solve an analytical example and then test your algorithm against it. Second, the value function is a fixed point, so you can google for some tests of convergence to a fixed point, at least one heurisitic rules comes to mind, the distance between i and i+1 is decreasing , strict concavity ensures global optimality – user157623 Jun 8 '15 at 12:20

The first panel shows the final value function after iterating 80k (!). Should probably try an implicit method. The second panel shows the evolution from frist guess to final result $v^0\to v^j$, $j=80k$. Third panel is the distance between the value functions and fourth shows the error.
Note that the value function is convex-concave. This is due to the fact that for the limit case the law of motion tends to infinty $\lim_{k\to 0} \dot{k} = \infty$.
By the maximum principle optimality requires that the Hamiltonian \begin{align} H_1(k,\tau_1,\tau_2^*,\lambda_1) := F(k,\tau_1,\tau_2^*) + \lambda_1 f(k,\tau_1,\tau_2^*) \end{align} is concave in $(k,\tau_1,\tau_2)$ (sufficient condition). Now we have