# Solving a HJB with a probability to transit to a new state

I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time.

The firm maximizes $$V(k)=\int_{t=0}^{\infty}e^{-rt} \pi_t dt$$ with $$\pi_t=f(k_t)-i_t$$ and $$\dot{k}=i_t-\delta k_t$$. There is a probability $$\rho$$ per unit of time to transit to a new state where a tax is imposed, and where the profit becomes $$f(k_t)-i_t-\tau k_t$$. This is a partial equilibrium problem and we assume that $$r$$, $$\delta$$ and $$\rho$$ are exogenous.

I am trying to solve this problem using Hamiltonian-Jacobi-Bellman (HJB) functions: \begin{align*} rV_1 &= \max_{i} \{f(k_t)-i_t+\rho (V_2-V_1)+\dot{V}_1\} \\ rV_2 &= \max_{i} \{f(k_t)-i-\tau k_{t}+\dot{V}_2\} \end{align*}

I know how to solve the second equation, following the method of Walde 2012 using dynamic programming. First, rewrite $$f(k_t)-i_t-\tau k_t + V'_2(k_t)\dot{k_t}=f(k_t)-i_t-\tau k_t+ V'_2(k_t)(i_t-\delta k_t)$$ Then, take the FOC with respect to $$i_t$$, which yields $$V'_2(k_t)=1$$ Then, use the envelope condition to find \begin{align*} rV'_2=f'(k_t)-\tau+V''_2(k_t)(i_t-\delta k_t)-\delta V'_2(k_t) \end{align*} This can be simplified, using the FOC, as\begin{align*} f'(k_t)=r+\delta+\tau \end{align*} which yields the same solution as a simple Hamiltonian would.

However, I am unsure how to proceed next and how to solve for the solution before the tax is imposed and the uncertainty resolved.

Additionally, if you have some references regarding dynamic control in continuous time, I would be very interested, especially if they treat the case of HJB with additional constraints.

EDIT: for clarification, once we reach the new state where the tax is imposed, there is no possibility to go back to the previous state. The only uncertainty is about when the tax will be imposed, i.e, when the uncertainty will be resolved.

• Hamiltonain is a physics concept and you are not even applying it correctly. Are you forced to do this? You could simply take the current utility function and integrate over time, with a odds parameter. You also need time discounting. How was this ignored? – D J Sims Jul 8 at 8:03
• Hi. My previous comment was deleted when your post was changed into a comment, but: 1. This is not the same Hamiltonian as in physics, this is the Pontryagin's maximum principle, which is not what I try to use here. I try to use the Hamiltonian-Jacobi-Bellman equation instead to solve this problem as it makes stochastic formulations easier. 2. There is indeed time discounting: the firm exponentially discount future profit at the exogenous interest rate $r$, which, in the HJB, is apparent through the $rV$. You can check Walde 2012 for the derivation of the HJB. – Mr. Fafa Jul 9 at 5:29

I would leave this as a comment but I cant. You are on the right track.

1. Once you know $$V_2(k)$$ then you can plug that into to the first hjb and solve.

2. To solve for $$V_2$$ you need to find the optimal $$i$$ as a function of $$k$$. Then plug $$i(k)$$ into the 2nd HJB. That will give you a second order ode. Solving that will give you $$V_2(k)$$ and you go to 1.

• Thank you a lot for your comment. Unfortunately, I am not sure how to get $i$ as a function of $k$ (the policy function right?). Also, when I "plug" that $V_2$ in $V_1$, should I distinguish the $k$ from $V_2$ from those from $V_1$? – Mr. Fafa Jul 16 at 10:59

Following the answer of user28714, I tried the following. First, substituting for the FOC, I rewrite $$V_2$$ as \begin{align*} rV_2 &= f(k_t)-i_t - \tau k_t + i_t-\delta k_t \\ &= f(k_t - \tau k_t - \delta k_t \end{align*} Thus, we get $$V_2 = \frac{1}{r}\left(f(k_t) - k_t(\tau + \delta) \right)$$ Substituting in $$V_1$$, we get $$rV_1 = \max_{i} \left\{ f(k_t)-i_t + \rho\left(\frac{1}{r}\left(f(k_t) - k_t(\tau + \delta) \right)-V_1\right) + V'_1(i_t-\delta k_t) \right\}$$

The FOC is unchanged: $$V'_1=1$$, and the envelope condition becomes \begin{align*} rV'_1 = f'(k_t)+\rho\left(\frac{1}{r}(f'(k_t)-\tau - \delta)-V'_1\right)+V''_1(i_t-\delta k_t) - \delta V'_1 \end{align*} Noting that $$\dot{V'_1} = V''_1 (i_t-\delta k_t)$$ and substituting using the envelope condition, we find \begin{align*} \dot{V_1}=V'_1(r+\delta+\rho)-f'(k_t)-\frac{\rho}{r}(f'(k_t)-\tau - \delta) \end{align*} Using $$V'_1=1$$ and $$\dot{V'_1}=0$$, we get \begin{align*} f'(k_t)(1+\frac{\rho}{r})&= r+\delta +\rho +\frac{\rho}{r}(\tau + \delta) \\ f'(k_t) &= \frac{r}{r+\rho}\left( r+\delta +\rho +\frac{\rho}{r}(\tau + \delta) \right) \\ f'(k_t) &= r + \delta + \frac{\rho }{r+\rho}(r+ \frac{\rho}{r}\tau) \end{align*}

Which is not the most elegant result... Could somebody confirm me this result?