# Differentiating Bellman equation

Assume that we have a Bellman equation that is

$$V(k)=\max_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$$ The textbook says that if we differentiate with respect to $$k'$$ and set equal to $$0,$$ we get: $$u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)$$

My questions are: i) What are we differentiating here? $$V(k)$$? Also, how can we differentiate under the max operator?

## 1 Answer

In a sense, the Bellman equation is a definition of $$V(k)$$, which might be more obvious if it's written $$V(k):=\max_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right).$$ The operator $$\max_{0\leq k'\leq f\left(k\right)}$$ tells us that, to get $$V(k)$$, we should find the specific $$k'$$ - let's call it $$k'^*$$ - at which the expression $$u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$$ is greatest (if such a maximum exists).

To do that, and provided the functions involved have the right properties, the first step is to take the derivative of $$u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$$ with respect to $$k'$$ and setting the result equal to zero. So, $$\frac{d}{dk'}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right) = 0,$$ from which follows (after some math) that $$u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)$$ at the maximum (given the right properties), so not everywhere, but where $$k'=k'^*$$. Without further knowledge of $$u$$ and $$V$$, this gives at least an implicit rule for how to obtain $$k'^*$$, namely via $$u'\left(f\left(k\right)-k'^*\right)=\beta V'\left(k'^*\right)$$ Because with $$k'^*$$ we have now found the specific $$k'$$ at which $$u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)$$ is greatest, we can substitute that specific value, $$k'^*$$, into the initial definition to get $$V(k):=u\left(f\left(k\right)-k'^*\right)+\beta V\left(k'^*\right).$$ where the $$\max$$ operator has disappeared, as $$k'^*$$ evaluates the function already at the max. And we see that $$V(k)$$ does not depend on $$k'$$.

• Would it be correct to write V(k) as actually V(k,k') and then differentiate both sides? I will accept this answer, if you could please write down explicitly what we are differentiating? In other words, when we take the derivative, does the max operator disappear? Aug 25, 2022 at 23:23
• @KwameBrown $V$ only depends on $k$, not on $k'$. For a given $k$, we only care about the specific value of $k'$ where the inner function is maximized. The rest is calculus, to find a (local) maxima of a (differentiable) function, compute its derivative and set it to zero (and check a few more things). Aug 26, 2022 at 7:52
• Answer edited for clarification.
– BrsG
Aug 26, 2022 at 8:22
• +1, nice answer Aug 26, 2022 at 9:03
• Thank you so much, this is very helpful indeed. Aug 26, 2022 at 12:58