# Dynamic programming in infinite horizon model

Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: $$V = \Gamma(V)$$.

1. How do I interpret the meaning of $$V$$? For example, when we decide a investment level in next time $$k_{t+1}$$ given the existing value $$k_t$$, we could calculate a value $$v(k_t)$$. Is this value equal the value given solving maximization problem after $$t$$ ($$t+1, t+2 \cdots$$)?

2. If the interpretation is correct, why a fixed point procedure yield such a maximized function?

Although I checked several lecture notes, I could not get the intuition by words about these points.

• My (almost forgotten) understanding of functional analysis and dynamic programming is tempted to say that your interpretation of (1) is correct. V is the maximised function of the state. Yet in infinite horizon (2), the max is not guaranteed to exist. But nevertheless, our model requires some type of convergent solution - and this is the fixed point, expressed in terms of something like Banach's fixed point theorem/contraction mapping theorem and Blackwell's sufficiency conditions etc. Sep 11, 2021 at 20:36

There are two interrelated maximisation problems. The first is the infinite horizon maximisation problem: \begin{align*} v(k) = &\max_{a_1, a_2, \ldots} \sum_{t = 0}^\infty \delta^t F(k_t, c_t),\\ \text{ subject to } & k_{t+1} = g(k_t, a_t),\\ & a_t = \Gamma(k_t),\\ & k_0 = k \end{align*} Here we call $$a_t$$ the decision variables, $$k_t$$ the state variables. This problem maximises an infinite sum of discounted values $$F(k_t, c_t)$$, subject to a law of motion that determines the next periods state depending on the action and state today. $$\Gamma(k)$$ gives a set of feasible actions $$a$$ that can be taken and finally $$k$$ is set equal to the initial state.

As such, $$v(k)$$ is the value of this optimisation problem when the initial state is $$k$$.

The second problem is the Bellman equation: $$v(k) = \max_{a \in \Gamma(k)}\left\{F(k,a) + \delta v(g(k,a))\right\}.$$ You should interpret this as an identity involving the function $$v$$, which appears both on the left hand right hand side of the equation, so the function $$v(.)$$ is the unknown in this equation (which has to hold for all $$k$$).

Under some conditions it can be shown that for every $$k$$, the function $$v(k)$$ of the first problem is equal to the value of the $$v(k)$$ that satisfies the second equation.

In order to find this function $$v(.)$$ one can define the Bellman operator $$T$$: $$(Tv)(k) = \max_{a \in \Gamma(k)}\left\{F(k,a) + \delta v(g(k,a))\right\}.$$ The operator $$T$$ takes a function $$v$$ (on the right hand side) and produces a new function $$Tv$$.

Again under suitable conditions, one can show that this operator is a contraction mapping. So by iterating this function over and over again we will converge to the fixed point of this operator, which then also gives the solution to the Bellman equation.

1. How do I interpret the meaning of $$V$$? For example, when we decide a investment level in next time $$k_{t+1}$$ given the existing value $$k_t$$, we could calculate a value $$v(k_t)$$. Is this value equal the value given solving maximization problem after $$t (t+1,t+2⋯)$$?

Yes by definition of the first problem, $$v(k_t)$$ gives the value of the infinite horizon maximisation problem when $$k_t$$ is the initial level of capital.

1. If the interpretation is correct, why a fixed point procedure yield such a maximized function?

This results from the equivalence between the first optimisation problem and the Bellman equation. The solution $$v$$ of the Bellman equation is a fixed point of the Bellman operator.