I have the following dynamic programming problem:

$$\max_{\{x_t,y_{t+1}\}_{t=0}^\infty}\sum_{t=0}^\infty \beta^tu(F(x_t)-y_{t+1})\;\;\;\;\;\text{s.t}\;\;\;\;y_{t+1}\in\Gamma(x_t)$$

where $\Gamma(x)=\{y\in\mathbb{R}_{+}\mid 0\le y\le F(x)\}$, and the functions $u$ and $F$ have the following properties:

  1. $u:\mathbb{R}_{+}\to\mathbb{R}$ is continuous, strictly increasing, and strictly concave;
  2. $F:\mathbb{R}_{+}\to\mathbb{R}$ is continuous, strictly increasing, and concave;
  3. $F(0)=0$;
  4. And there exists $\bar{x}>0$ such that:
  • If $x\in[0,\bar{x}]$ then $F(x)\in[x,\bar{x}]$.

  • If $x>\bar{x}$ then $F(x)<x$.

I've got to show that the function $G(x,y)\equiv u(F(x)-y)$ is bounded over the set $A\equiv\{(x,y)\in \mathbb{R}_{+}\times\mathbb{R}_{+}\mid y\in\Gamma(x)\}$.

What I have tried: Let $(x,y)\in A$, then $y\in\Gamma(x)$. First, by definition the functions $u$ and $F$ clearly $G$ is bounded below. Now in order to show that $G$ is bounded above, I noticed that the set of images $G(x,\Gamma(x))$ is compact in the second argument because $G$ is continuous and $\Gamma(x)$ is compact, i.e. given $x$ we have that the function $G(x,\cdot)$ is bounded in $\Gamma(x)$.

Now, trying to show that $G$ is bounded above in $x$ (i.e. in the first argument) has proven to be more involved, I'm trying to use the fourth property shown above about the existence of $\bar{x}$, where we have two cases, if $x\in[0, \bar{x}]$ then clearly $G(x,y)\le u(F(\bar{x})-y)\le u(F(\bar{x}))$. But when considering $x>\bar{x}$ we have $F(x)<x$, thus $G(x,y)<u(x-y)$ which does not show that $G$ is bounded since the "bound" still depends on $x$ which is in a unbounded space $(\bar{x},\infty)$.

I think there must be a trick with $F(x)<x$ but I have not been able to see it, any help would be appreciated.



1 Answer 1


The function $u(F(x)-y)$ is not necessarily bounded on $A$. For example, if $u(x) = F(x) = \sqrt{x}$ then: $$ u(F(x)-y) = \sqrt{\sqrt{x}-y}, $$ Taking $y = 0$, this gives $u(F(x)) = \sqrt{\sqrt{x}}$, which is clearly unbounded in $x$.

The problem you are looking at is the following: $$ \max_{(x_t, y_t)_{t = 0}^\infty} \sum_{t = 0}^\infty \beta^t u(F(x_t) - y_{t+1}) \text{ s.t. } y_{t+1} \in [0, F(x_t)]. $$ Notice that this problem is not well defined. In particular, given that $u$ is strictly increasing, the best choice for $y_{t+1}$ is to set it equal to zero at every instance $t$. This gives the payoff function $$ \sum_{t = 0}^\infty \beta^t u(F(x_t)), $$ which is clearly increasing in $x_t$ (as both $u$ and $F$ are strictly increasing). As such, the best thing to do is to set $x_t$ as large as possible (so equal to $+ \infty$). This means that your maximisation problem has no solution.

Now, assume that you have a better behaving model, for example, $y_t = x_t$. This then gives the problem: $$ \max_{(x_t)_{t = 0}^\infty} \sum_{t = 0}^\infty \beta^t u(F(x_t) - x_{t+1}) \text{ s.t. } x_{t+1} \in [0, F(x_t)]. $$ The function $u(F(x)-y)$ is still unbounded, but now you can show that the objective function $\sum_{t=0}^\infty \beta^t u(F(x_t) - x_{t+1})$ is bounded for every feasible path. Let $\partial u(0)$ be a supdifferential (derivative) for $u$ at zero. Then, using concavity of $u$, we have: $$ \begin{align*} \sum_{t = 0}^n \beta^t u(F(x_t) - x_{t+1}) &\le \sum_{t = 0}^n \beta^t \left(u(0) + \partial u(0)[F(x_t) - x_{t+1}]\right),\\ &= u(0) \sum_{t = 0}^n \beta^t + \partial u(0) \underbrace{\sum_{t = 0}^n \beta^t [F(x_t) - x_{t+1}]}_{=A}. \end{align*} $$ Now, $F(x) \le \overline{x} + x$ (if $x \le \overline{x}$ then $F(x) \le \overline{x}$ and if $x \ge \overline{x}$, then $F(x) \le x$). So, $$ \begin{align*} A &\le \sum_{t = 0}^n \beta^t [\overline{x} + x_t - x_{t+1}],\\ &= \overline{x} \sum_{t = 0}^n \beta^t + \underbrace{\sum_{t = 0}^n \beta^t (x_t - x_{t+1})}_{=B} \end{align*} $$ Then, $$ \begin{align*} B &= \sum_{t = 0}^n \beta^t (x_t - x_{t+1}),\\ &=x_0 \underbrace{- x_1 + \beta x_1}_{\le 0} \underbrace{- \beta x_2 + \beta^2 x_2}_{\le 0} \underbrace{- \beta^2 x_3 + \,\,}_{\le 0} \ldots \underbrace{\,\, + \beta^n x_n}_{\le 0} - \beta^n x_{n+1},\\ &\le x_0 - \beta^n x_{n+1} \le x_0 \end{align*} $$ Putting it all together, we obtain the bound: $$ \begin{align*} \sum_{t = 0}^n \beta^t u(F(x_t) - x_{t+1}) & \le (u(0) + \partial u(0) \overline{x}) \sum_{t = 0}^n \beta^t + \partial u(0) x_0 \end{align*} $$ Taking the limit $t \to \infty$, the right hand side converges to $\frac{u(0) + \partial u(0) \overline{x}}{1 - \beta} + \partial u(0) x_0$ (a fixed number).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.