Method of Lagrange multipliers with random variables

Question

I'll illustrate the issue I'm having with a simple problem.

Let $c_1, c_2 \in \mathbb{R}$, and $Z$ a real-valued random variable. Let $u:\mathbb{R} \rightarrow \mathbb{R} $ be a differentiable function, and $f(c_1,Z)$ be a real-valued function that is differentiable with respect to $c_1$.

The problem is: Maximise: $u(c_1) + \mathbb{E}[u(c_2)]$ such that $c_2=f(c_1, Z)$

This problem is easily solved by direct substitution, and the answer is

$u'(c_1) + \mathbb{E}\left[u'(f(c_1,Z)) \frac{\partial f(c_1,Z)}{\partial c_1}\right] = 0 \label{answer}$

The issue is how to write down a Lagrangian whose extrema correspond to the solution of this problem in the normal way.

My first instinct was to write $\mathcal{L} = u(c_1) + \mathbb{E}[u(c_2)] + \lambda(c_2 - f(c_1,Z))$.

However, this doesn't seem to make sense. If you treat $c_2$ as if it is a random variable, then the derivative of $\mathbb{E}[u(c_2)]$ with respect to $c_2$ gives zero, and this can't possibly give the right answer. On the other hand, it doesn't make sense to treat it as non-stochastic either, since it is 'forced' to be stochastic by the constraint.

Question: How do I write a Lagrangian whose extrema correpond to the solution of the above optimisation problem?

Answer 1

If you treat $c_2$ as if it is a random variable, then the derivative of $E[u(c_2)]$ with respect to $c_2$ gives zero.

Why? Such an assertion doesn't follow from anywhere. The subtleties lie elsewhere. The problem with the Lagrangian considered by the OP

$$\mathcal{L} = u(c_1) + \mathbb{E}[u(c_2)] + \lambda(c_2 - f(c_1,Z))$$

is that it also is a random variable since now $Z$ appears outside the expected value (The direct substitution approach followed by maximization with respect to $c_1$ only, does not create any such issues).

Now, do we / can we maximize a random variable? Well, no, because the essential characteristic of a random variable is that it is a function whose value cannot be set by command and control.

But one could say "ok, let's pretend that this Lagrangian is not a random variable, and just write down the conditions for maximization, even though we know that we can't force the solution".

But this won't work: if one attempts to do it one will eventually obtain

$$u'(c_1) + \mathbb{E}\left[u'(f(c_1,Z))\right] \cdot \frac{\partial f(c_1,Z)}{\partial c_1} = 0 $$

which is not the same as the condition obtained through direct substitution, because here the partial derivative is outside the expected value.

(for those who may think "hey, then how do we apply maximization procedures in the maximum likelihood approach" the answer is that there, nothing is random anymore when we get to apply the maximization steps).

Answer 2

This is not an answer, for that see Alecos's answer. The point of this post is to clarify that the question relies on the false assumption $$ \frac{d \ E(c_2)}{d \ c_2} = 0 $$ by way of an example. Consider the random variable that I get by rolling a six sided die and multiplying the result $X$ by a positive integer $n$. The expected value of this is $$ E(n \cdot X) = n \cdot \frac{7}{2}. $$ Would you claim that $$ \frac{d \ E(n \cdot X)}{d \ n} = 0 $$ because "Taking the derivative of the expectation [...] with respect to anything gives zero, since the expectation [...] is just a constant"?

Answer 3

Answer first:

The `objective function' is in fact a functional. The task is to find a pdf $g(y)$ folowed by $c_2$ and a real number $c_1$ that maximise the following Lagrangian

$$ \mathcal{L}(c_1, g) = u(c_1) + \int u(x)g(x) \mathrm{d}x + \int \lambda(y) \left[ g(y) - \eta(c_1, y) \right] \mathrm{d}y. $$

Here, $\eta(c_1, y)$ is the pdf followed by the random variable $f(c_1, Z)$. $\lambda(y)$ is a Lagrange multiplier, used to enforce the constraint $g(y)=\eta(c_1,y)$. All integrals are over $\mathbb{R}$. The first order conditions are

$$ u'(c_1) - \frac{\partial}{\partial c_1} \int \lambda(y) \eta(c_1,y) \mathrm{d} y =0, \\ u(y) + \lambda(y) =0, \\ g(y) - \eta(c_1, y) = 0. \\ $$

Combining the first two gives

$$ u'(c_1) + \frac{\partial}{\partial c_1} \int u(y) \eta(c_1,y) \mathrm{d} y =0. $$

Now using the 'law of the unconscious statistician' gives

$$ u'(c_1) + \frac{\partial}{\partial c_1} \int u(f(c_1,z)) \phi(z) \mathrm{d} z =0 \\ \Rightarrow u'(c_1) + \int u'(f(c_1,z)) \frac{\partial f(c_1,z)}{\partial c_1} \phi(z) \mathrm{d} z =0 .$$ Here, $\phi(z)$ is the pdf followed by the random variable $Z$. This is equivalent to the solution given by OP.

---

Now I want to address a few other things. I would ideally do this in the comments, but I don't have enough reputation to comment.

First of all, I am OP. I really am at a loss as to why I couldn't log in with original credentials. Still trying to figure that one out.

As denesp points out, I wrote an answer not so long ago which I immediately deleted. I did this for a few reasons:

1. I didn't actually answer the question in that post.
2. I was on the bus at the time, typing on my phone. The formatting I had wasn't correct, and the wording was very loose. I wanted to take more time over it and give a better and more precise answer. So, for example, some of denesp's comments are directed at imprecise wording in a post I deleted.

To address denesp's comments on the original post: I commented briefly that things don't work because

1. I thought it was very obvious why the suggested things don't work.
2. I didn't think a detailed post explaining exactly why they don't work with lots of maths was appropriate for the comments

Alecos' 'answer' isn't an answer at all. It may better be read as an expansion upon why the naive approach in the original post doesn't work.

Now let me address the matter of differentiation. It's true that I wasn't precise in the original post, but this was to illustrate the naive approach I was taking and why it wasn't working.

The confusion here stems from the following. Say I have a differentiable function $f: \mathbb{R} \rightarrow \mathbb{R}$. I can certainly talk meaningfully about the function that is the derivative of $f$, that I will denote by $f'$.

On the other hand, sometimes people 'allow the argument to be a random, $\mathbb{R}$-valued variable'. When they do this, they should probably call the resulting random variables something other than $f$ and $f'$. All the confusion has arisen from this. I did not make this distinction in the original post, because my thinking on the question was not clear, and this has been confounded by other comments not recognising this base cause of the confusion.