I'll illustrate the issue I'm having with a simple problem.

Let $c_1, c_2 \in \mathbb{R}$, and $Z$ a real-valued random variable. Let $u:\mathbb{R} \rightarrow \mathbb{R} $ be a differentiable function, and $f(c_1,Z)$ be a real-valued function that is differentiable with respect to $c_1$.

The problem is: Maximise: $u(c_1) + \mathbb{E}[u(c_2)]$ such that $c_2=f(c_1, Z)$

This problem is easily solved by direct substitution, and the answer is

$u'(c_1) + \mathbb{E}\left[u'(f(c_1,Z)) \frac{\partial f(c_1,Z)}{\partial c_1}\right] = 0 \label{answer}$

The issue is how to write down a Lagrangian whose extrema correspond to the solution of this problem in the normal way.

My first instinct was to write $\mathcal{L} = u(c_1) + \mathbb{E}[u(c_2)] + \lambda(c_2 - f(c_1,Z))$.

However, this doesn't seem to make sense. If you treat $c_2$ as if it is a random variable, then the derivative of $\mathbb{E}[u(c_2)]$ with respect to $c_2$ gives zero, and this can't possibly give the right answer. On the other hand, it doesn't make sense to treat it as non-stochastic either, since it is 'forced' to be stochastic by the constraint.

Question: How do I write a Lagrangian whose extrema correpond to the solution of the above optimisation problem?

  • $\begingroup$ You could take the expectation of everything. $\endgroup$ – VCG Sep 15 '16 at 14:00
  • $\begingroup$ I'm afraid that doesn't work either. $\endgroup$ – bobhawke Sep 15 '16 at 14:02
  • $\begingroup$ The expectation can be written as an integral perhaps, and then Leibniz's integral rule can be applied. $\endgroup$ – Kitsune Cavalry Sep 15 '16 at 17:12
  • $\begingroup$ That also doesn't work. $\endgroup$ – bobhawke Sep 15 '16 at 17:28
  • $\begingroup$ For future repliers: In a deleted answer the OP says he forgot his login. Hence no answer's will be accepted. Another unfortunate instance is that the OP keeps repeating "doesn't work" instead of explaining why he thinks so. This is unfortunate because I for one think Alecos's answer is the correct one and that the OP treats derivates w.r.t. random variables in a rather peculiar way. $\endgroup$ – Giskard Sep 25 '16 at 11:01

If you treat $c_2$ as if it is a random variable, then the derivative of $E[u(c_2)]$ with respect to $c_2$ gives zero.

Why? Such an assertion doesn't follow from anywhere. The subtleties lie elsewhere. The problem with the Lagrangian considered by the OP

$$\mathcal{L} = u(c_1) + \mathbb{E}[u(c_2)] + \lambda(c_2 - f(c_1,Z))$$

is that it also is a random variable since now $Z$ appears outside the expected value (The direct substitution approach followed by maximization with respect to $c_1$ only, does not create any such issues).

Now, do we / can we maximize a random variable? Well, no, because the essential characteristic of a random variable is that it is a function whose value cannot be set by command and control.

But one could say "ok, let's pretend that this Lagrangian is not a random variable, and just write down the conditions for maximization, even though we know that we can't force the solution".

But this won't work: if one attempts to do it one will eventually obtain

$$u'(c_1) + \mathbb{E}\left[u'(f(c_1,Z))\right] \cdot \frac{\partial f(c_1,Z)}{\partial c_1} = 0 $$

which is not the same as the condition obtained through direct substitution, because here the partial derivative is outside the expected value.

(for those who may think "hey, then how do we apply maximization procedures in the maximum likelihood approach" the answer is that there, nothing is random anymore when we get to apply the maximization steps).


This is not an answer, for that see Alecos's answer. The point of this post is to clarify that the question relies on the false assumption $$ \frac{d \ E(c_2)}{d \ c_2} = 0 $$ by way of an example. Consider the random variable that I get by rolling a six sided die and multiplying the result $X$ by a positive integer $n$. The expected value of this is $$ E(n \cdot X) = n \cdot \frac{7}{2}. $$ Would you claim that $$ \frac{d \ E(n \cdot X)}{d \ n} = 0 $$ because "Taking the derivative of the expectation [...] with respect to anything gives zero, since the expectation [...] is just a constant"?


To solve this with the exact constraint given in the question, one could set up the following Lagrangian:

$\mathcal{L}(c_{1},\lambda(\cdot))=u\left(c_{1}\right)+\int u\left(c_{2}\left(z\right)\right)g(z)\mathrm{d}z-\int\lambda(z)\left\{ f\left(c_{1},z\right)-c_{2}\left(z\right)\right\} \mathrm{d}z,$

where $g(z)$ is the probability density function for the random variable $Z$, and $\lambda(z)$ is the Lagrange multiplier corresponding to any value $z$.

First-order conditions for the functions $c_2$ and $\lambda$ and for the scalar $c_1$ then are:

  • $\forall z:\lambda(z)=-u^\prime (c_2(z))g(z)$
  • $\forall z: f(c_1,z)=c_2(z)$
  • $u^\prime(c_1)-\int\lambda(z)\frac{\partial{f}}{\partial{c_1}}g(z)\mathrm{d}z=0\Leftrightarrow u^\prime(c_1)+\int u^\prime(c_2(z))\frac{\partial{f}}{\partial{c_1}}g(z)\mathrm{d}z$

The last condition corresponds to what bobhawke found using the substitution method. Steven finds the same solution, but imposing equality of the distribution functions does not correspond exactly to what is asked.


Answer first:

The `objective function' is in fact a functional. The task is to find a pdf $g(y)$ folowed by $c_2$ and a real number $c_1$ that maximise the following Lagrangian

$$ \mathcal{L}(c_1, g) = u(c_1) + \int u(x)g(x) \mathrm{d}x + \int \lambda(y) \left[ g(y) - \eta(c_1, y) \right] \mathrm{d}y. $$

Here, $\eta(c_1, y)$ is the pdf followed by the random variable $f(c_1, Z)$. $\lambda(y)$ is a Lagrange multiplier, used to enforce the constraint $g(y)=\eta(c_1,y)$. All integrals are over $\mathbb{R}$. The first order conditions are

$$ u'(c_1) - \frac{\partial}{\partial c_1} \int \lambda(y) \eta(c_1,y) \mathrm{d} y =0, \\ u(y) + \lambda(y) =0, \\ g(y) - \eta(c_1, y) = 0. \\ $$

Combining the first two gives

$$ u'(c_1) + \frac{\partial}{\partial c_1} \int u(y) \eta(c_1,y) \mathrm{d} y =0. $$

Now using the 'law of the unconscious statistician' gives

$$ u'(c_1) + \frac{\partial}{\partial c_1} \int u(f(c_1,z)) \phi(z) \mathrm{d} z =0 \\ \Rightarrow u'(c_1) + \int u'(f(c_1,z)) \frac{\partial f(c_1,z)}{\partial c_1} \phi(z) \mathrm{d} z =0 .$$ Here, $\phi(z)$ is the pdf followed by the random variable $Z$. This is equivalent to the solution given by OP.

Now I want to address a few other things. I would ideally do this in the comments, but I don't have enough reputation to comment.

First of all, I am OP. I really am at a loss as to why I couldn't log in with original credentials. Still trying to figure that one out.

As denesp points out, I wrote an answer not so long ago which I immediately deleted. I did this for a few reasons:

  1. I didn't actually answer the question in that post.
  2. I was on the bus at the time, typing on my phone. The formatting I had wasn't correct, and the wording was very loose. I wanted to take more time over it and give a better and more precise answer. So, for example, some of denesp's comments are directed at imprecise wording in a post I deleted.

To address denesp's comments on the original post: I commented briefly that things don't work because

  1. I thought it was very obvious why the suggested things don't work.
  2. I didn't think a detailed post explaining exactly why they don't work with lots of maths was appropriate for the comments

Alecos' 'answer' isn't an answer at all. It may better be read as an expansion upon why the naive approach in the original post doesn't work.

Now let me address the matter of differentiation. It's true that I wasn't precise in the original post, but this was to illustrate the naive approach I was taking and why it wasn't working.

The confusion here stems from the following. Say I have a differentiable function $f: \mathbb{R} \rightarrow \mathbb{R}$. I can certainly talk meaningfully about the function that is the derivative of $f$, that I will denote by $f'$.

On the other hand, sometimes people 'allow the argument to be a random, $\mathbb{R}$-valued variable'. When they do this, they should probably call the resulting random variables something other than $f$ and $f'$. All the confusion has arisen from this. I did not make this distinction in the original post, because my thinking on the question was not clear, and this has been confounded by other comments not recognising this base cause of the confusion.

  • $\begingroup$ Here you set a different constraint on $c_2$ than in the original question. You say that $c_2$ must be equal to the probability density function of $f(c_1,Z)$. In the original question you say that $c_2$ must be equal to $f(c_1,Z)$ and not to the latter's pdf. What is the correct one? Also: a random variable is a function, usually a real-valued measurable function. $\endgroup$ – Alecos Papadopoulos Sep 29 '16 at 20:38
  • $\begingroup$ I think you've misread. Above, the constraint is that the pdf $g(y)$ followed by $c_2$ must be equal to the pdf $\eta(c_1, y)$ followed by $f(c_1, Z)$. There is no difference. You're correct on the second point, I did not speak precisely. Small edit has been made. $\endgroup$ – Steven Sep 29 '16 at 21:34
  • 1
    $\begingroup$ Indeed the system appears to work. Your answer answers a different question than the one posted, so even though closely connected, it is not an answer to the question asked. My answer is an answer because it explains why we cannot use the Lagrangian in the circumstances specified in the question. $\endgroup$ – Alecos Papadopoulos Oct 4 '16 at 17:09
  • 1
    $\begingroup$ The constraint you have imposed in your answer is indeed implied by the original one, but it does not imply the original one. So the solution in your answer does not guarantee that $c_2 = f(c_1, Z)$, only that their probability densities are the same. This in no way is enough to have $c_2 = f(c_1, Z)$. $\endgroup$ – Alecos Papadopoulos Oct 4 '16 at 20:38
  • 1
    $\begingroup$ Ok, so let's just make certain that this correspondence holds. If you can find the time, it would be helpful to enhance this answer with the detailed steps of deriving the second first-order condition, the one with respect to $g$. $\endgroup$ – Alecos Papadopoulos Oct 4 '16 at 21:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.