Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form
$$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$
This differs from the discounted payoff in the other dynamic optimization settings, e.g., see the Bellman equation in control theory.
Why the difference?
In my experience, it's mainly just for cleanliness for results.
Consider an infinite horizon repeated game, with discounted payoff representation (where I use $\delta = (1-\lambda)$ in your notation) $$ (1-\delta)\sum_{t=0}^{\infty}\delta^t R_t $$ where $0 < \delta < 1$.
Suppose I play a strategy that gives me the same payoff, say $a$, for each period $t$. Then, $$ (1-\delta)\sum_{t=0}^{\infty}\delta^t a = (1-\delta)\frac{a}{1-\delta} = a $$ which is cleaner than $$ \frac{a}{1-\delta} $$ As a side note, multiplying the utility by a constant does not change preferences, so we keep the inherent preferences the same.
Are you referring to repeated games, in which player $i$ obtains a payoff of the form $$(1-\delta)\sum_{t=0}^{\infty}\delta^{t}u_{i}(x_{i}^{t})$$
from the sequence of payoffs $\left\{x_{i}^{t}\right\}$? The $(1-\delta)$ is appended to the front to give the average per period payoff.
