I do not know of any theory of repeated games without a beginning. However, some formalization may help regarding your question about stationary equilibria. I will answer your question with respect to the simple Nash equilibrium, but my remarks should also apply to more refined equilibrium concepts. Note first that we do not need to change our definition of a Nash equilibrium, as it is abstractly defined for any strategy space.
A Nash equilibrium in a repeated game with a starting point does not need to be stationary. For example, an infinitely repeated prisoners dilemma can have any sequence of "cooperate" and "defect" for both players given players do not discount future payoffs too much. This is the same for a game without a starting point. Suppose that the players have access to a binary randomization device to coordinate their strategies. Suppose they both play "cooperate" if the randomization device is 1 and "defect" otherwise. Any player deviating from this, will be punished with the other player playing "defect" for the remainder of the game. Now set the randomization device to probability $\frac{t^2}{t^2-1}$ of being 1 in period t. Clearly, the equilibrium strategies are non-stationary. However, players have no incentive to deviate as long as their time discounting is not too high.