Though this probably doesn't fully answer your question, given that it's been active for a little while with no other responses, this might hopefully be able to give some direction.
First, it's important to note that the Arrow-Pratt measures of risk aversion are just one possible measure of risk aversion. While it certainly has nice properties in many set-ups, if, for example, the agent doesn't follow expected utility theory, then the Arrow Pratt measure loses it's explanatory value.
When it comes to intertemporal risk aversion, it's generally worthwhile to consider risk aversion to be "how much an agent, in period $t$ would be willing to pay to avoid a particular gamble in period $t+1$?" For this, there seem to be several different derivations of measures of risk aversion, depending on the specific assumptions made about the within-period and between period utility function. Given that you've mentioned Epstein Zin, it's probably worth pointing out that their 1989 paper's focus was on divorcing intertemporal elasticity of substitution from risk preferences, and they take 15 or so pages to go through what risk aversion means in their non-separable framework, and how to define it. Perhaps more helpful is this recent work that considers a relatively general definition of risk aversion in a multi-period macro context, which can include some recursive inter-temporal aspects of utility. This paper also considers a model with non-additively separable utility, and shows that the CRRA coefficient is still the same as it is in the additively separable case. This might be another interesting paper since it also considers risk aversion over a planning horizon without separability.
That said, as this small set of lecture notes shows, if you're primarily interested in measures of RRA using a traditional measure, define it at period $t$ by taking the overall utility function's derivatives at the period of interest. There might be some non $t$ period consumption variables in that measure, but that's fine. This set also points out the interplay between risk aversion parameters and intertemporal substitution parameters.
If I find more I'll edit this response, but hopefully this provides at least a good start!!