# Derivation of index decomposition analysis

I’m currently reading a paper on index decomposition. The paper is here for reference : https://www.sciencedirect.com/science/article/pii/S0140988315001772

The paper is setting out how it has gone about deriving the index decomposition, and I’m getting a bit lost at the final step. The aim is to decompose the changes in an aggregate variable between time $$T$$ and time $$0$$.

We start from the position that we want to break an aggregate category $$V$$ down into a series $$j$$ of subcategories, $$V_j$$, each of which is impacted by a range of $$i$$ factors over time. So we have :

$$V(t) = \sum_j^m V_j (t) = \sum_j^m \big ( \prod_1^i x_{ij}(t) \big )$$

So what we are interested in is: $$V(T) - V(0) = \sum_j^m \big ( \prod_1^i x_{ij}(T) \big ) - \sum_j^m \big ( \prod_1^i x_{ij}(0) \big )$$

We can also consider $$V(T) - V(0)$$ as: $$V(T) - V(0) = \int_0^T \frac{dV(t)}{d(t)}$$

Which then leads us to consider $$\frac{dV(t)}{d(t)}$$, from the definition above we have:

$$\frac{dV(t)}{d(t)} = \sum_j^m \frac{dV_j(t)}{d(t)}$$

$$=\sum_j^m \left( \frac{dx_{1j}(t)}{dt}x_{2j}(t)x_{3j}(t)...x_{nj}(t)+\frac{dx_{2j}(t)}{dt}x_{1j}x_{3j}(t)...x_{nj}(t)+... +\frac{dx_{nj}(t)}{dt}x_{1j}(t)x_{2j}(t)...x_{(n-1)j} \right)$$

We can then re-write this as:

$$\frac{dV(t)}{dt} =\sum_j^m \left( \frac{V_j(t)}{x_{1j}(t)}\frac{dx_{1j}(t)}{dt}+\frac{V_j(t)}{x_{2j}(t)}\frac{dx_{2j}(t)}{dt} + ... + \frac{V_j(t)}{x_{nj}(t)}\frac{dx_{nj}(t)}{dt} \right)$$

Which, as $$\frac{dln(x_{ij}(t))}{dt}) = \frac{1}{x_{ij}(t)}\frac{dx_{ij}(t)}{dt}$$ we can re-write as:

$$\frac{dV(t)}{dt} = \sum_j^m V_j(t) \left( \frac{dln(x_{1j}(t))}{dt}+\frac{dlnx_{2j}(t))}{dt} +\frac{dln(x_{3j}(t))}{dt}+ ... + \frac{dln(x_{nj}(t))}{dt} \right)$$

Which gives us finally that: $$V(T)-V(0) = \int_0^T \frac{dV(t)}{dt} = \int_0^T \left(\sum_j^m V_j(t) \left( \frac{dln(x_{1j}(t))}{dt}+\frac{dlnx_{2j}(t))}{dt} +\frac{dln(x_{3j}(t))}{dt}+ ... + \frac{dln(x_{nj}(t))}{dt} \right) \right)$$

It is at this point that the paper state that as the data we want to evaluate the integral over is not continuous (which it's not) we need to use a discrete integration, which gives the solution: $$V(T)-V(0) = \sum_j^m \left( w_j \times ln \left( \frac{x_{1j}(T)}{x_{1j}(0)} \right) + w_j \times ln \left( \frac{x_{2j}(T)}{x_{2j}(0)} \right) + ... + w_j \times ln \left( \frac{x_{nj}(T)}{x_{nj}(0)} \right) \right)$$

For some weights $$w_j$$.

At this point I'm lost. I've no idea how we went from the integral to the sum, using some discrete integration, nor what these weights are that have been introduced.

Can anyone point me in the right direction to understand this better?

Thanks for any help,

Hmmm16

Unfortunately the paper is behind a paywall, so the answer is based entirely on your question.

What is done under the hood to get the result is a rather straight forward log-linearisation and, hence, approximation. If your outline is correct, the paper uses the "tricks" of taken continuous time derivative of discrete functions and discrete integration derivation and discrete integration, but ultimately it comes down to the same thing.

I'll provide an alternative derivation of the result, making the approximation explicit as well as identifying the weights $$w$$. So even if it's not the same, you can follow and confirm the result.

We use that we can approximate a variable $$X_t$$ as $$X_t \approx X\biggl(1+\log\frac{X_t}{X}\biggr)$$ where $$X$$ is a constant. The closer $$X$$ is to $$X_t$$ the better the approximation. Note that $$V(T)$$ may be far from $$V(0)$$ in which case the approximation is bad, say if the rate of change of $$V$$ from $$0$$ to $$T$$ is more than 10%.

Let's start by approximating $$V_j(T)$$ around $$V_j(0)$$. Apart from the definitions, we use that $$\log \frac{x}{y}=\log x - \log y$$. We get $$\begin{equation*} \begin{split} V_j (T) & \approx V_j(0) \biggl( 1+\log \frac{V_j(T)}{V_j(0)}\biggr)= V_j(0)\biggl(1+\log V_j(T) - \log V_j(0)\biggr) \\ & = V_j(0)\biggl(1+\log \prod_{i=1}^n x_{ij}(T) - \log \prod_{i=1}^n x_{ij}(0)\biggr) \\ & = V_j(0)\biggl(1+ \sum_{i=1}^n \log x_{ij}(T) - \sum_{i=1}^n \log x_{ij}(0)\biggr) \\ & = V_j(0)\biggl(1+ \sum_{i=1}^n [\log x_{ij}(T) - \log x_{ij}(0) ] \biggr) \\ & = V_j(0)\biggl(1+ \sum_{i=1}^n \log \frac{x_{ij}(T)}{x_{ij}(0)} \biggr) \end{split} \end{equation*}$$

Subtracting $$V_j(0)$$ from both sides, we get $$\begin{equation*} \begin{split} V_j (T) - V_j(0) &\approx V_j(0)\biggl(1+ \sum_{i=1}^n \log \frac{x_{ij}(T)}{x_{ij}(0)} \biggr) - V_j(0) \\ & = V_j(0)\sum_{i=1}^n \log \frac{x_{ij}(T)}{x_{ij}(0)} \end{split} \end{equation*}$$ At this point you can already recognize similarities in the structure. Now, all that remains to do is to sum up, simplify, and identify the weights. $$\begin{equation*} \begin{split} V(T)-V(0) & = \sum_{j=1}^m [V_j (T) - V_j(0)] \\ & \approx \sum_{j=1}^m V_j(0)\sum_{i=1}^n \log \frac{x_{ij}(T)}{x_{ij}(0)}\\ & = \sum_{j=1}^m \biggl(V_j(0) \log \frac{x_{1j}(T)}{x_{ij}(0)}+ \cdots + V_j(0)\log \frac{x_{ij}(T)}{x_{nj}(0)} \biggr) \end{split} \end{equation*}$$

Note that the last line correspond exactly to the result in your question, with the weights identified as $$w_j=V_j(0)$$. So both derivations are equivalent.

• Looks legit. The only further thing I can contribute is the paragraph from the paper following the one written down in the question. This might be of further help: "In the literature, different sets of weights have been proposed by researchers. This has led to several Divisia decomposition methods: Parametric Divisia method 1 (PDM1) (Ang, 1994, Ang and Lee, 1994), AMDI (Boyd et al., 1987), LMDI-I (Ang and Liu, 2001), and LMDI-II (Ang and Choi, 1997). PDM1 and AMDI are parametric, while LMDI-I and LMDI-II are non-parametric and do not rely on any specification for the weights." Commented Dec 9, 2022 at 14:18
• "A summary of the decomposition formulae for AMDI, LMDI-I, and LMDI-II can be found in Ang et al. (2003) and Ang (2004). Eqs. (9), (10) imply that the observed arithmetic and ratio changes of the aggregate indicator may not be exactly the same as the sum and product of the estimated effects, respectively. A residual term may appear in the decomposition results. In IDA, a perfect decomposition method is one that does not give a residual term. Among the above Divisia methods, LMDI-I and LMDI-II are the only ones that give perfect decomposition results at the aggregate level (Ang et al., 2003)." Commented Dec 9, 2022 at 14:18
• @VARulle: appreciate that. For me the paper is behind a paywall.
– BrsG
Commented Dec 9, 2022 at 15:57

Sorry for this down-to-earth answer, but I think that the underlying assumption simply states that $$V_j(t)$$ is constant (up to an uncorrelated random term), or in other words: $$V_j(t)=w_j,$$ which implies that $$\int_0^T V_j(t) \frac{dln(x_{1j}(t))}{dt} dt = w_j\int_0^T \frac{dln(x_{1j}(t))}{dt} dt= w_jln\left(\frac{x_{1j}(T)}{x_{1j}(0)}\right),$$ and yields the expression for $$V(T)-V(0)$$. I do not see in the paper any effort for using numerical integration methods.

EDIT: If we try to go beyond constant values for $$V_j$$, the result will be different from the one given by the authors, in general. For instance, in the linear case, $$V(t)=a_j+b_jt$$, with $$V'(t)=b_j \neq 0,$$ we find that

\begin{align*} \int_0^T \left( a_j+b_js \right) \frac{d\ln(x_{1j})}{dt}(s) ds &= a_j\int_0^T \frac{d\ln(x_{1j})}{dt}(s) ds + b_j\int_0^T s \frac{d\ln(x_{1j})}{dt}(s) ds \\ &= a_j\ln\left(\frac{x_{1j}(T)}{x_{1j}(0)}\right) + b_j G_j(T,0), \end{align*} which is generally different from the equation given by the authors. If in top of that, we either assume that $$\ln(x_{1j}(t))' = c_{1j},$$ or that $$x_{1j}'(t) = c_{1j},$$ we still end up with an expression with $$G_j \neq 0$$ and NOT linear in $$\ln \left(x_{1j}(T)/x_{1j}(0)\right)$$.

The authors explicitly say that their equation is an approximation, and its quality will depend on the empirical properties of the data. A possibility would be to determine the weights by a regression, provided sufficient observations are available.

• To tell the truth, I don’t think that the $V_j$ are constant. In the question it is written that $\frac{dV(t)}{d(t)} = \sum_j^m \frac{dV_j(t)}{d(t)}$ Therefore, if $V_j(t)$ would be constant, the above derivatives should be $0$, so end of the story. I see that that $w_{ij}$ come from the discrete integration, are not an initial hypothesis abot $V_j$ being constant. Commented Dec 7, 2022 at 10:20
• Of course, $V_j(t)$ always gives the same value for a specific $t$. But the whole purpose is to see what happens if $t$ changes. The authors abuse notation with respect to derivatives and integrals, which can be confusing when the function is really discrete.
– BrsG
Commented Dec 7, 2022 at 12:06
• It sems to me that the author thinks of $𝑉(𝑡)$ as a function of time defined on an interval, say $[0,∞)$, so the integral makes sense. But we have data, discrete, not the function itself, that is only punctual values of the function $𝑉(𝑡)$ . Therefore, the author must resort to discrete integration methods. Commented Dec 7, 2022 at 13:00
• In other words, if one wants to speak of a discrete function, we can say that we have a function $V(t)$ defined on an interval $[0, \infty)$, but as we have discrete data, we know only a function, let’s call it $V’(t)$, defined on a discrete set, with the same values of $V(t)$ on this set. The $V’(t)$ can be thought of as a restriction to these discrete values of $V(t)$. $V(t)$ is integrable, $V’(t)$ of course not. Commented Dec 7, 2022 at 13:48
• I agree with your comments and have added a paragraph to address them. Commented Dec 7, 2022 at 14:16

Can anyone point me in the right direction to understand this better?

I give you a partial answer, that I hope can help you to have a guidance to understand the matter.

Unfortunately, the paper you linked is not readable, so I read only the abstract. And we can't know a priori the particular method used there to calculate the integral. So, without reading the paper, a complete, specific, answer is difficult to give.

the paper states that as the data we want to evaluate the integral over is not continuous (which it's not) we need to use a discrete integration

[...] I've no idea how we went from the integral to the sum, using some discrete integration, nor what these weights are that have been introduced.

As I suppose you know, besides analytical methods to solve integrals, there are numerical or discrete methods to calculate integrals.

These methods are important in two particular cases:

1. When the integral cannot be calculated via analytical methods, or, as it is said, in closed form: many integrals cannot be calculated or doesn't exist a closed form (a formula) for the integral, or better, for the primitive.

2. We have data. Data are by nature discontinuos, you have specific numerical values indexed, for instance, by time. That is, you don't know the function to be integrated, you don't have it, but only some specific, discrete values of it. Without knowing the function, of course, we cannot make the integral, so we have to resort to numerical or discrete methods.

Also discontinuos functions can be integrable, and in most cases they are, but the fact is that in your case we have no function, but only points of the function.

That's the reason why you pass from the integral to a sum, because a sum is on discrete values, integration requires a function (and that function must be integrable).

The subject of numerical or discrete integration is part of numerical analysis.

There are many numerical methods to solve integrals.

We should know the specific method used in your paper, so that we can understand also what these 'weights' $$w$$ are in that particular case:

Numerical integration methods can generally be described as combining evaluations of the integrand to get an approximation to the integral. The integrand is evaluated at a finite set of points called integration points and a weighted sum of these values is used to approximate the integral. The integration points and weights depend on the specific method used and the accuracy required from the approximation.

This quote is from an article of Wikipedia on numerical integration:

https://en.wikipedia.org/wiki/Numerical_integration

From a mathematical point of view, if one could read the original paper, we could see if , in numerical integration, some interpolatory method has been used, as Lagrange polynomials to find the weights $$w_j$$. In this case the interpolation points are given by the available observations.

Without reading the paper, it is just a guess. But, as I understand from the comments of Varulle, who quotes parts of the original article, this is not the case, other methods are used.

To continue the guessing, it is enough clear that a numerical quadrature has been used, that is a formula as

$$\int_0^T f(t) = \sum_ {j=1,..,n}w_jf(x_j)$$ ,

what is not clear is how the derivatives $$\frac{dln(x_{1j}(t))}{dt}$$ disappear.