Hoover index is one of the simplest inequality indices to measure the deviation from an ideal equal distribution. It can be interpreted as the maximum vertical deviation of the Lorenz curve from the 45 degree line.
Theil index is an inequality measure related to the Shannon entropy. It is often used to measure economic inequality.
Like the Shannon entropy, Theil index can be decomposed in two independent components, for example to descbribe inequality “within” and “in between” subgroups. Low Theil or Hoover index means low inequality, high values stand for a high deviation from an equal distribution.
Ei – Effect in group i, i = 1 to N
Et – Total sum of effects in all N groups
Ai – Number of items in class i
At – Total number of items in all N groups
Eq. 1a TT = ln (At/Et) – ∑[ Ei/Et ln (Ai/Ei)]
Eq. 1b TL = ln (At/Et) – ∑[ Ai/At ln (Ei/Ai)]
Taking relative (proportional) variables
pi = Ei/Et
wi = Ai/At we get
Eq. 2a TT = ∑[ pi ln (pi/wi)]
Eq. 2b TL = ∑[ wi ln (wi/pi)]
The symmetric Theil index Ts = ½ ( TT + TL) can be expressed as:
Eq. 3 Ts = ½ ∑[ (pi –wi) ln (pi/wi)]
Comparing the symmetric Theil index with the
Eq. 4 Hv = ½ ∑ |pi – wi|
we see that for the symmetric Theil index the difference (pi – wi) is weighted with the logarithm of pi/wi.
The normalized Theil index ranges from 0 to 1:
Eq. 5 Tnorm = 1 – e–T
How does the Theil index relate to Shannon entropy?
For wi = 1/N (same number of items in all groups) we get with Shannon entropy
H = – ∑ pi ln pi and true diversity D = exp (H):
Eq. 6a TT = ln (N) – H
Eq. 6b TTnorm = 1 – D/N
MLD = (1/N) ∑ ln (1/pi)
(MLD = mean logarithmic deviation)
Eq. 7 TL = MLD – ln (N)
For the symmetric Theil index:
Eq. 8 Ts = ½ (MLD – H)
The symmetric Theil index is simply half of the difference between mean log deviation and Shannon entropy.
The Theil index can be decomposed to find “within group” (w) and “between group” (b) components:
Eq. 9 T = Tw + Tb
For j subgroups (j = 1 to K) with individual Theil index Tj
Eq. 10a TT = ∑ sj TTj + ∑ sj ln (sj/wj)
Eq. 10b TL = ∑ wj TLj + ∑ wj ln (wj/sj)
sj is the share of E in group j (Ej/Etot); wj the relative number of items in subgroup j (Nj/Ntot). The first term in (10) gives the “within group” component, the second the “between group” component.
Incoming search terms:
- hoover index
- HooverIndex TheilIndexandShannonEntropy|BPMSG
- short note on hoover index