The Relationship Between Weighted Geometric Mean and Shannon's Entropy

The geometric mean is a type of average that is useful for comparing quantities that have different units or vary over a wide range. It is defined as the n-th root of the product of n numbers

$GM={\left(\underset{i=1}{\overset{n}{\prod \; <!--\; n-ary\; product\; -->}}{x}_i\right)}^{1/n}$

The geometric mean has some interesting properties that make it useful for certain applications. For instance, the geometric mean is always less than or equal to the arithmetic mean (the ordinary average), and it is always greater than or equal to the harmonic mean (another type of average that is useful for rates and ratios). The geometric mean also preserves the ratios of the data values, meaning that multiplying or dividing all the data values by a constant does not change their geometric mean, and it tends to be closer to the smaller values in the data set. This makes it more robust to outliers and skewed distributions.
However, the geometric mean has a limitation: it assumes that all the values in the data set have equal importance or probability. This may not be true in some cases, such as when the data represents different scenarios with different likelihoods of occurrence. In such situations, a better way to calculate the mean is to use a probability-weighted geometric mean (PWGM), which we can define as follows:

$Z={\left(\underset{i=1}{\overset{n}{\prod \; <!--\; n-ary\; product\; -->}}{x}_i^{p_i}\right)}^{1/n}$


This quantity is simply the weighted geometric mean of the observations$x_1,\dots \; <!--\; horizontal\; ellipsis\; -->,x_n$ with weights equal to their probabilities $p_1,\dots \; <!--\; horizontal\; ellipsis\; -->,p_n$. This means that Z is a measure of the average value of the random variable that takes into account how likely each observation is to occur. Z will be closer to the observation that has the highest probability and farther from the observation that has the lowest probability.

Let T be the sum of n observed values sampled from x. Since for every i, it holds that

$x_i^{p_i}=\exp \left(p_i\log \left(x_i\right)\right)$

then we have that

$\log \left(Z\right)=\underset{i=1}{\overset{n}{\sum \; <!--\; n-ary\; summation\; -->}}{p}_i\log \left(x_i\right)=\underset{i=1}{\overset{n}{\sum \; <!--\; n-ary\; summation\; -->}}{p}_i\log \left(\frac{x_i}{p_i}\right)+\underset{i=1}{\overset{n}{\sum \; <!--\; n-ary\; summation\; -->}}{p}_i\log \left(p_i\right)$

The second term in the last equality, is the Shannon's entropy H(x), while for every i the ratio $\frac{x_i}{p_i}$ is equal to T, therefore we can rewrite in a simplified form

$\log \left(Z\right)=\log \left(T\right)-H\left(x\right)$

whereby we finally get:

$Z=T·\; <!--\; middle\; dot\; -->\exp (-H\left(x\right))$

where the term T represents the scale of the observations, and exp(-H) represents the degree of concentration or dispersion of the distribution around its mean.This finding suggests that the weighted geometric mean of the observations is proportional to the total sum of the observations, but also inversely proportional to the exponential of Shannon's entropy of the distribution. This means that the more uniform the distribution is, the higher the entropy and the lower the weighted geometric mean. Conversely, the more skewed the distribution is, the lower the entropy and the higher the weighted geometric mean.

The bottom line is that geometric mean and Shannon's entropy are two related concepts that can help us understand the characteristics of a random variable, and by using a simple formula, we can see how they are influenced by the scale and order of the random variable.