BPMSG’s AHP excel template and AHP online software AHP-OS can be used for group decision making by asking several participants to give their inputs to a project in form of pairwise comparisons. Aggregation of individual judgments (AIJ) is done by calculating the geometric mean of the elements of all decision matrices using this consolidated decision matrix to derive the group priorities.
AHP consensus indicator
In [1] I proposed an AHP group consensus indicator to quantify the consensus of the group, i.e. to have an estimate of the agreement on the outcoming priorities between participants. This indicator ranges from 0% to 100%. Zero percent corresponds to no consensus at all, 100% to full consensus. This indicator is derived from the concept of diversity based on Shannon alpha and beta entropy, as described in [2]. It is a measure of homogeneity of priorities between the participants and can also be interpreted as a measure of overlap between priorities of the group members.
How to interpret?
If we would categorise group consensus in the three categories low, moderate and high, I would assign the following percentages to these categories:
- very low: below 50%
- low consensus: 50% to 62.5%
- moderate consensus: 62.5% to 75%
- high consensus: 75% to 87.5%
- very high consensus: above 87.5%
Values below 50% indicate that there is practically no consensus within the group and a high diversity of judgments. Values in the 75% – 90% range indicate a high overlap of priorities and excellent agreement of judgments from the group members.
AHP Consensus indicator and AHP Consistency Ratio CR
AHP allows for (logical) inconsistencies in judgments; the AHP consistency ratio CR is an indicator for this, and – as a rule of thumb – CR should not exceed 10% significantly. Please read my posts here and here.
It can be shown that, given a sufficiently large group size, consistency of the aggregate comparison matrix is guaranteed, regardless of the consistency measures of the individual comparison matrices, if the geometric mean (AIJ) is used to aggregate [3] . In other words, if the group of participants is large enough, the consistency ratio of the consolidated group matrix CR will decrease below 10% and is no longer an issue.
Consensus has to be strictly distinguished from consistency. The consensus is derived from the outcoming priorities and has nothing to do with the consistency ratio. Whether you have a small or a large group, in both cases consensus could be high or low, reflecting the “agreement” between group members. Even if you ask a million people, there could be no agreement (consensus) on a certain topic: half of them have the exact opposite judgment as the other half. As a result, the consensus indicator would be zero: there is no overlap, the total group is divided into two sub-groups having opposite opinions.
Analyzing group consensus – groups and sub-groups
The beauty of the proposed AHP consensus indicator based on Shannon entropy is the possibility to analyse further, and to find out, whether there are sub-groups (cluster) of participants with high consensus among themself, but with low consensus to other sub-groups. This can be done using the concept of alpha and beta diversity [2]. The latest version of my AHP online software comes with this feature [4].
References
[1] Klaus D. Goepel, (2013). Implementing the Analytic Hierarchy Process as a Standard Method for Multi-Criteria Decision Making In Corporate Enterprises – A New AHP Excel Template with Multiple Inputs, Proceedings of the International Symposium on the Analytic Hierarchy Process, Kuala Lumpur 2013
[2] Lou Jost, (2006). Entropy and Diversity, OIKOS Vol. 113, Issue 2, pg. 363-375, May 2006
[3] Aull-Hyde, Erdoğan, Duke (2006). An experiment on the consistency of aggregated comparison matrices in AHP, European Journal of Operational Research 171(1):290-295 · February 2006
[4] Goepel, K. D. (2022) Group consensus cluster analysis using Shannon alpha and beta entropy (preprint)