AHP Group Consensus Indicator – how to understand and interpret?

BPMSG’s AHP excel template and AHP online software AHP-OS can be used for group decision making by asking several participants to give their inputs to a project in form of pairwise comparisons. Aggregation of individual judgments (AIJ) is done by calculating the geometric mean of the elements of all decision matrices using this consolidated decision matrix to derive the group priorities.

AHP consensus indicator

In [1] I proposed an AHP group consensus indicator to quantify the consensus of the group, i.e. to have an estimate of the agreement on the outcoming priorities between participants. This indicator ranges from 0% to 100%. Zero percent corresponds to no consensus at all, 100% to full consensus. This indicator is derived from the concept of diversity based on Shannon alpha and beta entropy, as described in [2].  It is a measure of homogeneity of priorities between the participants and can also be interpreted as a measure of overlap between priorities of the group members.

How to interpret?

If we would categorise group consensus in the three categories low, moderate and high, I would assign the following percentages to these categories:

  • low consensus: below 65%
  • moderate consensus: 65% to 75%
  • high consensus: above 75%

Values below 50% indicate that there is practically no consensus  within the group and a high diversity of judgments. Values in the 80% – 90% range indicate a high overlap of priorites and excellent agreement of judgments from the group members.

AHP Consensus indicator and AHP Consistency Ratio CR

AHP allows for (logical) inconsistencies in judgments; the AHP consistency ratio CR is an indicator for this, and – as a rule of thumb – CR  should not exceed 10% significantly. Please read my posts here and here.

It can be shown that,  given a sufficiently large group size, consistency of the aggregate comparison matrix is guaranteed, regardless of the consistency measures of the individual comparison matrices, if the geometric mean (AIJ) is used to aggregate [3] . In other words, if the group of participants is large enough, the consistency ratio of the consolidated group matrix CR will decrease below 10% and is no longer an issue.

Consensus has to be strictly distinguished from consistency. The consensus is derived from the outcoming priorities and has nothing to do with the consistency ratio. Whether you have a small or a large group, in both cases consensus could be high or low, reflecting the “agreement” between group members. Even if you ask a million people, there could be no agreement (consensus) on a certain topic: half of them have the exact opposite judgment as the other half. As a result, the consensus indicator would be zero: there is no overlap, the total group is divided into two sub-groups having opposite opinions.

Analyzing group consensus – groups and sub-groups

The beauty of the proposed AHP consensus indicator based on Shannon entropy is the possibility to analyse further, and to find out, whether there are  sub-groups (cluster) of participants with high consensus among themself, but with low consensus to other sub-groups. This can be done using the concept of alpha and beta diversity [2]. I have published an excel template to to analyze similarities between the samples based on partitioning diversity in alpha and beta diversity. It can be also be used for your AHP results to analyse group consensus.

References

[1] Klaus D. Goepel, (2013). Implementing the Analytic Hierarchy Process as a Standard Method for Multi-Criteria Decision Making In Corporate Enterprises – A New AHP Excel Template with Multiple Inputs, Proceedings of the International Symposium on the Analytic Hierarchy Process, Kuala Lumpur 2013

[2] Lou Jost, (2006). Entropy and Diversity, OIKOS Vol. 113, Issue 2, pg. 363-375, May 2006

[3] Aull-Hyde, Erdoğan, Duke (2006). An experiment on the consistency of aggregated comparison matrices in AHP, European Journal of Operational Research 171(1):290-295 · February 2006

Share on Facebook

AHP Frequently Asked Questions

Over the year I receive many questions about AHP, my AHP excel template and my AHP online Software. Here a selection of frequently asked questions:

General Questions

Q: I have 15 criteria and 20 alternatives, can you extend your template/software?
A: The excel template can handle up to 10 criteria, my online software is limited to 12 criteria (in one hierarchy level) and 10 alternatives. In principle it could be extended, but the limitation is inherit to the AHP method. Please read my explanations here.

Q: I have more than 10 alternatives, can I use AHP for priority evaluation of criteria and a different method for the evaluation of alternatives?
A: Yes, you can combine AHP for criteria evaluation with another method for alternative evaluation. Alternative evaluation could be done for example using a simple table with a yes/no or applicable/not applicable scale, or any other scale, e.g. Likert scale, how good the individual alternative matches the specific criterion.

Q: I have 150 participants, can I use your excel template/software?
A: My AHP excel template is limited to 20 inputs, my AHP online software can handle a (practically) unlimited number of participants. Use the AHP online software.

Q: How can I resolve the inconsistency (CR>0.1), when participants are done with their pairwise comparisons.
A: Once the pairwise comparison is done and submitted, data can not be changed and consistency ratio is what it is. Ask your decision makers to adjust their judgments  in direction of the most consistent input during the pair-wise comparisons for the highlighted three most inconsistent comparisons. Please see also my posting here.

Q: Can I use the Likert scale instead of the AHP scale?
A: No, AHP is based on the rational scale 1/9 … 1 … 9. It cannot be replaced by the Likert scale.

Q: Do you support Fuzzy AHP?
A: No, I have made no provisions to support Fuzzy AHP, neither in Excel, nor in my online software.

Q: How is the computation done, where do I find the description and formulas?
A: Please download the manual for the excel template from here, and the software description from here.

Q: How can I cite your work, can you give me a reference?
A: Please cite my paper: Implementing the analytic hierarchy process as a standard method for multi-criteria decision making in corporate enterprises–a new AHP excel template with multiple inputs

AHP Excel template

Q: Can I extend the number of participants to more than 20?
A: Though it is possible in principle, the better way is to use my AHP online software with (practically) unlimited number of participants. I will not do a further extension of the template.

Q: Do you have a version of the Excel template w/o multiple inputs?
A: Yes, a simplified version is available on request from the author.

Q: How can I do alternative evaluation using your Excel template.
A: It is not possible. The template can only handle one category of a hierarchy and calculate the priority of one set of criteria.

AHP Online software

Q: If I have a group of decision makers, do they need to register for the online software?
A: No, they don’t need to register. As the owner of a project you get a link for group decision inputs. Simply send them the link, and they can start the pairwise comparisons.

Q: Can I erase/delete inputs from individual participants from the group results?
A: Sorry, at the moment there is no possibility to erase/delete inputs of individual participants. You can open a project with participants’ input and click “Use consol. priorities”. Then “Reset priorities” and “Save hierarchy”. Then you will have the same hierarchy as a new project without participants’ input.

Q: The alternative evaluation is not working?
A: Criteria evaluation (priorities) and alternative evaluation have to be handled as two different projects. Only when you have a decision hierarchy with completed comparisons and evaluated priorities, you can define the alternatives from the group result page clicking on “Use consol. priorities”. Define number and name of alternatives from there and save as new project. Hierarchy evaluation and alternative evaluation projects appear as type “H” for the first and type “A” for the latter in your project list.

Q: Can I get the source code of your online software?
A: Sorry, it is not an open source project.

Share on Facebook

AHP Excel Template Version 2016-05-04

Just a small correction in the summary sheet: Now the correct name of the selected scale is displayed.

You can download the latest version of the AHP Excel Template from here.

Please consider a small donation, if you find the template useful for your work.

Incoming search terms:

  • AHP 20 latest correction
Share on Facebook

Improving AHP consistency

In the implementation of both, AHP excel template as well as AHP online software, inconsistent judgments are highlighted and recommendations for consistent judgments are given.  How it is done, and what is the method behind?

The method is based on Saaty’s article “Decision-making with the AHP: Why is the principal eigenvector necessary“. European Journal of Operational Research 145 (2003) 85–91. He describes three methods how to transform a positive reciprocal matrix to a near consistent matrix.

In my implementation I construct the matrix εij = aij wj/wi to identify the three judgments for which εij is farthest from one. In Saaty’s paper it is shown as table 3, in my excel template it is called Consistency Error Matrix.  As I have to do it on each individual input sheet, I use as an approximation the RGMM (row geometric mean method) results, before calculating the eigenvector solution in the summary sheet.

Share on Facebook

How to validate the correct software implementation of AHP?

Sometimes I receive the question, how to validate the correct implementation of my AHP excel template or my AHP online software, i.e. are the results correct and reliable? There are a number of steps to test the correct implementation of the method:

  1. Check for a 2 x 2 decision matrix.
  2. Use “1” for all elements of the decision matrix, i.e. each criterion has the same importance as all other criteria.
  3. Use extremes, i.e. one criterion is extremely (9 times) more important than all other criteria.
  4. Take specific examples from the literature – published research papers containing data of the decision matrix and priorities – and compare.
  5. Compare the results for same input data with the results from another software implementation of the AHP method.

This is what I have done, and in the following I will explain each step in more detail.

1. Check for a 2 x 2 decision matrix

For two criteria there is only one comparison and one solution only: If criterion A is x-times more important than B, the weight w(A) = x/(1+x). w(B) = 1-w(A) as w(A)+w(B) = 1. Eigenvalue = 2; CR is always 0.

Example: Criterion A is 3 times more important than criterion B:
w(A) = 3/4 (75%), w(B) = 1/4 (25%). Check: w(A)/w(B) = 3.

2. Each criterion has the same importance as all other criteria

If all criteria have the same importance, the resulting weight should be 1/n, with n the number of criteria.

Example: Four criteria should result in a weight of 1/4 = 25% for each criterion.

3. One criterion is 9 times more important than all other criteria

If one criterion is 9 times more important than all other criteria, the weights depend on the number of criteria, the maximum weight or maximum priority wmax is always

wmax = M/(n + M – 1)

with M = 9, the maximum of the AHP scale, and n the number of criteria. All other weights should be

wmin = (1 – wmax )/(– 1)

Example: 5 criteria, one criterion 9 times more important than all others

wmax = 9/(5 + 9 – 1) = 9/13 = 69.2%

wmin = (1 – 69.2% )/(5 – 1) = 0.31/4 = 7.7%

4. Specific examples from the literature

Here a practical example comparing the results with  an example (7 criteria) given by Saaty in Int. J. Services Sciences, Vol. 1, No. 1, 2008 (p 86, table 2). The AHP matrix is:

1 9 5 2 1 1  1/2
 1/9 1  1/3  1/9  1/9  1/9  1/9
 1/5 3 1  1/3  1/4  1/3  1/9
 1/2 9 3 1  1/2 1  1/3
1 9 4 2 1 2  1/2
1 9 3 1  1/2 1  1/3
2 9 9 3 2 3 1

The result according Saaty is
(0.177,  0.019, 0.042, 0.116, 0.190, 0.129, 0.327) with consistency ratio of 0,022

My AHP Excel template and my online software should give the same results.

5.  Comparison with other software implementations

As I have implemented AHP under Excel and written in php script language for the online version, I can simply compare the results from both implementations, using the same input data.

Share on Facebook

AHP Excel Template Version 2015-06-07

Thanks to feedback from Frank, this updated version got some minor modifications for the case of two criteria only.

You can download the latest version of the AHP Excel Template from here.

Share on Facebook

AHP Excel Template Version 2015-04-09

In this version I changed the minimum number of criteria from three to two; the number of criteria  ranges now from 2 to 10.

It seems in the Turkish language version of Excel the selection of the consolidated matrix (“0”) gives an error. I did some minor modifications, but I am not sure whether it is now working. Please give me your feedback.

You can download the latest version of the AHP Excel Template from here.

Share on Facebook

AHP Excel Template Version 2014-07-26

Thanks to a comment from Patric, I corrected a wrong reference in the multInp sheet. The matrix of participant 13 was referring to the matrix of participants 12. You can download the latest version of the AHP Excel Template from here.

Share on Facebook

Updated AHP Excel Template Version 2014-05-09

Thanks to feedback from Benedikt, this latest update contains a minor change, to show the convergence of the power method, when calculating the eigenvalue. In the summary sheet a threshold (squared Euclidean distance d2) can be set, to show how many iterations it takes, until the change of the approximated eigenvector is below the given threshold. By default the value is set to Thresh: 1E-07:

thrh

In the above example it takes 7 iterations until d2 is below 1E-07. The actual difference is  3.5E-08 (EVM check). As the number of iterations in the template is fixed to 12, care should be taken if the value reaches 12.

You might download the latest version from my AHP template download page.

Share on Facebook

Updated AHP Excel template Version 2013-12-24

In this latest update of my AHP excel template, input sheets were modified to show the proposed ideal judgments for the three most inconsistent inputs, resulting in a lower consistency ratio CR.

Example

On the left side judgment A9, A7 and A6 are highlighted as inconsistent, CR is 32%. The consistent judgment is shown as A4, A9 and A3. After correction with the proposed intensities (right side) the consistency ratio decreases to 7% below the required threshold of 10%.

You might download the latest version from my AHP template download page.

Share on Facebook