AHP Group Consensus Indicator – how to understand and interpret?

BPMSG’s AHP excel template and AHP online software AHP-OS can be used for group decision making by asking several participants to give their inputs to a project in form of pairwise comparisons. Aggregation of individual judgments (AIJ) is done by calculating the geometric mean of the elements of all decision matrices using this consolidated decision matrix to derive the group priorities.

AHP consensus indicator

In [1] I proposed an AHP group consensus indicator to quantify the consensus of the group, i.e. to have an estimate of the agreement on the outcoming priorities between participants. This indicator ranges from 0% to 100%. Zero percent corresponds to no consensus at all, 100% to full consensus. This indicator is derived from the concept of diversity based on Shannon alpha and beta entropy, as described in [2].  It is a measure of homogeneity of priorities between the participants and can also be interpreted as a measure of overlap between priorities of the group members.

How to interpret?

If we would categorise group consensus in the three categories low, moderate and high, I would assign the following percentages to these categories:

  • low consensus: below 65%
  • moderate consensus: 65% to 75%
  • high consensus: above 75%

Values below 50% indicate that there is practically no consensus  within the group and a high diversity of judgments. Values in the 80% – 90% range indicate a high overlap of priorites and excellent agreement of judgments from the group members.

AHP Consensus indicator and AHP Consistency Ratio CR

AHP allows for (logical) inconsistencies in judgments; the AHP consistency ratio CR is an indicator for this, and – as a rule of thumb – CR  should not exceed 10% significantly. Please read my posts here and here.

It can be shown that,  given a sufficiently large group size, consistency of the aggregate comparison matrix is guaranteed, regardless of the consistency measures of the individual comparison matrices, if the geometric mean (AIJ) is used to aggregate [3] . In other words, if the group of participants is large enough, the consistency ratio of the consolidated group matrix CR will decrease below 10% and is no longer an issue.

Consensus has to be strictly distinguished from consistency. The consensus is derived from the outcoming priorities and has nothing to do with the consistency ratio. Whether you have a small or a large group, in both cases consensus could be high or low, reflecting the “agreement” between group members. Even if you ask a million people, there could be no agreement (consensus) on a certain topic: half of them have the exact opposite judgment as the other half. As a result, the consensus indicator would be zero: there is no overlap, the total group is divided into two sub-groups having opposite opinions.

Analyzing group consensus – groups and sub-groups

The beauty of the proposed AHP consensus indicator based on Shannon entropy is the possibility to analyse further, and to find out, whether there are  sub-groups (cluster) of participants with high consensus among themself, but with low consensus to other sub-groups. This can be done using the concept of alpha and beta diversity [2]. I have published an excel template to to analyze similarities between the samples based on partitioning diversity in alpha and beta diversity. It can be also be used for your AHP results to analyse group consensus.

References

[1] Klaus D. Goepel, (2013). Implementing the Analytic Hierarchy Process as a Standard Method for Multi-Criteria Decision Making In Corporate Enterprises – A New AHP Excel Template with Multiple Inputs, Proceedings of the International Symposium on the Analytic Hierarchy Process, Kuala Lumpur 2013

[2] Lou Jost, (2006). Entropy and Diversity, OIKOS Vol. 113, Issue 2, pg. 363-375, May 2006

[3] Aull-Hyde, Erdoğan, Duke (2006). An experiment on the consistency of aggregated comparison matrices in AHP, European Journal of Operational Research 171(1):290-295 · February 2006

Share on Facebook

Group Decision Making with AHP-OS

My AHP free online software AHP-OS has a feature to involve a group of decision makers to give their inputs to a decision problem. In contrast to my AHP Excel template, in AHP-OS the number of participants is practically unlimited. As of now, I see users having up to 100 participants in one project.

How to use AHP-OS for Group Decision Making?

As registered user you need to start with a new project by defining your decision hierarchy. In the Project Administration Menu click on New, define your hierarchy, Submit and Save as project. You have the possibility to give a short project description, explaining the project, before it is saved.

Once saved, you will be automatically brought back to your project page, and the project will be shown on top of your project list.

grdecproject

The session code is a unique code identifying the project. When you click on the session code link, the Group Session Input screen will open, and you see a message like “Hierarchy evaluation. Session code EvUhUn Session Chair: Klaus. Project has 0 participants’ inputs. You are participating as Klaus.” Now you can either participate yourself as a decision maker in the project, or get a link to provide to your participants for their inputs.

Participating by yourself as a member of the group

Simply click on the AHP button to start the pairwise comparisons, and – once completed – click on Submit for group eval to submit your inputs.

Provide a link to let others participate as members of the group

In the group input menu click on Leave group input mode and confirm ok to leave the input mode without submitting your pairwise comparisons in the pop-up window.

groupinpmenu

Now you will see the link that you can copy/paste and send to your group members for their participation in the project.

grouplink

Click on Done in the Active session menu to go back to your project list.

Participants input

Your participants don’t need to be registered users. If you want to test by yourself, you need to logout from AHP-OS, before you open the group link.

When your participants open the link, they will be asked for their name, before they can start their pairwise comparisons. They will then see the decision hierarchy with a red lined AHP button and  a message “Click on AHP to complete pairwise comparisons. Click on Submit for group eval once completed.”

grhier1

Now they can start their pairwise comparisons. When completed, all AHP buttons and priorities will be marked green.

grhier2

After a click on Submit for group eval the AHP button disappears and a message “Ok. Data were submitted. Thank You for your participation!” is displayed.

Group results

Once you have participants submitted data, a click on the session code link in your project list will automatically bring you to the group result page. As a participant you will get a button View group result in the Group input menu.

groupinpmenu1

The group result page gives you on top the project data and list of all participants. Individual participants can be selected or deselected to analyze the group results in further details.

projdata  partlist

The next sections of the group result page shows the decision hierarchy with consolidated priorities, a breakdown by nodes (categories) with priorities for the node and the consolidated (aggregated) decision matrix. It also shows the AHP consensus indicator as a measure of group consensus.

grresult

Input data for each participant is available under View Input Data in the group result menu:

groupresmenu

For you as chair of the group session, it is also possible to download data as csv text file for further processing in a spreadsheet program. With Use consol. priorities you now can define Alternatives with the consolidated priorities of selected participants.

 

 

Share on Facebook

Welcome to BPMSG – May 2014

Concepts, Methods and Tools to manage Business Performance

Dear Friends, dear Visitors,

The latest beta version of my AHP online software (AHP-OS) has now additional features to manage complete AHP projects. AHP stands for Analytic Hierarchy Process, and is a decision support tool.  To use the full functionality, please register and log in; it’s all free.

You can store complete decision hierarchies, use them to estimate the weights of criteria and sub-criteria and evaluate up to seven decision alternatives.

AHP is also helpful to support group decision making; participants can input their individual judgments and a consolidated group result is calculated. I have prepared a practical example, where you can participate, input your judgments and view the overall group results and consensus. Just click on the link and try it out.

The development is still continuing. I am further optimizing the handling and plan to implement additional analysis, especially for group decision making. Bookmark the page and revisit from time to time to get the latest updates.

Now please enjoy your visit on the site and feel free to give me feedback – it is always appreciated.

Klaus

Klaus D. Goepel,

Singapore, May 2014

BPMSG stands for Business Performance Management Singapore. As of now, it is a non-commercial website, and information is shared for educational purposes. Please see licensing conditions and terms of use. Please give credit or a link to my site, if you use parts in your website or blog.

About the author

Share on Facebook

Participate in an AHP group session – AHP practical example

This is a practical example of an AHP group session, using AHP-OS, where you can input your judgment and see the consolidated group result of all participants.

Imagine, you plan to buy a tablet computer. Decide on the importance of criteria, like display size, battery life etc. Click on the image below, input your name, click on AHP and start the pairwise comparisons.

What feature is more important and how much on a scale from 1 to 9? Once finished,  submit your input for group evaluation and see the results.

Result for 140 Participants:

View the detailed group results here.

Share on Facebook

AHP group decision making

The figure below shows, how a group session is conducted to determine group priorities using BPMSG’s AHP online system. The group session chair initiates a group session (You need to be registered and logged in). A six character session code is generated. Participants can use this session code to log into the group session and provide their judgments. Try out a practical example, where you can participate and input your judgments



Example showing the result for two participants. See also my post about group decision making.

Share on Facebook

AHP Online Software – Version 2014-03-15

I just released a new version of my AHP Online Software. It includes the possibility to do group sessions: several participants can make the pairwise comparison, and you can view the consolidated group result.

After definition of the hierarchy using the AHP online system, click on “New Group Session” (You need to login  as  registered user). A session code will be provided. Enter your name, start the pairwise comparisons and submit for group evaluation.

Note down the session code, and provide the link, shown under the active group session, to your participants to log in and do their pairwise comparisons.

You can view the group result, once data are submitted, with a click on “View group results”. You might always review results by using the session code and your name using the AHP Group Session link.

Please be aware that this version is still a beta version. Kindly feedback any problems or bugs.

Share on Facebook