Tag: Q&A

AHP – High Consistency Ratio

Question: I know how AHP is working, but what I’m struggling with is, how to resolve the inconsistency (CR>0.1), when participants are done with their pairwise comparisons. It is time consuming if they go through the matrix and re-evaluate all their inputs. Do you have any suggestions?

Answer:  Yes, CR often is a problem. Also my projects show that, making the pair-wise comparisons, for many participant CR ends up to be higher than 0.1.  Based on a sample of nearly 100 respondents in different AHP projects, the median value of CR is 16%, i.e. only half of the participants achieve a CR below 16%  in my projects; 80-percentile is 36%. There seems also to be a tendency of increasing CR with the number of criteria, i.e. the median value significantly increases for more than 5 criteria.

From my experience, CR > 0.1 is not critical per se. I get reasonable weights for CR 0.15 or even higher (up to 0.3), depending on the number of criteria. The acceptance of a higher CR also depends on the kind of project (the specific decision problem), the out coming  priorities and the required accuracy (what is the actual impact on the result due to minor changes of criteria weights?).

In my latest AHP excel template and AHP online software AHP-OS the three most inconsistent judgments will be highlighted. The ideal judgment (resulting in lowest inconsistency) is shown. This will help participants to adjust their judgments on the scale to make the answers more consistent.

The first measure to keep inconsistencies low is to stick to the Magical Number Seven, Plus or Minus Two, i.e. keep the number of criteria in a range between 5 and 9 max. It has to do with the human limits on our capacity for processing information, originally published by George A. Miller in 1956, and taken-up by Saaty and Ozdemir  in a publication in 2003. Review your criteria selection, and try to cluster them in groups of 5 to 9, if you really need more.

Another possibility to improve consistency is to select the balanced-n scale instead of the standard AHP scale.  In my sample, changing from standard AHP scale to balanced scale decreases the median from 16% to 6%. You might select different scales in my template.

Conclusion

  • Try to keep the number of criteria between 5 or 7, never use more than 9.
  • Ask decision makers to adjust their judgments  in direction of the most consistent input during the pair-wise comparisons for the highlighted three most inconsistent comparisons. A slight adjustment of intensities 1 or 2 up or down can sometimes help.
  • Accept answers with CR > 10%, practically up to 20%, depending on the nature and objective of your project.
  • Do the eigenvector calculation with the balanced scale instead of the AHP scale, and compare resulting priorities and consistency. This does not require to redo the pairwise comparisons.

References

George A. Miller, The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, The Psychological Review, 1956, vol. 63, pp. 81-97

Saaty, T.L. and Ozdemir, M.S. Why the Magic Number Seven Plus or Minus Two, Mathematical and Computer Modelling, 2003, vol. 38, pp. 233-244

Goepel, K.D., Comparison of Judgment Scales
of the Analytical Hierarchy Process - A New Approach, Preprint of an
article submitted for consideration in International Journal of
Information Technology and Decision Making © 2017 World Scientific
Publishing Company http://www.worldscientific.com/worldscinet/ijitdm
(2017)

AHP template – numerical accuracy

Thanks to feedback from Mihail, here a few words about the numerical accuracy when using the AHP excel template.

AHP requires the calculation of the principal Eigenvalue, the weights are derived from the Eigenvector.  In my calculations I use the power method.  It is an iterative method, and  only one of several techniques that can be used to approximate the eigenvalues of a matrix.

Update 11.12.12

The whole calculation is shown in work sheet ’10×10′. I use 12 iterations; at the end of the sheet I do a check (the reverse calculation), using the Eigenvalue equation: (A–λ I) X = 0,  with A the AHP matrix; λ the principal Eigenvalue, and X the estimated Eigenvector. The resulting check value in cell B33 shows the sum of all matrix element of the Eigenvalue equation using the iterated Eigenvector and Eigenvalue. Ideally it should be zero.

Update 9.5.14

From version 2014-05-09 onward the template shows the convergence of the power method, when calculating the eigenvalue. In the summary sheet a threshold (squared Euclidean distance d2) can be set, to show how many iterations it takes, until the change of the approximated eigenvector is below the given threshold. By default the value is set to Thresh: 1E-07. As the number of iterations in the template is fixed to 12, care should be taken if the value reaches 12.

Examples

Here a practical example comparing the results from the power method, as now implemented in my template, with  an example (7 criteria) given by Saaty in Int. J. Services Sciences, Vol. 1, No. 1, 2008 (p 86, table 2). The AHP matrix is:

1 9 5 2 1 1  1/2
 1/9 1  1/3  1/9  1/9  1/9  1/9
 1/5 3 1  1/3  1/4  1/3  1/9
 1/2 9 3 1  1/2 1  1/3
1 9 4 2 1 2  1/2
1 9 3 1  1/2 1  1/3
2 9 9 3 2 3 1

The result according Saaty is
(0.177,  0.019, 0.042, 0.116, 0.190, 0.129, 0.327) with consistency ratio of 0,022

The result from my AHP Excel template is
(0.1775, 0.0191, 0.0418, 0.1164, 0.1896, 0.1288, 0.3268) with CR 0f 0.022
exactly the same. The check value in sheet ‘8×8 is 4E-12.

More examples

Latest Excel template download

 

 

 

AHP Consistency Ratio CR

Q: I read in some texts that a consistency ratio (actually inconsistency ratio) of less than 0.1 (10%) is good. I am not sure if your consistency ratio is a consistency ratio (i.e. the higher the percentage of the CR, the better and the more consistent the results are) vs inconsistency ratio (i.e. the consistency ratio percentage in your spreadsheet should be less in order to be more consistent).

Can you please let me know if a lower of higher percentage of the consistency ratio reflects a better more consistent response? Also, how important is the CR in the interpretation of results? If two consecutive rounds of solicited info yields very similar results, would that be acceptable even if the consistency ratio may not be good?

A: The CR in my spreadsheet is exactly the same you can find in the literature. A value less than 0.1 (10%) is good, but the threshold of 0.1 is a rule of thumb . Lower values are better than higher values, but values above 0.1 can be acceptable. It depends on the nature of your project. When you process the inputs from a group (several participants), it happens that individual CRs are above 10%, but the consolidated matrix CR is ok. Please read also my comment here.

How to use the AHP excel template in a project?

Q: I’m very new to AHP and I want to use it to identify which country is the best location to offshore a certain function of a company for my MBA project. I need to find relative importance of different factors for such decision and the relative ranking of different ountries from those factors.

How do I use this excel for such purpose? Do I run it multiple times; first for finding the priority of the factors, and then for the comparison of the countries one by one for each of the factors? And lastly multiply the priorities of the factors by each country’s priority? Is there an easier way via your template to do it?

A: There is no easier way. My template only calculates the priorities of factors in one single category of a hierarchy. If you have different categories, you have to run it multiple times (once in each category/ sub-category); then calculate the final weighting factors and make the evaluation of alternatives in your own sheet. NEW since Dec 2013: You might also use my online tool BMPSG AHP hierarchy.

I cannot generalize my template, as the hierarchy could be very different from one to another project.

;