Klaus D. Goepel, BPMSG, Contact, last update: Sep 4, 2017
290 thoughts on “Feedback”
Hye Klaus, if i have 10 respondents and the overall result show 0.1% CR yet the individual weightage result not averagely same, is it ok? If it is acceptable, can you explain to me why?
Hi Aiman,
It can be shown that, given a sufficiently large group size, consistency of the aggregate comparison matrix is guaranteed, regardless of the consistency measures of the individual comparison matrices, if the geometric mean (AIJ) is used to aggregate. Please see:
Aull-Hyde, Erdoğan, Duke (2006). An experiment on the consistency of aggregated comparison matrices in AHP, European Journal of Operational Research 171(1):290-295 · February 2006.
Hello,
I’m using the latest excel version for doing analysis but the template is limited to 20 participants. Any version can allow 30 participants? Can you help to fix it? Thank you so much.
Danny
As discussed per email: if you have more than 20 participants, it is better to use my online software AHP-OS. Please select on the left menu “Tools” -> “AHP Online System”.
Have a nice day. When calculating iterations in the 10×10 study file, the formula “= SUM (B5: K5) / 10” is used. I have a matrix of 4×4 in my example. Should I divide by 4?
No need to change anything. It’s a bit tricky, but calculations are always done for 10×10. Eigenvector and Eigenvalue results are correct for all dimensions.
Thank you.
based on your template, I have edited a study with 381 participants. even if the number of participants is 381, should I calculate the eigenvalues and weights with the same formula?
The number of participants has nothing to do with the number of Iterations to solve the eigenvector using the power method. 20 is usually sufficient.
How did you extend the template for 381 participants?
I recorded the responses of 381 participants to the AHP questionnaire in the worksheet (1 participant in 1 row). I derived the comparison matrix from this data. I’ve created a consolidated matrix of all the processes in the multInp section of your template (RGMM (ln, exp etc.)) in a separate worksheets. I have completed the process by calculating eigenvalues and weights from this matrix.
(1) If the overall consistency is 10%, does it matter than the individual consistency ratings are much worse? I feel that high consistency ratings among the individuals means they need to have more training and discussions about the criteria. Don’t feel comfortable just changing their ratings to be more consistent. Seems like it’s an indicator of something wrong with the criteria (or the people chosen to vote!)
(2) How to perform sensitivity analysis? I don’t see this in the downloaded version. Do you have to use the online version for this?
(1) If the overall consistency of the consolidated matrix is 10%, individual consistency ratio does not matter, as the eigenvector solution is calculated from the consolidated (near consistent) matrix. Aggregation of individual judgments “smoothens” out the individual high CR values.
Don’t change participants’ ratings, as this would be a manipulation of data, if you do it without their consensus. In my projects sometimes I have a few people, who are not able to do a consistent pairwise comparison. It has something to do with their way of thinking. From my experience, additional training or explanations do not help. Usually it works well with rational/logical participants.
(2) Sensitivity analysis is only possible when you have a complete project, i.e. criteria weightings and alternative evaluations. This is only possible with my online software, not with the excel template.
Hye Klaus, if i have 10 respondents and the overall result show 0.1% CR yet the individual weightage result not averagely same, is it ok? If it is acceptable, can you explain to me why?
Hi Aiman,
It can be shown that, given a sufficiently large group size, consistency of the aggregate comparison matrix is guaranteed, regardless of the consistency measures of the individual comparison matrices, if the geometric mean (AIJ) is used to aggregate. Please see:
Aull-Hyde, Erdoğan, Duke (2006). An experiment on the consistency of aggregated comparison matrices in AHP, European Journal of Operational Research 171(1):290-295 · February 2006.
Hello,
I’m using the latest excel version for doing analysis but the template is limited to 20 participants. Any version can allow 30 participants? Can you help to fix it? Thank you so much.
Danny
As discussed per email: if you have more than 20 participants, it is better to use my online software AHP-OS. Please select on the left menu “Tools” -> “AHP Online System”.
Have a nice day. When calculating iterations in the 10×10 study file, the formula “= SUM (B5: K5) / 10” is used. I have a matrix of 4×4 in my example. Should I divide by 4?
No need to change anything. It’s a bit tricky, but calculations are always done for 10×10. Eigenvector and Eigenvalue results are correct for all dimensions.
Thank you.
based on your template, I have edited a study with 381 participants. even if the number of participants is 381, should I calculate the eigenvalues and weights with the same formula?
I’m very sorry. One more question I had. In a study where the number of participants is 381, should the number of iterations be 381? 20 is enough?
The number of participants has nothing to do with the number of Iterations to solve the eigenvector using the power method. 20 is usually sufficient.
How did you extend the template for 381 participants?
I recorded the responses of 381 participants to the AHP questionnaire in the worksheet (1 participant in 1 row). I derived the comparison matrix from this data. I’ve created a consolidated matrix of all the processes in the multInp section of your template (RGMM (ln, exp etc.)) in a separate worksheets. I have completed the process by calculating eigenvalues and weights from this matrix.
Two questions:
(1) If the overall consistency is 10%, does it matter than the individual consistency ratings are much worse? I feel that high consistency ratings among the individuals means they need to have more training and discussions about the criteria. Don’t feel comfortable just changing their ratings to be more consistent. Seems like it’s an indicator of something wrong with the criteria (or the people chosen to vote!)
(2) How to perform sensitivity analysis? I don’t see this in the downloaded version. Do you have to use the online version for this?
Great tool!
(1) If the overall consistency of the consolidated matrix is 10%, individual consistency ratio does not matter, as the eigenvector solution is calculated from the consolidated (near consistent) matrix. Aggregation of individual judgments “smoothens” out the individual high CR values.
Don’t change participants’ ratings, as this would be a manipulation of data, if you do it without their consensus. In my projects sometimes I have a few people, who are not able to do a consistent pairwise comparison. It has something to do with their way of thinking. From my experience, additional training or explanations do not help. Usually it works well with rational/logical participants.
(2) Sensitivity analysis is only possible when you have a complete project, i.e. criteria weightings and alternative evaluations. This is only possible with my online software, not with the excel template.