Pairwise judgments consistency impact on quality of multi-criteria group decision-making with AHP[S1]

195 4, XXII, 2019 Information Management 10.15240/tul/001/2019-4-013 Introduction Presumably complex systems can be better understood when they are broken down into their constituent elements and structured hierarchically. Then, judgments about these elements can be synthesized on the basis of their relative importance at each level of the hierarchy into a set of overall priorities. By breaking down a reality into homogenous clusters and subdividing them into smaller ones, it is possible to integrate large amounts of information into the structure of a problem and form a more comprehensive picture of the whole system. There is a decision support methodology (DSM) which conforms to the above prescription. It is called the Analytic Hierarchy Process (AHP) and was devised at the Wharton School of Business by Thomas Saaty (1980). Its contemporary applications can be found, for example in Lidinska and Jablonsky (2018), Abdelmaguid and Elrashidy (2016), Kramulová and Jablonský (2016), and Ponis et al. (2015). This DSM is based on the pairwise judgments technique which comes from an infl uential paper of Marquis de Condorcet (1785), who used this technique in the election process (Young, 1988), and which was popularized by Thurstone (1927) thanks to his fi rst contemporary application at the beginning of the 20th century. When group decision-making (GDM) is taken into consideration, the AHP seems a particularly attractive methodology, and although it has been examined numerous times from the perspective of its effectiveness and applicability in GDM processes (see e.g. Scala et al., 2016; Saaty & Peniwati, 2008; Saaty & Vargas, 2012; Aguarón et al., 2014; Hosseinian et al., 2012; Moreno-Jiménez et al., 2005, 2008; Altuzarra et al., 2010; Sun & Greenberg, 2006), still a research gap could have been identifi ed. Thus, this paper examines judgments consistency infl uence on the credibility of priority ratios (PRs) within a particular priority vector (PV) derived from inconsistent pairwise judgments made by a decision maker (DM). Examination results generalize to the synthesized pairwise comparison matrix that is obtained on the basis of individual pairwise comparison matrices for all group members. Having in mind that a consistency index for the PCM denoting group preferences cannot be greater than the consistency index of the most inconsistent individual PCM it became possible to designate the credibility of the priority vector for the group on the basis of the most inconsistent individual PCM. The article is organized around three main sections: the introductory section which elaborates on pairwise judgments, AHP, and GDM with application of AHP; the methodological section devoted to the research methodology, comprising an illustrative example of the problem and selected pitfalls during priority ratios estimation process which builds on preselected measures of estimation errors; the investigational section encompassing the research outcome, its contribution to the research fi eld and the examination breakthrough from the viewpoint of other research papers. The fi nal part of the article constitutes the section ‘Conclusions‘ which summarizes examination fi ndings. 1. Background The AHP can be considered to be both a descriptive and prescriptive model of decision making. It promotes pairwise judgments (i.e. valuation on the basis of pairwise comparisons) of criteria and alternatives with respect to a criterion. Genuinely (as proposed by the creator of AHP), the comparison process proceeds with the application of a fundamental scale of absolute numbers that has been PAIRWISE JUDGMENTS CONSISTENCY IMPACT ON QUALITY OF MULTI-CRITERIA GROUP DECISION-MAKING WITH AHP Pawel Tadeusz Kazibudzki, Jiří Křupka EM_4_2019.indd 195EM_4_2019.indd 195 13.12.2019 12:44:4813.12.2019 12:44:48 196 2019, XXII, 4 Information Management proven in practice i.e. Saaty’s numerical scale which comprises of the integers from one (equivalent to the verbal judgment - ’equally preferred‘) to nine (equivalent to the verbal judgment - ’extremely preferred‘), and their reciprocals. Other numerical scales have been considered also see e.g. Dong et al. (2008). The methodology of AHP is based on the welldefi ned mathematical structure of consistent matrices and their associated principal right eigenvector’s (REV) ability to generate true or approximate weights, see e.g. Merkin (1979), Saaty and Vargas (1984). Generally, the problem of deriving PRs from a pairwise comparison matrix (PCM) de no ted as nxnij aA ][ with elements jiij aaa , is to estimate w = [w1, w2, w3,…, wn]T on the basis of matrix A which comprises a decision maker’s pairwise judgments (denoting DM preferences) concerning the importance of a given binary set of alternatives. Commonly PRs wi , where I = 1,…, n, are selected to be positive and normalized to unity  n ii w1, and the elements aij of matrix A are then the DM’s judgments about the PRs jiij www , where i, j = 1,…,n, and n is the number of all alternatives being considered. In a perfect judgment case then, the problem can be designated as: wwA   (1) and w can be computed by solving the eigenvector equation (1). In a perfect case (matrix A is consistent) λ is the only nonzero eigenvalue of A i.e. the nonzero solution of the characteristic equation:  0det  IA  (2) where I denotes the identity matrix of order n. In this case, also λ = n. On the other hand, when the case is not perfect (matrix A is not consistent) an estimate of the true w is the normalized principal right eigenvector (REV) associated with the maximal eigenvalue. Thus, in order to obtain the estimate it is needed to solve the general eigenvector equation: wwA  max  (3) where λmax denotes the principal eigenvalue which is not smaller than n, is simple and its existence is guaranteed by the Perron-Frobenius Theorem, see e.g. Saaty and Vargas (1984). The matrix of ratios A = (w i./.w j) is consistent, if and only if n is its principal eigenvalue and A ∙ w = n ∙ w. Further, w > 0 is unique to within a multiplicative constant. If the elements of a matrix A satisfy the condition wij = 1/wji for all i, j = 1,…, n then the matrix A is said to be reciprocal. If its elements satisfy the condition wikwkj = wij for all i, j, k = 1,…, n and the matrix is reciprocal, then it is called cardinally transitive or consistent. Matrix A can also be only transitive if the following conditions hold: (i) if for any i = 1,…, n, an element wij is not less than an element wik then wij ≥ wik for i = 1,…, n, and (ii) if for any i = 1,…, n, an element wji is not less than an element wki then wji ≥ w ki for i = 1,…, n. In the case of reciprocal PCMs – which are the only accepted PCMs for the AHP although counterarguments exist in literature (see e.g. Linares et al., 2016) the two conditions (i) and (ii) are equivalent. Fundamentally, all theories are based on axioms, so is the AHP. Its creator Saaty (2006) defi nes fi ve conditions for good approximations: reciprocity, homogeneity (the elements being compared must be of the same order of magnitude), independency (judgments about, or the priorities of, the elements in a hierarchy cannot depend on lower level elements), near consistency and uniform continuity (elements wi, I = 1,…, n should be relatively insensitive to small changes in the elements aij, only then good approximations to aij remain wi / wj ratios). The central point of AHP and the key issue for a theory of choice that is based upon AHP is the methodology of capturing (in)consistency of PCMs within the AHP. In order to derive credible priority vectors (PV) within AHP, it is necessary to impose some boundaries on (in) consistency of PCMs involved in the process. Indeed, signifi cant violation of the PCM (in) consistency may mislead the true values of priority ratios within the PV making the entire methodology itself useless. On the other hand, it does not mean that a high consistency of PCM guarantees credible values of PV because even perfectly consistent PCMs may not be error free, see e.g. Grzybowski (2016) and Temesi (2011). That is why establishing some relations between (in)consistency of PCM and the credibility of priority ratios estimates seems so important. The AHP genuine measure of PCM (in) consistency belongs to Saaty (1980) and is strictly related to the REV, which makes it EM_4_2019.indd 196EM_4_2019.indd 196 13.12.2019 12:44:4913.12.2019 12:44:49 197 4, XXII, 2019 Information Management especially attractive. It does not mean that the other PCM inconsistency measures (called consistency or inconsistency indices) do not exist. To the contrary, a number of other indices can be found in literature, see e.g. Mizuno (2019), Peláez, Martínez and Vargas (2018), Dixit (2018), Fedrizzi and Ferrari (2017), proposed quite recently. A detailed analysis of all consistency indices available in literature is beyond the scope of this research. However, a reader interested in various approaches to consistency measurement during pairwise comparisons may want to review those references. It behooves to mention that pairwise judgments consistency measurement was also a topic of more cross-sectional surveys e.g. Brunelli (2018), Kou et al. (2016). Having the perspective on a scale of research devoted to various ways of pairwise comparisons consistency identifi cation, it remains to mention that Saaty’s concept for pairwise comparisons consistency measurement proposed for AHP is currently systematically criticized, see e.g. Xu et al. (2008), Koczkodaj and Szwarc (2014), Koczkodaj and Urban (2018). However, taking into account that the AHP creator’s concept is still applied in the way it was proposed a few decades ago, interested readers in a detailed perspective of Saaty’s concept, as well a more fundamental analysis of the whole AHP approach, may want to study a more detailed examination of this methodology for instance in Wu and Kou (2016), Kou et al. (2016), and Saaty (2008b). Further discussion within this area, for reasons of brevity, is deliberately omitted. Instead, some key issues of group decision making with the application of AHP will be briefl y depicted. It is a fact that most of real-life decisions are not made by individuals, but by groups of individuals e.g. committees, councils, etc. From that perspective the relation between the quality of pairwise judgments made by individuals and the quality of the representative judgment for a group of individuals is of great importance. Thus, the prescription exists for an individual judgments aggregation in a way which enables obtaining a representative group judgment. The reciprocal property of the AHP plays an important role from that perspective. Generally, judgments have to be combined in such a way that reciprocals of the synthesized judgments are equal to the syntheses of these judgment reciprocals. It has been deduced that the only unique way to do that is to apply the geometric mean procedure. It can be done in two ways (Saaty, 2008b):  if experts are appointed as decision makers, then rather than combining their individual judgments, their fi nal outcome from a hierarchy is synthesized with application of a geometric mean;  on the other hand, if the individuals themselves have different degrees of importance i.e. voting powers, their individual judgments are raised to their voting power and the group outcome is established on the basis of their individual judgments i.e. the weighted geometric mean is formed (Formulae 7 and 8).  1 1 1              n kk k w n k w ijkij aa (7)            n k k n k w ijkij waa k 1 1 lnexp (8) where wk denotes a priority of importance for the individual. In the latter case, the fi nal outcome for a hierarchy is computed with the application of the standard AHP aggregation i.e. with application of the weighted arithmetic mean (the priority of the particular alternative under its criterion is weighted by the priority of its criterion(s), then the total priority of the given alternative is determined by the sum of their weighted priorities). It has been proven that application of the geometric mean procedure for individual preferences aggregation is the only one which satisfi es a number of important properties (Aczel & Saaty, 1983). It has been also proven, (see e.g. Liu, Zhang, & Wang, 2012; Grošelj & Stirn, 2012; Escobar, Aguarón, & MorenoJimenez, 2004; Xu, 2000), that the CI(A) of the group preferences cannot be greater than the CI(A*) of the most inconsistent individual PCM = A*, i.e.: Cl(A) ≤ max{Cl(A1), Cl(A2),...CI(An)}. However, despite the relevance of the fi ndings stated above, it needs to be EM_4_2019.indd 197EM_4_2019.indd 197 13.12.2019 12:44:4913.12.2019 12:44:49 198 2019, XXII, 4 Information Management stressed that relatively consistent individual pairwise judgments do not guarantee 100% preferences credibility derived thereof, see e.g. Temesi (2011). Thus, rather then focusing on inconsistency of the group preferences, primarily the consistency of individual pairwise judgments must be meticulously controlled from the perspective of its relationship with priority ratios estimation errors which can distort credibility of a particular priority vector. The necessity of research in this area seems to be paramount. 2. Research Methodology It is emphasized, that few research papers have dealt in depth with the above presented problem i.e. the relation between a level of the pairwise judgments (in)consistency and the range of possible estimation errors for established priority ratios, see e.g. Grzybowski (2016) and Kazibudzki (2019a). However, despite of the relevance of fi ndings published therein, those research studies concentrate on average estimation errors within particular priority vectors. The consequence of such a perspective is a tacit assumption that estimation errors for particular priority ratios are more or less the same as the mean error for the particular priority vector. It turns out, that it is not necessarily true, especially when relative measures for estimation errors are considered. The examination of these issues is in order and will be made briefl y in the below subsection entitled ‘Problem illustration’, and then thoroughly in the paper’s subsection entitled ‘Examination breakthrough’. The problem exemplifi cation is fi rst to be considered. 2.1 Problem Illustration The following hypothetical normalized priority vector (the vector of priority ratios) is considered: PV(w) = [0.0625, 0.1042, 0.1458, 0.1875, 0.2292, 0.2708]. It is assumed that the vector refl ects ‘true’ (not estimated) the DM‘s relative preference toward six objects whose relative characteristics are known e.g. the strength of preferences toward the objects is associated with the size of these objects (their mass, volume, circumference etc.), so their relative importance can be calculated by dividing the particular object’s size by the total size of all objects. On the basis of this ‘true’ PV(w), the PCM(w) is formed denoted as A(w) with elements wij = wi / wj:                      11.18181.44441.85712.64.3333 0.846211.22221.57142.23.6667 0.69230.818211.28571.83 0.53850.63640.777811.42.3333 0.38460.45450.55560.714311.6667 0.23080.27270.33330.42860.61 )(wA Then, A(w) is perturbed by perturbation factor e which single value in this example is given e = 0.5. This technique allows to emulate inconsistency during DMs judgments concerning objects and is widely accepted for this purpose since its fi rst application i.e. Zahedi (1986). In this way the perturbed matrix A(x) is obtained, where xij = wije for i ≠ j and i, j  N = {1,…,6}.                      10.59090.72220.92861.32.1667 0.423110.61110.78571.11.8333 0.34620.409110.64290.91.5 0.26920.31820.388910.71.1667 0.19230.22730.27780.357110.8333 0.11540.13640.16670.21430.31 )(xA Further, the upper triangle elements of A(x) are rounded to the closest value of Saaty’s scale and reciprocity is imposed (only the upper triangle elements i.e. elements above A(x) diagonal are considered for rounding while the lower triangle elements are computed as reciprocals of the upper triangle elements). In this way the scaled and reciprocal A(v) is obtained which refl ects DMs judgments concerning objects expressed with application of the particular preference scale, in this example Saaty’s scale. Other scales can be applied also, for references see e.g. Dong et al. (2008).                      123459 0.513347 0.33330.33331346 0.250.33330.3333135 0.20.250.250.333313 0.11110.14290.16670.20.33331 )(vA It should be emphasized that A(v), on the basis of Saaty’s consistency philosophy, should be considered as acceptably consistent because its CI(A(v)) = 0.0670, RI(6) = 1.24 thus CR(A(v)) = 0.0541 which informs of EM_4_2019.indd 198EM_4_2019.indd 198 13.12.2019 12:44:4913.12.2019 12:44:49 199 4, XXII, 2019 Information Management an acceptable level of consistency (Saaty suggested that CR < 0.1). On the basis of A(v), PV(v) is calculated, in this example, with the application of the REV method as the genuine AHP prioritization technique (PT). It behooves to mention that other PTs, which were suggested in literature for this purpose, can be also applied. For brevity, they will not be discussed in this research. However, the interested reader may want to fi nd references where these PTs are scrutinized. The most recent are Orbán-Mihálykó et al. (2017), Kazibudzki (2016b), Kułakowski (2015). Continuing the main stream of the research, on the basis of A(v) the following PV(v) is obtained with application of the REV method: PV(v) = [0.0277, 0.0566, 0.1027, 0.1714, 0.2679, 0.3736], which is different than PV(w) = [0.0625, 0.1042, 0.1458, 0.1875, 0.2292, 0.2708]. Having those two vectors of priority ratios, it is possible to compute deviations among their elements i.e. maximal absolute deviation (MaxAD), mean absolute deviation (MAD), minimal absolute deviation (MinAD), maximal relative deviation (MaxRD), mean relative deviation (MRD), and minimal relative deviation (MinRD), see formulae in Tab. 1. From the perspective of this research, devoted to designating the priority ratios estimate credibility, four deviations presented in Tab. 1 i.e. MaxAD, MaxRD, MAD and MRD, become especially signifi cant because they enable designation of confi dence intervals for ‘true‘ priority ratios (as in the classic statistical estimation theory). In this exclusively illustrative example, the confi dence for those intervals is purely hypothetic because it cannot be designated only on the basis of one case. However, iterations of similar cases are possible using Monte Carlo simulations which results can provide meaningful data in this matter. This issue is scrutinized further in the article’s subsection ‘Examination methodology’. Returning to the problem’s illustration issue, the values of four deviations especially signifi cant for the considered problem study are presented in Tab. 2. On the basis of these values (Tab. 2), the hypothetic confi dence intervals for priority ratios estimates (PRE) can be established and illustrated (Fig. 1–2). As can been noticed (Fig. 1–2), the hypothetic confi dence intervals for priority ratios have different features i.e. range and symmetry in relation to the estimated priority ratios values. Those features depend upon applied deviation.  ni ii vwvwMaxAD ...,1 max,             ni i ii w vw vwMaxRD ...,1 max,                 n i ii vw n vwMAD 1 1 ,      n ii ii w vw n vwMRD 1 1 ,  ni ii vwvwMinAD ...,1 min,             ni i ii w vw vwMinRD ...,1 min,             Source: own Tab. 1: Formulae for deviations among ‘true’ and estimated vectors of priority ratios MAD MaxAD MRD MaxRD 0.0472 0.1028 0.3239 0.5568 Source: own Tab. 2: Deviations among ‘true’ and estimated vectors of priority ratios EM_4_2019.indd 199EM_4_2019.indd 199 13.12.2019 12:44:4913.12.2019 12:44:49 200 2019, XXII, 4 Information Management Noticeably, a higher spread of hypothetic confi dence intervals and their higher asymmetry is observed when relative and/or maximum deviations are applied. For relative deviations, the spread of hypothetic confi dence intervals also depends on the value of the particular priority ratio i.e. higher values of priority ratios entail a larger spread for their hypothetic confi dence intervals: the problem is clearly visible for MaxRD (Fig. 2). It is very important to notice that although the spread and asymmetry of the hypothetic confi dence intervals for relative deviations are signifi cant, the hypothetic confi dence intervals for the fi rst and second priority ratio i.e. ]041.0,0209.0[ 1v, ]0837.0,0427.0[ 2v, established with the application of MRD do not encompass the ‘true’ values of the fi rst and second priority ratio which equal respectively x1 = 0.0625, x2 = 0.1042. It bears mentioning that the MRD has also another very unattractive feature i.e. it can mask signifi cant dispersion among particular priority ratios deviation. For example, let two normalized hypothetic vectors of priority ratios be given as PV(z) = [0.2262, 0.2729, 0.1390, 0.3619], and its estimate PV(s) = [0.2143, 0.3571, 0.1429, 0.2857]. In this case PV(z) is the ‘true‘ priority vector in relation to which a mean relative Fig. 1: Hypothetic confi dence intervals for PRE set with application of MAD and MaxAD Source: own Fig. 2: Hypothetic confi dence intervals for PRE set with application of MRD and MaxRD Source: own 0.421 0.315 0.219 0.150 0.104 0.075 0.326 0.221 0.124 0.056 0.009 0.000 0.028 0.057 0.103 0.171 0.268 0.374 0.476 0.371 0.274 0.206 0.159 0.130 0.271 0.165 0.069 0.000 0.000 0.000 0.374 0.268 0.171 0.103 0.057 0.028 0.553 0.396 0.254 0.152 0.084 0.041 0.282 0.202 0.129 0.078 0.043 0.021 0.374 0.268 0.171 0.103 0.057 0.028 0.843 0.605 0.387 0.232 0.128 0.063 0.240 0.172 0.110 0.066 0.036 0.018 0.374 0.268 0.171 0.103 0.057 0.028 MAD MRD MaxAD MaxRD EM_4_2019.indd 200EM_4_2019.indd 200 13.12.2019 12:44:5013.12.2019 12:44:50 201 4, XXII, 2019 Information Management deviation for priority ratios is calculated. In such a situation, one receives MRD = 0.15 which seems a rather small relative deviation. However, as can be noticed, singular relative deviations (SRD) among priority ratios within PV(z) and PV(s) equal, respectively SRD = [0.0527, 0.3087, 0.0280, 0.2106], and are highly divergent. In consequence, a reversal of priority ratios ranking is noted i.e. PV(z) = {3, 2, 4, 1} and PV(s) = {3, 1, 4, 2}. This is the exemplary situation which needs prevention, thus it is argued to withdraw relative deviations from further application to similar problems. Nevertheless, because the above proposition is based only on a hypothetic illustrative example, it is thoroughly examined further in this paper in the subsection entitled ‘Examination breakthrough’ for more credible conclusions. 2.2 Examination Methodology Continuing the main stream of the research, in real AHP applications, the ‘true’ vector of priority ratios (in the illustrative example denoted as PV(w)) is unknown. The entire AHP concept assumes it can be estimated with the application of the selected prioritization technique (PT), which in classic AHP is the REV method described earlier in this paper. As can be noticed from the earlier provided example, an estimate of the unknown PV can be more or less credible. In general, this credibility generally depends on the applied preference scale, PT, and the consistency of PCM on the basis of which the estimate of unknown PV is derived. An examination concerning differences between various preference scales and PTs in relation to credibility of PVs obtained with their application is beyond the scope of this research. However, the relation between the consistency of PCM and the PV estimate credibility seems particularly attractive. The examination proceeds with the application of Monte Carlo simulations coded and performed in Wolfram Mathematica Software. Taking into account the fact that Saaty’s concept of PCM consistency measurement was seriously questioned (see e.g. Xu et al., 2008; Grzybowski, 2012 Koczkodaj & Szwarc, 2014; Grzybowski, 2016; Koczkodaj & Urban, 2018), and the credibility of REV as the PT is slightly undermined (see e.g. Kazibudzki, 2019b; Bana e Costa & Vansnick, 2008; Schoner & Wedley, 1989; Budescu et al., 1986; Belton & Gear, 1983; Johnson et al., 1979), for the Monte Carlo simulations in this research, the Logarithmic Least Squares Method (LLSM) developed by Crawford and Williams (1980, 1985) is applied, as the oldest alternative for the REV (Formulae 9 and 10), as well LLSM based consistency index CI(LLSM) proposed by the same authors (Formula 11), and examined by Aguarón and Moreno-Jimenez (2003).             n i n ji j ijLLSM w w aw 11 2 lnmin (9)  n n i n j ij n n j ijLLSMi aaw /1 11 /1 1                   (10)               ji i jij LLSM w wa nn 2 log 21 2 Cl (11) To properly examine the problem from the given perspective, the following simulation scenario is considered. It behooves to mention that its assumptions come from Grzybowski (2016) who fi rst devised its framework for a similar analysis. Thus, the following steps in the scenario are considered: Step 1: For the assumed n, randomly generate a ‘true’ [n × 1] priority vector w = [w1,…, wn]T and the corresponding ‘genuine’ PCM(w) = G(w). Step 2: Randomly select an element wxy for x < y of G(w), and replace it with wxyeB, where eB is a relatively signifi cant error, randomly drawn (with application of uniform distribution) from the interval eB[2;4]. Errors of that magnitude are basically considered as relatively “signifi cant”, see e.g. Dijkstra (2013), Grzybowski (2016). Step 3: For every element wij, i < j ≤ n, other than wxy, randomly select a value eij for the relatively small error in accordance with the given probability distribution  (applied in equal proportions as gamma, log-normal, truncated normal, and uniform distribution) and replace the element wij with the element wijeij where eij is randomly drawn from the interval eij[0,5;1,5] with application of uniform distribution. Step 4: For all i, j such that i < j, round all values of wijeij of G(w) to the closest value from the selected scale. EM_4_2019.indd 201EM_4_2019.indd 201 13.12.2019 12:44:5013.12.2019 12:44:50 202 2019, XXII, 4 Information Management Step 5: Replace all elements wij for i > j of G(w) with 1/wij. The perturbed PCM(w) in Steps 2–5 denote as P(v). Step 6: On the basis of P(v) compute the value of the examined consistency index CI as well as the estimate of the vector w denoted as v derived from P(v) with application of assigned prioritization technique. Then calculate MaxAD1 and MaxAD2 i.e. the maximum and the second maximum absolute deviation between priority ratios of w and v in accordance with formula presented in Tab. 1. Save the values computed in this step as one record. Step 7: Repeat Steps 2–6 NP times. Step 8: Repeat the scenario NT times. Step 9: Save all the records as one database fi le. The above presented simulation framework enables examination of relations between performance of a given consistency index and greatest deviations between a ‘true’ and estimated vector of priority ratios. This way it is possible to associate various values of a given consistency index with highest potential estimation errors for obtained priority ratios. For formality, the simulation framework presented above exactly emulates steps scrutinized in the example provided earlier in this paper in subsection ‘Problem illustration’. All parameters of the applied probability distributions in the simulation framework i.e. gamma, log-normal, truncated normal, and uniform, are set in such a way that the expected value EV(eij) = 1. In this way the simulation examination and its results refl ect the reasonable assumption concerning human nature i.e. decision makers judgments are more or less deviated from optimal outcome but ‘close’ to it. 3. Results and Discussion For brevity it was decided to scrutinize the examination results for n = 4. It behooves to mention that for n = 3, direct interrelation between consistency indices is observed, see e.g. Bozóki and Rapcsák (2008) and/or Dijkstra (2013). 3.1 Research Outcome The simulation results are presented in Tab. 3 and 4. They are based on NP = 100, and NT = 1000. 3.2 Examination Contribution Having the empirical distribution of maximal absolute deviations between ‘true‘ and estimated priority vectors, the empirical confi dence intervals for particular priority ratios can be established e.g. with the application of ‘average maximum absolute deviation’ established during simulations. On the basis of selected statistics, the credibility of the priority vector can also be designated with the selected rank of a quantile. Thus, one can expect a confi dence interval with an average level of certainty for maximal absolute deviation when the average maximum absolute deviation is applied. In addition, one can expect a confi dence interval noted by the rank of the quantile, when quantiles of maximum absolute deviations are applied. Noticeably, confi dence intervals established on the basis of quantiles will be slightly exaggerated because the maximum absolute deviation among given priority ratios within two priority vectors cannot repeat itself (remaining deviations must be smaller). That is why another approach is proposed. Noticeable, in multicriteria decision making processes, a decision maker (DM) is usually interested in the most attractive alternative. So, the probability of the highly ranked alternative reversal is of great importance. Thus, it is proposed to apply a maximum absolute deviation and the second maximum absolute deviation for examination, if the risk of rank reversal for the fi rst two alternatives exists, and for examination purposes, how high the risk is. To exemplify, the following hypothetic normalized vector of priority ratios is considered: PV(v) = [0.62, 0.24, 0.1, 0.04]. The PV(v) designates the following ranks for evaluated options: A1 ≺ A2 ≺ A3 ≺ A4. It is assumed that the PV(v) was derived from the PCM for which CI(LLSM) = 0.109736. In this case, a decision maker may wonder about the probability of the highly ranked option reversal. In light of the research outcome, the answer for this inquiry depends on the level of certainty assumed by a decision maker. When this level equals 95%, then the 0.95-quantile of the maximum absolute deviation distribution for CI(LLSM) = 0.109736 equals 0.208885 (Tab. 3), and 0.95-quantile of the second maximal absolute deviation distribution equals 0.150579. Thus, if a difference between the fi rst two priority ratios of the hypothetic PV(v) is higher than 0.150579 + 0.208885 i.e. 0.359464 EM_4_2019.indd 202EM_4_2019.indd 202 13.12.2019 12:44:5013.12.2019 12:44:50 203 4, XXII, 2019 Information Management ii th interval for CILLSM Average CILLSM within i th interval p–quantiles of MaxAD1 between w and v for i th interval of CILLSM Average MaxAD1 between w and v p = 0.8 p = 0.9 p = 0.95 p = 0.98 p = 0.99 1 [0.0, 0.0202) 0.009789 0.055462 0.081919 0.123431 0.194786 0.241852 0.042064 2 [0.0202, 0.081) 0.050333 0.083198 0.127897 0.173985 0.230384 0.266555 0.058667 3 [0.081, 0.141) 0.109736 0.128978 0.167430 0.208885 0.257923 0.292974 0.086182 4 [0.141, 0.201) 0.170316 0.135142 0.175823 0.215673 0.260070 0.291170 0.097059 5 [0.201, 0.261) 0.230957 0.139692 0.181055 0.217441 0.260860 0.294106 0.100639 6 [0.261, 0.322) 0.290961 0.148930 0.185320 0.216985 0.258141 0.293103 0.104673 7 [0.322, 0.382) 0.351898 0.152956 0.183949 0.214520 0.257819 0.291483 0.108191 8 [0.382, 0.442) 0.411428 0.154353 0.184168 0.216551 0.261614 0.295555 0.111088 9 [0.442, 0.503) 0.472071 0.151550 0.181402 0.215974 0.259817 0.291782 0.111199 10 [0.503, 0.563) 0.532404 0.150774 0.184129 0.220709 0.265578 0.302564 0.111745 11 [0.563, 0.623) 0.591957 0.151667 0.187019 0.224346 0.271002 0.304387 0.112368 12 [0.623, 0.684) 0.652322 0.152885 0.189522 0.229630 0.275033 0.309352 0.112625 13 [0.684, 0.744) 0.713164 0.156414 0.196321 0.236259 0.286920 0.318705 0.114256 14 [0.744, 0.804) 0.773112 0.159058 0.201239 0.240192 0.290268 0.326163 0.115300 15 [0.804, 0.865) 0.833323 0.161285 0.202429 0.240903 0.290718 0.326226 0.116094 16 [0.865, 0.925) 0.893977 0.160628 0.203376 0.242114 0.294927 0.334319 0.116101 17 [0.925, 0.985) 0.954481 0.167689 0.210992 0.248777 0.297410 0.334572 0.119510 18 [0.985, 1.046) 1.014420 0.171211 0.215302 0.257097 0.307838 0.342969 0.120865 19 [1.046, 1.106) 1.075330 0.175970 0.219679 0.259016 0.312604 0.352029 0.122409 20 [1.106, 1.166) 1.135290 0.177132 0.222577 0.262351 0.312678 0.345093 0.123456 21 [1.166, 1.226) 1.194820 0.183751 0.229224 0.270164 0.324909 0.374050 0.128275 22 [1.226, 1.287) 1.256030 0.179100 0.228531 0.269188 0.317348 0.356733 0.125110 23 [1.287, 1.347) 1.316700 0.184083 0.229390 0.271408 0.324651 0.369775 0.128810 24 [1.347, 1.407) 1.376290 0.192343 0.237683 0.275558 0.337192 0.369280 0.131952 25 [1.407, 1.468) 1.437290 0.193035 0.244625 0.285832 0.336465 0.377465 0.133709 26 [1.468, 1.528) 1.497320 0.199676 0.241012 0.285629 0.345064 0.388863 0.136678 27 [1.528, 1.588) 1.557600 0.197001 0.245495 0.291895 0.348678 0.381236 0.134634 28 [1.588, 1.649) 1.618450 0.202120 0.252202 0.299193 0.359296 0.400384 0.138595 29 [1.649, 1.709) 1.678350 0.198996 0.242366 0.282109 0.342629 0.387303 0.140573 30 [1.709, oo) 2.494600 0.241975 0.296090 0.340766 0.392593 0.431660 0.164883 Source: own Note: The results were generated for n = 4 on the basis of the presented simulation framework. The outcome is based on 100,000 perturbed reciprocal PCMs. The simulation scenario assumed LLSM as the PT and Saaty’s preference scale. Tab. 3: Distribution of MaxAD1 i.e. the maximal absolute deviations for estimated priority ratios in relation to performance of the consistency index CILLSM EM_4_2019.indd 203EM_4_2019.indd 203 13.12.2019 12:44:5113.12.2019 12:44:51 210 2019, XXII, 4 Information Management in AHP-group decision making. Group Decision and Negotiation, 17(3), 249–265. https://doi. org/10.1007/s10726-007-9072-z. Mor eno-Jimenez, J. M., Joven, J. A., Pirla, A. R., & Lanuza, A. T. (2005). A spreadsheet module for consistent consensus building in AHP-group decision making. Group Decision and Negotiation, 14(2), 89–108. https://doi. org/10.1007/s10726-005-2407-8. Orb án-Mihálykó, É., Mihálykó, C., & Koltay, L. (2017). A generalization of the Thurstone method for multiple choice and incomplete paired comparisons. Central European Journal of Operations Research, 27(1), 133–159. https://doi.org/10.1007/s10100-017-0495-6. Peláez, J . I., Martínez, E. A., & Vargas, L. G. (2018). Consistency in positive reciprocal matrices: an improvement in measurement methods. IEEE access 6: 25600–25609. Ponis, S. T ., Gayialis, S. P., Tatsiopoulos, I. P., Panayiotou, N. A., Stamatiou, D.-R. I., & Ntalla, A. C. (2015). An application of AHP in the development process of a supply chain reference model focusing on demand variability. Operational Research, 15(3), 337–357. https://doi.org/10.1007/s12351-014-0163-8. Saaty, T. L . (1980). The Analytic Hierarchy Process. New York, NY: McGraw Hill. Saaty, T. L . (2006). Fundamentals of decision making and priority theory with the Analytic Hierarchy Process. Pittsburgh, PA: RWS Publication. Saaty, T. L. (2008a). Decision making with the Analytic Hierarchy Process. International Journal of Services Sciences, 1(1), 83–98. https://doi.org/10.1504/IJSSCI.2008.017590. Saaty, T. L . (2008b). Relative measurement and its generalization in decision making. Why pairwise comparisons are central in mathematics for the measurement of intangible factors. The Analytic Hierarchy/Network Process. Revista de la Real Academia de Ciencias Exactas, Fí sicas y Naturales. Serie A, Matemá ticas, 102(2), 251–318. https://doi. org/10.1007/BF03191825. Saaty, T. L., & Peniwati, K. (2008). Group decision making. Pittsburgh, PA: RWS Publications. Saaty, T. L., & Vargas, L. G. (1984). Comparison of eigenvalue, logarithmic least square and least squar e methods in estimating ratio. Mathematical Modeling, 5(5), 309–324. https://doi.org/10.1016/0270-0255(84)90008-3. Saaty, T. L., & Vargas, L. G. (2012). The possibility of group choice: pairwise comparisons and merging functions. Social Choice and Welfare, 38(3), 481–496. Scala, N. M., Rajgopal, J., Vargas, L. G., & Needy, K. L. (2016). Group decision making with dispersion in the Analytic Hierarchy Process. Group Decision and Negotiation, 25(2), 355–372. https://doi.org/10.1007/s10726-015-9445-7. Schoner, B., & Wedley, W. C. (1989). Ambiguous criteria weights in AHP: Consequences and solutions. Deci sion Sciences, 20(3), 462–475. https://doi. org/10.1111/j.1540-5915.1989.tb01561.x. Sun, L., & Greenberg, B. S. (2006). Multicriteria Group Decision Making: Optimal Priority Synthesis from Pairwise Comparisons. Journal of Optimization Theory and Applications, 130(2), 317–338. Temesi, J. (2011). Pairwise comparison matrices and the error-free property of the decision maker. Centr al European Journal of Operations Research, 19(2), 239–249. https://doi.org/10.1007/s10100-010-0145-8. Thurstone, L. L. (1927). A law of comparative judgments. Psychological Reviews, 34, 273–286. Wu, W., & K ou, G. (2016). A group consensus model for evaluating real estate investments alternatives. F inancial Innovation, 2(8), 1–10. https://doi.org/10.1186/s40854-0160027-8. Xu, W. J., Dong, Y. C., & Xiao, W. L. (2008). Is it reasonable for Saaty’s consistency test in the pairw ise comparison method? Proceedings of 2008 ISECS International Colloquium on Computing, Communication, Control, and Management, 3, 294–298. EM_4_2019.indd 210EM_4_2019.indd 210 13.12.2019 12:44:5213.12.2019 12:44:52 211 4, XXII, 2019 Information Management Xu, Z. S. (2000). On consistency of weighted geometric mean complex judgment matrix in AHP. European Jou rnal of Operations Research, 126(3), 683–687. https://doi. org/10.1016/S0377-2217(99)00082-X. Young, H. P. (1988). Condorcet’s theory of voting. American Political Science Review, 82(4), 1231–1244. https://doi.org/10.2307/1961757. Zahedi, F. (1986). A simulation study of estimation methods in the analytic hierarchy process. Socio-Eco nomic Planning Science, 20(6), 347–354. https://doi.org/10.1016/00380121(86)90046-7. Appendix – Kazibudzki, P., & Krupka, J. (2019). Deviations distributions. RepOD. http://dx.doi.org/10.18150/repod.2906388 Ing. Pawel Tadeusz Kazibudzki, PhD. Opole University of Technology Faculty of Economics and Management Department of Enterprise Organization and Management Poland [email protected] doc. Ing. Jiří Křupka, PhD. University of Pardubice Faculty of Economics and Administration Institute of Engineering Systems and Informatics Czech Republic [email protected] EM_4_2019.indd 211EM_4_2019.indd 211 13.12.2019 12:44:5313.12.2019 12:44:53 212 2019, XXII, 4 Information Management Abstract PAIRWISE JUDGMENTS CONSISTENCY IMPACT ON QUALITY OF MULTI-CRITERIA GROUP DECISION-MAKING WITH AHP Pawel Tadeusz Kazibudzki, Jiří Křupka The scope of this research encompasses issues associated with group decision making (GDM) as the most challenging process which entails various viewpoints and preferences of individuals that must be taken into consideration and somehow combined into one meaningful outcome. When GDM is taken into consideration, the AHP seems to be a particularly attractive methodology. From the perspective of its applications, an existing research gap has been identifi ed and examined in this research paper. Thus, the inconsistency of judgments impact on priority vector quality has been examined from the perspective of group decision making. Examination results generalize to the synthesized pairwise comparison matrix that is obtained on the basis of individual pairwise comparison matrices for all group members. The examination process has proceeded with the application of Monte Carlo simulations coded and executed in Wolfram Mathematica Software. Having in mind that a consistency index for the PCM denoting group preferences cannot be greater than the consistency index of the most inconsistent individual PCM it became possible to designate the credibility of the priority vector for the group on the basis of the most inconsistent individual PCM. It is emphasized that thus far only a few papers have dealt with the problem concerning the relation between a level of the pairwise judgments inconsistency and the degree of possible estimation errors for established vector of priority ratios. This research paper overcomes limitations of other examinations which distinguishes it from other papers and emphasizes its novelty. Keywords: Group decision making, AHP, prioritization quality, pairwise judgments consistency. JEL Classifi cation: D70, C02, C15, C44, C63. DOI: 10.15240/tul/001/2019-4-013. EM_4_2019.indd 212EM_4_2019.indd 212 13.12.2019 12:44:5313.12.2019 12:44:53