Adequacy, suitability, effectiveness and efficiency of quality management systems: How to perceive and assess them?

Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 39 Adequacy, Suitability, Effectiveness and Efficiency of Quality Management Systems: How to Perceive and Assess them? DOI: 10.12776/QIP.V20I2.736 Jaroslav Nenadal Received 05 June 2016, Revised 16 September 2016, Accepted 27 September 2016 ABSTRACT Purpose: The paper brings set of original information related to the ISO 9001:2015 standard´s requirements focused on assessment and review of quality management systems adequacy, suitability and effectiveness. Methodology/Approach: Brainstorming, field research, seminars, comparative literature analysis, interviews and design review were used. Findings: According to the ISO 9001:2015 the quality management systems adequacy, suitability and effectiveness must be assessed and reviewed, in spite of the terms adequacy and suitability are not defined at the ISO 9000:2015 standard at all. Also literature review has discovered serious absentation in this area of interest. Additionally: the most of organizations managers (including quality professionals) do not understand these features of the modern quality management systems. Research Limitation/implication: Special research activities focused on perception and practical using the quality management systems adequacy, suitability and effectiveness assessment and review was performed on sample of 172 Czech organizations (with 30 % response rate). A hypotheses described by Fig. 1 below cannot be confirmed as relevant data are unobtainable from Czech organizations at present. Originality/Value of paper: The paper brings original set of information, regarding to definitions of terms as well as development of the quality management systems adequacy, suitability and effectiveness assessment and review at different types of organizations. Category: Research paper. Keywords: quality management systems; adequacy; suitability; effectiveness; efficiency. Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 40 1 INTRODUCTION Such terms as “adequacy”, “suitability” or “effectiveness” related to the quality management systems have firstly occurred at the ISO 9001:2000 standard, but without any remarkable or practical impact on these systems. The newest version of this standard published in 2015 (ISO, 2015a) is more exacting in this area: requirements regarding to the quality management systems adequacy, suitability and effectiveness assessment or review are included minimally at two clauses: a) “Top management shall review the organization´s quality management system at planned intervals, to ensure its continuing suitability, adequacy, effectiveness and alignment with the strategic direction of the organization.” (cl. 9.3.1). b) “The organization shall continually improve the suitability, adequacy, and effectiveness of the quality management system”. (cl. 10.3). Additionally, another requirement related to the quality management system performance is repeatedly stressed at different clauses of this standard. When consulting text of this standard in more detail, we are able to discover some serious facts which can influence practical implementation or assessment of the quality management systems against the ISO 9001:2015: a) the ISO 9001:2015 takes use the terms “adequacy”, “suitability” “effectiveness” or “performance” somewhat arbitrarily without explanation of these terms with relation to the quality management system, b) the ISO 9001:2015 standard is not concerned with mutual relationships among these terms at all, although these relationships really exist and play important role in practice, c) the ISO 9001:2015 standard ignores term “efficiency” related to the quality management system although this quality management system´s feature should be vital, d) these three shortcomings can lead to different interpretation or misunderstandings of all these terms from the point of quality professionals and managers view, e) but also internal and external auditors will be able to explain these terms differently what can influence objectivity of all types of audits, including third party audits performed by the certification bodies. Therefore, the main goal of this article is to contribute to elimination of these uncertainties by: Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 41 • defining these terms, • analyzing how these terms are perceived by quality professionals at present, • developing a set of steps within methodology focused on the quality management systems adequacy, suitability, efficiency and effectiveness assessment and review. 2 METHODOLOGY To achieve defined goals of this article, a following methods and approaches were used: • a literature review, especially focused on terms as quality management system adequacy, suitability, effectiveness and efficiency, • a brainstorming conferences held with groups of quality managers and quality technicians from Czech organizations with aim to reach consensus regarding definition of key terms, • an empirical field research how the terms as quality management system adequacy, suitability, effectiveness and efficiency are perceived by practice, • obtained finding synthesis into methodology of the quality management system adequacy, suitability, effectiveness and efficiency assessment and review. 3 LITERATURE REVIEW Unfortunately, it is not difficult to discover that such terms as quality management system adequacy and suitability are not frequently discussed throughout the world. Overwhelming majority of articles and books deals with term „performance“ only. I can select from this majority following examples: Hoyle describes how to perform the quality management performance review in area of automotive industry (Hoyle, 2009). Oakland proposed a performance measurement framework (Oakland, 2014) and both also recommend some steps for quality management systems performance review, including performance indicators. Gale has already argued that key performance indicator is customer value (Gale, 1994). Neely, et al., 2010 proposed using a process approach principle as a base for performance management system development. Set of various key performance indicators was proposed by (Namešanská, et al., 2014). Závadský and Hiadlovský searched answer to questions about various performance indicators consistency (Závadský and Hiadlovský, 2014). And we can remind also all books from Kaplan and Norton oriented to the Balanced Scorecard methodology implementation – (Kaplan and Norton, 1996; 2006; Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 42 2008) for example. This methodology can be used also within special parts of processes. For example, (Bhagwat and Sharma, 2007) introduced their approach to implementation of Balanced Scorecard methodology within supply chain. Striteska and Spickova presented results of analysis and comparison the strong and weak points of the most widely cited performance management systems (Striteska and Spickova, 2012). But unfortunately, there are only minimum resources where are terms as “adequacy” or “suitability” discussed. We have discovered only two websites which can be referred to mentioned terms: (MAS Solutions, 2015) and (Whittington & Associates, 2015). Such acute shortage of relevant resources made us to formulate and define key terms ourselves. 4 DEFINING OF KEY TERMS As it was mentioned above, the ISO 9000:2015 standard does not know such terms as adequacy or suitability with relation to the quality management system in spite of the ISO 9001:2015 standard requires the quality management systems adequacy, suitability and effectiveness assessment and review. That was why we had to define these terms first of all. We have inspired by websites (MAS Solutions, 2015) and (Whittington & Associates, 2015), as well as by the Random House Unabrigded Dictionary (Random House, 2002) on this purpose. Now, we are able to put forward a propoal of following definitons: Quality management system adequacy: is ability of this system to meet applicable requirements, specified by the organization or standards. For example, the requirements may be about the ISO 9001, contractual, organizational or regulatory demands. Simply to say: adequacy means being equal to the requirements, no more, no less. Quality management system suitability: is capability or fitness of this system to meet defined purpose. The organizations can identify various kinds of quality management system´s purpose. To guarantee a maximum level of customer´s satisfaction and loyalty, to support improvement culture at the organization or to be a catalyst in the area of organization´s excellence should serve as example of the quality management system´s purpose. On the contrary, terms effectiveness, efficiency and performance are defined at the ISO 9000:2015 standard by following way: Effectiveness: extent to which planned activities are realized and planned results are achieved. Efficiency: relationship between the results achieved and the resources used. Performance: measurable result. Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 43 See also (ISO, 2015b). Let us have a look to these definitions. From the core economic point of view, basic indicator of effectiveness is relation between benefits and costs – see (Boardman, 2011) and many others. And practically: all technical sciences associate the term “efficiency” with evaluation how a certain capacity delivered to input of the technical system is successfully converted to desirable outputs (Fried, Lovell and Schmidt, 1993) or (Hiltner, et al., 2002). As to performance definition: what is measurable result of the quality management system – that is a question! A number of certificates seem to be doubtful result, I am sure. These notes make us to define these terms more preciously: Quality management system effectiveness: relationship between the results achieved by the quality management system and the resources used. We will consider effective quality management system as system which brings undoubtful economic or social effects. Quality management system efficiency: extent to which planned activities within the quality management system are realized and planned results are achieved. Briefly, an efficient quality management system must be in rational operation. Quality management system performance: extent to which quality management system fulfils its functions and goals. By the way: author this term as well as possibilities of this performance measurement has already described (Nenadal, 2016). When giving all mentioned definitions thought we are able to come to the logic conclusion: strong relationships must exist among all these quality management features! We can depict this fact by Fig. 1. What can we read from this figure? The quality management system can be suitable and efficient, but this system need not be effective as a large amount of various resources was wasted for example. All arrows illustrated in Fig. 1 can be seen also as hypotheses which wait for future confirmation. Unfortunately, we are not able to confirm these hypotheses at present as it asks for huge amount of relevant data – and these data are simply unobtainable at Czech organizations now. 5 RESULTS OF EMPIRICAL FIELD RESEARCH As a part of special research project sponsored by VSB-TU of Ostrava we performed an empirical field research in Czech organizations during January and Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 44 Figure 1 – Mutual relationships among the quality management system´s features February 2016. A principal goal of this research was to investigate how the terms as adequacy, suitability; effectiveness and efficiency (in relation to the quality management systems) are practically perceived, used and assessed. 172 organizations from various areas of business were randomly selected. Data gathering was based on structured questionnaire which could be filled electronically. Additionally, interviews with some quality professionals were also held. A response rate was 29,7 % what means that 51 organizations gave relevant data for processing. Tab. 1 shows the organization´s distribution from business area point of view. Table 1 – Organization´s distribution from business area point of view Business area Per cent Automotive industry 19 Machinery 20 Metallurgy 4 Services 8 Chemical industry 10 Civil engineering 14 Food industry 4 Other 21 Of this sample, 49 % were large organizations, on the contrary, only 4 % were organizations with less than 10 employees. 77 % of all organizations had established and certified quality management system minimally against the ISO 9001:2008 standard. Quality management system adequacy Quality management system suitability Quality management system efficiency Quality management system effectiveness Quality management system performance Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 45 First of all, we launched above mentioned definitions of terms to all respondents and afterwards, the quality professionals had to declare if they understand these definitions. A proportion Yes (I understand it) versus No (I do not understand it) is clear from Fig. 2 – 5: Figure 2 – How the term “quality management system effectiveness” is understood Figure 3 – How the term “quality management system efficiency” is understood Figure 4 – How the term “quality management system adequacy” is understood Effectiveness 86 % 14% No Yes Efficiency 29 % 71 % No Yes Adequacy 45 % 55 % Yes No Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 46 Figure 5 – How the term “quality management system suitability” is understood The respondents were also asked to describe what approach their organizations apply for quality management system performance assessment: if this system is assessed through individual features as adequacy, suitability, effectiveness and efficiency (it is marked as Yes) or as a whole. Fig. 6 shows results. Figure 6 – Approach to quality management system assessment The organizations which answered “yes” in this case were additionally asked to list specific indicators used for quality management system adequacy, suitability, effectiveness and efficiency evaluation. Analysis of these lists allowed us to recognize that organizations take use wide range of indicators but most of them are not relevant for quality management system features evaluation. For example: we have occurred that indicators related to customer satisfaction or internal auditing are used for evaluation and assessment of all features (such indicators are about efficiency for one organization, while another organizations the same Suitability 55 % 45 % No Yes 0% 10% 20% 30% 40% 50% 60% 70% Yes Only as a whole No at all Q UALITY I NNOVATION P ROSPERITY / K VALITA I NOVÁCIA P ROSPERITA 20/2 – 2016 ISSN 1335-1745 (print) ISSN 1338-984X (online) 47 indicators apply in the field of adequacy, and so on). It is evidence that understanding of discussed terms is little bit confusing in practice, in spite of fact, that the same respondents declared sooner that these terms are fully understandable for them! We are able to summarize main lessons learned from this empirical study: a) the organizations have not mostly a problem to understand terms “effectiveness” or “efficiency” inversely against the ISO 9000:2015 definitions, b) but 50 % of organizations or so have problems related to the terms “adequacy” or “suitability”, c) the most of organizations are not aware of fact that effectiveness, efficiency, adequacy or suitability represents only a partial features of overall quality management systems performance, d) the approach to the quality management systems performance assessment as a whole seems to be logic and rationale. On the contrary: such information that 23 % of organizations do not generally perform this type of assessment is strongly correlative of number of organizations without quality management system certification, e) quality professionals are mostly confused when assign relevant indicators to such features of the quality management systems performance as effectiveness, efficiency, adequacy and suitability really are. Therefore we see as challenge all answers obtained to the last question within the survey, focused on exploring if the organizations are interesting in special methodology for quality management system´s adequacy, suitability, effectiveness, efficiency and performance assessment: 89 % of all respondents declared this concern without any hesitation as they perceive low level of knowledge in this field on one hand and as important hindrance to objective and fair quality management system assessment and review on the other hand. 6 FUNDAMENTALS OF QUALITY MANAGEMENT SYSTEMS PERFORMANCE ASSESSMENT Before anything else, we must identify set of indicators corresponding with particular features of the quality management system performance. A proposal of such set of indicators is presented by Tab. 2 – Tab. 5.