An Investigation of Organizational Level Continuance of Cloud-Based Enterprise Systems

An Investigation of Organizational Level Continuance of Cloud-Based Enterprise Systems Dissertation zur Erlangung des Grades eines Doktors der Wirtschaftswissenschaft der Rechtsund Wirtschaftswissenschaftlichen Fakultät der Universität Bayreuth Vorgelegt von Sebastian Andreas Walther aus Speyer Dekan: Prof. Dr. Herbert Woratschek Erstberichterstatter: Prof. Dr. Torsten Eymann Zweitberichterstatter: Prof. Dr. Nils Urbach Tag der mündlichen Prüfung: 22.01.2014 To My Parents ABSTRACT i ABSTRACT Cloud-based enterprise systems are a growing trend in today’s business software market. With a steadily expanding number of implementations, cloud service providers are now turning their attention from adoption issues towards retaining their existing customer base. The difficulties even established cloud players, like e.g. Salesforce.com, face, in retaining their customers, have been emphasized by tech bloggers and practitioners alike, where the subscriptions of cloud-based enterprise systems are cancelled even at an early stage after the system has been adopted. This discontinuance of enterprise systems at an early stage is a rather new phenomenon, which is related to the subscription-based payment model of cloud services, which (theoretically) allows service cancellation without the customers having to fear financial penalties. In contrast, traditional on-premise systems (e.g. SAP ERP) are on a long term license base, where customers are contractually bound. Therefore the research question of the thesis is as follows: What factors influence the organizational level continuance intention of cloud-based enterprise systems? In an effort to answer this research question, the thesis presents five interrelated papers. The first paper develops a conceptual model to study the continuance of cloud-based enterprise systems. Building on this, paper two develops a formative measurement instrument to assess the success of operational cloudbased enterprise systems. The third paper quantitatively explores the influence of the variables identified in the conceptual model. Building on these findings, paper four conducts a stakeholder analysis to solve the problem of broad samples. Finally, the fifth paper uses the formative measurement instrument to test the final research model, which is a revision of the a priori conceptual model. The results show that continuance intention is influenced both, by information systems success variables as well as continuance inertia. In addition, behavioral variables, such as attitude towards usage also explained a decent amount of variance in the dependent variable. ACKNOWLEDGEMENTS ii ACKNOWLEDGEMENTS This dissertation is the result of my work as doctoral student at the Chair of Information Systems Management at the University of Bayreuth, Germany, and would not have been possible without the help and support of numerous people. First, I would like to express my greatest gratitude to my academic teacher and supervisor Prof. Dr. Torsten Eymann who always was there for me with great advice, both, of academic and personal nature. I don’t think there are many other places than at this chair, where I could have evolved and developed like this, focusing on research topics which I truly enjoyed. I also thank him for giving me the possibility to research abroad and to connect with scholars around the world. Second, I want to thank Prof. Saonee Sarker from the Washington State University, United States, and Dr. Darshana Sedera from the Queensland University of Technology, Australia, for their help to improve the theoretical framing of my research. Without the help of Dr. Sedera the sample size of these studies would not have been satisfactory. I also would like to thank Prof. Dr. Hubert Österle and Prof. Dr. Boris Otto from the University of St. Gallen, Switzerland, for their warm reception at their institute. In addition, I would like to thank Prof. Dr. Herbert Woratschek who I really appreciate as “oldschool” empirical scholar, and whose discussions with me, especially at the beginning of my dissertation, had a significant impact on the rigorosity of my research. Further, I would like to thank Prof. Dr. Claas Christian Germelmann for the discussions on experimental research design. Third, I would like to thank Dr. Philipp Wunderlich and Dr. Chris Horbel for their openness to discuss methodological issues on empirical research. I also would like to thank Dr. Rüdiger Eichin and Henning Schmitz from SAP, which helped me to gather data for the exploratory interviews and to test the research model. Further, I want to thank my friend Niraj Singh from SAP, who gave me access to his excellent academic network, as well as Anna Heid, who encouraged me to follow my dreams. Beyond this, I would like to thank my colleagues at the University of Bayreuth: Dr. Raimund Matros, Gaurang Phadke, Christoph Buck, Thomas Süptitz, Kathrin Nitsche, Christopher Kühn, Friederike Weissmann, Michael Stadelmann, Severin Österle and Kathrin Tauscher. I also would like to thank my colleagues at the Queensland University of Technology: Maduka Subasingha, Abdulrahman Alarifi, Maura Atapattu, Shailesh Palekar, Siti Salim and Rebekah Eden. Finally, I would like to thank my colleagues from the University of St. Gallen: Clarissa Falge, Ehsan Baghi, Simon Schlosser, Peter Schenkel, Pascal Tuppi, Torben Küpper, and Rene Abraham. I would also like to thank my students Alexander Wieneke and Andreas Plank for their excellent work. Further, I would like to thank my friends Stefanie Kreuzer, Sophia Eschenfelder and Philip Michaelis for their support. Finally, I would like to express my heartfelt gratitude to my parents and sister for their yearlong support of my studies. Without them this journey would have been a lonely one. TABLE OF CONTENTS iii TABLE OF CONTENTS ABSTRACT ..................................................................................................................................................... I ACKNOWLEDGEMENTS ............................................................................................................................ II TABLE OF CONTENTS ............................................................................................................................... III CHAPTER I: INTRODUCTION ............................................................................................................... 1 1. MOTIVATION ......................................................................................................................................... 1 2. SOFTWARE AS A SERVICE .................................................................................................................. 3 2.1. ESSENTIAL CHARACTERISTICS .............................................................................................................. 3 2.2. CLOUD SERVICE MODELS ..................................................................................................................... 4 2.3. DEPLOYMENT MODELS ......................................................................................................................... 5 3. RESEARCH DESIGN .............................................................................................................................. 6 3.1. AN INITIAL EXPLORATION OF CLOUD-BASED ENTERPRISE SYSTEMS SUCCESS .................................... 6 3.2. QUANTITATIVE ASSESSMENT ............................................................................................................... 7 4. THESIS ORGANIZATION .................................................................................................................... 10 CHAPTER II: CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION .................................... 17 1. INTRODUCTION .................................................................................................................................. 18 2. THEORETICAL BACKGROUND ........................................................................................................ 22 2.1. DISTINGUISHING BETWEEN TECHNOLOGY ADOPTION AND IS CONTINUANCE .................................... 22 2.2. AN EXPECTANCY-CONFIRMATION THEORY OF IS CONTINUANCE ...................................................... 22 2.3. DELONE AND MCLEAN MODEL OF IS SUCCESS .................................................................................. 24 3. RESEARCH MODEL ............................................................................................................................ 26 4. CONCLUSION....................................................................................................................................... 31 CHAPTER III: CLOUD ENTERPRISE SYSTEMS SUCCESS ........................................................... 35 1. INTRODUCTION .................................................................................................................................. 36 2. INFORMATION SYSTEMS SUCCESS ................................................................................................ 39 3. INSTRUMENT DEVELOPMENT PROCESS ...................................................................................... 40 3.1. CONCEPTUALIZATION AND CONTENT SPECIFICATION ........................................................................ 40 3.2. ITEM GENERATION ............................................................................................................................. 42 3.3. ASSESSING CONTENT VALIDITY ......................................................................................................... 43 3.4. PRE-TEST, REFINEMENT, AND FIELD TEST.......................................................................................... 44 3.5. QUANTITATIVE ASSESSMENT OF MEASUREMENT INSTRUMENT ......................................................... 45 3.6. RE-SPECIFICATION AND FINAL MEASUREMENT INSTRUMENT ............................................................ 48 4. FINDINGS, LIMITATIONS, AND FUTURE RESEARCH .................................................................. 50 TABLE OF CONTENTS iv CHAPTER IV: SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS ..................... 55 1. INTRODUCTION .................................................................................................................................. 56 2. THEORETICAL FRAMING .................................................................................................................. 59 2.1. THE DEPENDENT VARIABLE: SYSTEM CONTINUATION ....................................................................... 59 2.2. THE INFORMATION SYSTEMS SUCCESS MODEL .................................................................................. 59 2.3. THE DECISION MAKERS’ COGNITIVE AND AFFECTIVE RESPONSES ..................................................... 60 2.4. SOCIAL AND TECHNOLOGICAL COMMITMENT: CONTINUATION INERTIA ............................................ 61 3. HYPOTHESES DEVELOPMENT ......................................................................................................... 62 3.1. IS SUCCESS AND SUBSCRIPTION RENEWAL INTENTION ...................................................................... 62 3.2. COGNITIVE AND AFFECTIVE RESPONSES AND SUBSCRIPTION RENEWAL INTENTION .......................... 63 3.3. CONTINUATION INERTIA AND SUBSCRIPTION RENEWAL INTENTION .................................................. 64 4. METHODOLOGY ................................................................................................................................. 66 4.1. DATA COLLECTION ............................................................................................................................. 66 4.2. DATA ANALYSIS ................................................................................................................................. 67 5. RESULTS ............................................................................................................................................... 68 5.1. MEASUREMENT MODEL ...................................................................................................................... 68 5.2. STRUCTURAL MODEL ......................................................................................................................... 69 6. FINDINGS, LIMITATIONS, AND FUTURE RESEARCH .................................................................. 71 CHAPTER V: CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES ............... 78 1. INTRODUCTION .................................................................................................................................. 79 2. THEORETICAL BACKGROUND ........................................................................................................ 81 2.1. INFORMATION SYSTEMS CONTINUANCE ............................................................................................. 81 2.2. INFORMATION SYSTEMS SUCCESS ...................................................................................................... 81 2.3. EMPLOYMENT COHORT CLASSIFICATION ........................................................................................... 82 3. HYPOTHESES DEVELOPMENT ......................................................................................................... 84 4. METHODOLOGY ................................................................................................................................. 87 4.1. DATA GATHERING .............................................................................................................................. 87 4.2. DATA ANALYSIS ................................................................................................................................. 88 5. RESULTS ............................................................................................................................................... 89 5.1. MEASUREMENT MODEL ...................................................................................................................... 89 5.2. STRUCTURAL MODEL ......................................................................................................................... 90 6. GROUP COMPARISON ........................................................................................................................ 92 7. DISCUSSION, CONCLUSION, AND LIMITATIONS ......................................................................... 93 v CHAPTER VI: CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS ............................ 98 1. INTRODUCTION .................................................................................................................................. 99 2. THEORETICAL FRAMEWORK ........................................................................................................ 101 2.1. ADOPTION, CONTINUANCE, AND DISCONTINUANCE ......................................................................... 101 2.2. CONTINUANCE FORCES: INFORMATION SYSTEMS SUCCESS .............................................................. 102 2.3. CONTINUANCE INERTIA: COMMITMENT ............................................................................................ 103 3. HYPOTHESES DEVELOPMENT ....................................................................................................... 105 3.1. CONTINUANCE FORCES ..................................................................................................................... 105 3.2. CONTINUANCE INERTIA .................................................................................................................... 106 4. METHODOLOGY ............................................................................................................................... 109 4.1. DATA COLLECTION ........................................................................................................................... 109 4.2. INSTRUMENT DEVELOPMENT ............................................................................................................ 109 4.3. DATA ANALYSIS ............................................................................................................................... 112 5. RESULTS ............................................................................................................................................. 113 5.1. MEASUREMENT MODEL .................................................................................................................... 113 5.2. PATH MODEL .................................................................................................................................... 115 6. DISCUSSION ....................................................................................................................................... 117 CHAPTER VII: CONCLUSION ........................................................................................................... 126 1. RESEARCH SUMMARY .................................................................................................................... 126 2. IMPLICATIONS FOR INFORMATION SYSTEM THEORY ............................................................ 128 2.1. AN IMPERATIVE FOR INFORMATION SYSTEMS SUCCESS RESEARCH ................................................. 128 2.2. THE DIVERGENCE OF ADOPTION, CONTINUANCE, AND DISCONTINUANCE RESEARCH ..................... 129 3. IMPLICATIONS FOR RESEARCH ON CLOUD-BASED ENTERPRISE SYSTEMS ...................... 131 4. LIMITATIONS..................................................................................................................................... 133 5. FUTURE DEVELOPMENT OF RESEARCH ..................................................................................... 135 VITA ........................................................................................................................................................... 137 LIST OF PUBLICATIONS ........................................................................................................................ 138 INTRODUCTION 1 CHAPTER I: INTRODUCTION 1. MOTIVATION A growing trend in today’s business software market is the provision of enterprise applications over the internet, also known as software as a service (SaaS). From chemical companies, like e.g. BASF, to consumer goods companies, like e.g. 20th Century Fox, a steadily rising number of companies have implemented cloud-based enterprise systems (ES), such as customer relationship management (e.g. Salesforce.com), human resource management (e.g. SuccessFactors) or enterprise resource planning (ERP) systems (e.g. SAP Business ByDesign). This widespread diffusion of cloud-based ES has strong practical implications for small to medium sized enterprises (SME). While historically on-premise ES were costly and therefore mainly used by large enterprises to gain an advantage towards their competitors (Klaus et al. 2000), the emergence of cloud computing has now made sophisticated enterprise software available to SME (Salleh et al. 2012). The economic importance of cloud-based software can best be underlined by recent economic figures, where, according to Gartner, worldwide SaaS revenue is predicted to reach $22.1 billion in 2015 (Gartner 2012). Despite this outlined economic relevance, which also reflects in extensive growth rates, there are not only success stories of cloud service providers, as tech bloggers and practitioners have highlighted. Quite the contrary, even settled providers of cloud-based ES solutions, like e.g. Salesforce.com, have been facing problems in retaining their customers. Hence, investigating the antecedents of cloud service continuance is a topic of outstanding practical importance for cloud service providers to understand the demands of their clients. Studying cloud customer retention, and more specifically, the continuance of the cloud service, is not only interesting from a business perspective, but also provides an ideal setting for studying organizational level continuance of information systems from a theoretical perspective, where an extensive lack of research has been identified (e.g. Furneaux and Wade 2011). The reason for this is the payment model of cloud services, which is usually subscription-based (Mell and Grance 2009), with the (theoretical) possibility of seamless service cancellation without any direct financial penalties. Therefore, this payment model strongly contrasts the license-based models of classical on-premise ES and long-term IT ousourcing contracts, where clients are usually contractually bound over a pre-determined time, clarifying the partially mandatory organizational setting where IT decision makers face INTRODUCTION 8 into the conceptual model. Step 2: To test the conceptual model, a formative measurement instrument was created to assess the success of cloud-based ES, whereas continuance inertia and behavioral variables were measured using well-validated reflective scales. The reason why a formative instrument was chosen to measure the success of cloud-based ES, is, that it provides “actionable attributes” (Mathieson et al. 2001), which can be practically used to influence the focal construct (see Chapter 3 for a detailed discussion). Step 3: Finally, the formative measurement instrument was used to assess the research model. Based on the feedback on Chapter 4, which was given at the European Conference on Information Systems (2013), the hypotheses of the final research model were developed on an organizational level, as one reviewer noted that developing the hypotheses on an individual level of analysis might lead to a “mixed level fallacy” 3 . This also included a re-framing of the research model. Due to the page limitations, which were given in the conference proceedings, the data gathering procedure is not described in full detail in the following Chapters. Therefore, this paragraph is dedicated to providing additional information on the data collection procedure, which took place between mid-September to mid-December in 2012. In the collection of the data, we were supported by one of the largest software companies worldwide, which made our online survey available via their cloud-related social media platform. In addition, they also distributed the survey to several cloud-based ES user group executives. In this context, Dr. Darshana Sedera, lecturer at the Queensland University of Technology, gathered 46 survey responses on a user group meeting in Sydney (10/2012), where the survey was handed out as anonymous print out during a break between presentation sessions. In addition, the survey was made available via platforms liked LinkedIn or Xing, which included direct contacting of IT decision makers with adequate background. It is also noteworthy that SmartPLS (Ringle et al. 2005) was used to analyze the data. The reason for this is described specifically in each research paper. PLS has traditionally been used a lot in information systems research, including recent top publications (Furneaux and Wade 2011; Wunderlich et al. 2013). In the following, the arguments for each analysis method are not repeated in detail, however, the structural equation modeling (SEM) 3 This refers to the problem when e.g. the dependent variable is measured on an organizational level of analysis, whereas the independent variables are measured on an individual level of analysis. INTRODUCTION 9 guidelines for administrative and social science research by Gefen et al. (2011) were used to evaluate the appropriateness of the analysis method. Despite convergence problems due to the small sample sizes, which made the usage of co-variance based methods technically impossible, Table 2 shows valid and obsolete arguments where to use PLS-SEM. Table 2. Reasons for Using PLS-SEM (Gefen et al. 2011) Valid Reason Obsolete Reason Exploratory Research Objectives Model Interactions/Moderation Lack of Strong Theory Base Distribution Assumptions Formative Scales in Research Model Sample Sizes In line with the previous discussions, Chapter 2 proposed a conceptual model, which was built on theories from social psychology and information systems theory. Therefore, in contrary to the methodology discussion in the mentioned paper, co-variance based SEM would have been more appropriate, as an explicit behavioral mechanism should be tested. In contrary, the research models we tested quantitatively in Chapters 4 and 6 focused on the identified variables to answer the research question, where we did not explicitly look at the interlinkages between the constructs. In this case, the goal of the paper was not to find a model which represented the empirical reality as closely as possible (i.e. goodness of fit), but to find influence factors explaining the variance in the intention to continue system usage, which basically also would have been possible using linear regression techniques. Figure 1. General Research Design Conceptual Model Instrument Development Cloud-Based ES Success Quantitative Assessment Model Testing Semi-Structured Expert Interviews Content-Based Literature Review General IS Success Initial Exploration INTRODUCTION 10 4. THESIS ORGANIZATION The central research question of the thesis is: What factors influence the organizational level continuance intention of cloud-based ES? Based on this, the following chapters contribute to answering this question. Figure 2 summarizes the organization of the thesis, which is structured according to the research design 4 . The articles featured in this thesis only compromise the quantitative assessment to answer the research question, which is an enclosed research project by itself. All papers present revised versions of the original publications to gain a consistent layout throughout the dissertation. Figure 2. Thesis Organization 4 It is important to note that the main Chapters representing the research design are the Chapters 2 (step 1: conceptual model), 3 (step 2: instrument development) and 6 (step 3: model assessment). Chapter 4 was thought to be an initial exploration whether the identified variables in the conceptual model influence continuance intention. In addition, reviews on Chapter 4 led to the stakeholder analysis in Chapter 5 to tackle the problems of broad samples. Conclusion (Chapter 7) Model Assessment (Chapter 6) Quantitative Assessment Stakeholder Analysis (Chapter 5) Content of Thesis Quantitative Model Exploration (Chapter 4) Instrument Development (Chapter 3) Conceptual Model (Chapter 2) Introduction (Chapter 1) Initial Exploration (e.g. Walther et al. 2012; Dörr et al. 2013; Wieneke et al. 2013) INTRODUCTION 11 The process of research was coupled to a deeper understanding of concepts, theories, and terminology over the time. This steady learning is reflected in the fact that the terminology and the hypotheses development can vary between the papers. For instance, at the beginning of the research project the term on-demand ES was used, whereas in the latter stages of research this was changed to cloud-based ES. Another example was the usage of continuance and subscription renewal, which were used interchangeably between the papers and basically have the same meaning. The reason for this alternating terminology is that subscription renewal is a term used in the area of cloud-based ES, and therefore allowed to frame the model more cloud-based ES specific, whereas continuance is a general term used in information systems research for all kinds of different information systems, which in turn reflects a higher external validity and allows a better comparability between empirical results. Below, the following the papers are summarized and the contributions of the specific coauthors are outlined. Figure 3 shows the content of all papers “in a nutshell”. The first paper conceptually investigates the following research question: Which role plays confirmation in the continuation of an on-demand enterprise system in the post-acceptance phase? In this effort, expectations, confirmation, as well as organizational and technological beliefs influencing the company’s intention to continue the subscription of their operational cloud-based ES are examined. The expectation confirmation model (Bhattacherjee 2001) is integrated with variables of the information systems success model to theorize a model of information systems continuance on organizational level, as it is argued that cloud-based ES are usually used in SME, therefore it is appropriate to explain organizational continuance including behavioral variables like attitude and cognitive variables like confirmation. Especially confirmation is highlighted, as IT decision makers have usually worked with onpremise systems before using cloud-based ES, therefore their past experience with an onpremise system could influence pre-purchase expectations, which in turn influence the level of confirmation which an IT decision maker experiences. Special attention is drawn on cloud washing, which is a term used to describe characteristics of software systems, which are attributed to cloud computing, but basically are infrastructure independent. To overcome the problem of cloud washing, technological quality of a system is represented by service and application quality. I thank Prof. Dr. Torsten Eymann for his contribution to the paper by helping me to develop the research model prior to writing the paper as well as proof-reading several iterations. The second paper’s research question is as follows: How can operational cloud-based ES INTRODUCTION 12 success be measured on an organizational level? Therefore, this paper develops a formative measurement scale to assess the success of operational cloud-based ES. This is done using the scale development procedure proposed by Moore and Benbasat (1991), with newer scale development elements focusing on the development of formative scales (Diamantopoulos and Winklhofer 2001; Petter et al. 2007). The developed measurement scale includes general information systems success dimensions (e.g. Wixom and Todd 2005), as well as ES-specific (Gable et al. 2008) and SaaS-specific (Walther et al. 2012) success dimensions. The measurement instrument is quantitatively assessed using survey responses of 103 IT decision makers. Based on the results of the quantitative assessment, system quality and net benefits are re-specified as second-order constructs based on theoretical considerations. Net benefits is re-specified in line with the original information systems success model (DeLone and McLean 1992), where net benefits is the sum of organizational and individual impact. System quality is also modeled as a second order construct, where a literature-based classification scheme is developed, which includes architecture agility, system performance, business requirements, ease of utilization and security as first order constructs. The revised model shows desirable statistical properties concerning the significance 5 of single indicator’s tvalues. I thank Dr. Darshana Sedera for providing me with important literature concerning the development and interpretation of scales, as well as the thoughtful revision concerning singular paragraphs which needed clarification. I also would like to thank Prof. Dr. Saonee Sarker and Prof. Dr. Torsten Eymann for their important comments on statements and paragraphs which had to be re-written related to weaknesses which could be pointed out by reviewers. The research question imposed in the third paper is: What factors influence the subscription renewal intention of cloud ES adopters? To answer this research question a socio-technical approach is taken. In this effort, technological variables are identified, that is information and system quality, as well as technical integration from organizational level discontinuance literature. As the hypotheses are developed on an individual level of analysis, social-related variables are included, such as cognitive and affective responses of the IT decision makers to explain continuance. In addition, system investment as financial commitment (Furneaux and Wade 2011) is included, which can be seen as a variable limiting behavioral control (Ajzen 1991). Finally, net benefits is introduced (Delone and McLean 2003) as emergent interrelation between social and technology-specific variables. The model is tested using survey 5 Refers to significance at least at the p=0.1 level. INTRODUCTION 13 responses of 98 IT decision makers. The results show that the identified variables are able to explain 50.4% of the variance in subscription renewal intention. Only information quality does not significantly impact subscription renewal intention. Foremost, I have to thank Prof. Dr. Saonee Sarker for the discussions and revisions about the way of consistently framing the research model. I also have to thank Dr. Darshana Sedera for his great discussions to focus my research on the central concept of subscription renewal, and not to focus on the theoretical integration of IS success and technology continuance. Finally, I want to thank Prof. Dr. Torsten Eymann to help me outline the practical relevance and the practical contributions section. The fourth paper investigates the following research question: Which information systems success dimensions influence the subscription renewal intention of the strategic and management cohort and are there significant differences between both? Therefore, the impact of information quality, system quality, and net benefits on subscription renewal intention is tested, using two distinct samples of top managers (strategic cohort) and IT executives (management cohort). The model is tested using small sample sizes of 43 for the strategic cohort and 33 for the management cohort. In contrary to our prediction, system quality contributes most to the explanation of continuance intention for the strategic cohort, whereas information quality explains most of the variance in the dependent variable concerning the management cohort. There is also a significant difference between the two cohorts in the impact of information quality on continuance intention. I thank Prof. Dr. Saonee Sarker for the discussions about how to frame the research model and the research design, Dr. Darshana Sedera for his guidance on inter-cohort analyses, as well as Prof. Dr. Boris Otto and Dr. Philipp Wunderlich for their thoughtful revisions and contributions about the practical relevance and application scenarios. The fifth paper investigates the research question: What factors influence the organizational level continuance of cloud-based ES? In this effort, a research model is tested, which is adapted from the organizational level discontinuance framework developed by Furneaux and Wade (2011), which includes “continuance forces” 6 (the information systems success variables (Delone and McLean 2003)) and “continuance inertia” (system investment and technical integration (Furneaux and Wade 2011)). In contrary to the previous chapters, the hypotheses development, and therefore the research model, is based on organizational level mechanisms to avoid the “mixed level fallacy”. This results in the cancellation of individual 6 “Continuance forces” are intended to capture the opposite of “change forces“. INTRODUCTION 14 level variables as per Chapters 2 and 4. The developed research model is tested using the formative scale developed in Chapter 3, as well as well-validated reflective scales (Furneaux and Wade 2011), using 115 survey responses of IT decision makers. The identified variables are able to explain 55.9 % of the variance in continuance intention. System quality has the highest positive effect on the dependent variable, whereas information quality has no significant effect on continuance intention. Surprisingly, in contrast to hypotheses development, technical integration has a significant, negative effect on continuance intention. I thank Prof. Dr. Saonee Sarker for the discussions on how to interpret the negative impact of technical integration, Dr. Darshana Sedera for his guidance on ES success measures, as well as Prof. Dr. Torsten Eymann and Prof. Dr. Boris Otto for their thoughtful revisions and contributions about the practical relevance and application scenarios. Figure 3. Paper Summary Operational Cloud-Based ES Success (Paper 2, Chapter 3) •Development of formative measurement instrument •Quantitative assessment of measurement instrument •Re-specification of primary success constructs Operational Cloud-Based ES and Confirmation (Paper 1, Chapter 2) •Development of conceptual model •Individual level of analysis •Theoretical integration of expectancy confirmation model (Bhattacherjee 2001) with IS success model (DeLone and McLean 2003) Subscription Renewal of Cloud-Based ES (Paper 3, Chapter 4) •Quantitative exploration of variables identified in conceptual model •Individual level of analysis •Variables drawn from social psychology (e.g. Ajzen 1991; Oliver 1980), IS success model (DeLone and McLean 2003) and discontinuance framework (Furneaux and Wade 2011) Operational Cloud-Based ES –Stakeholder Perspectives (Paper 4, Chapter 5) •Quantitative exploration of influence of distinct success variables on continuance intention of the strategic and management cohorts •Individual level of analysis •Variables drawn from IS success model (DeLone and McLean 2003) Continuance of Cloud-Based ES (Paper 5, Chapter 6) •Quantitative assessment of influence of continuance forces and continuance inertia on continuance intention •Organizational level of analysis •Continuance forces measured formatively •Variables drawn from IS success model (DeLone and McLean 2003) and discontinuance framework (Furneaux and Wade 2011) INTRODUCTION 15 REFERENCES Ajzen, I. 1991. “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes (50:2), pp. 179–211. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. 2010. “A View of Cloud Computing,” Communications of the ACM (53:4), pp. 50–58. Benlian, A., Koufaris, M., and Hess, T. 2011. “Service Quality in Software-as-a-Service: Developing the SaaS-Qual Measure and Examining Its Role in Usage Continuance,” Journal of Management Information Systems (28:3), pp. 85–126. Bhattacherjee, A. 2001. “Understanding Information Systems Continuance: An ExpectationConfirmation Model,” MIS Quarterly (25:3), pp. 351–370. Delone, W., and McLean, E. 2003. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems (19:4), pp. 9–30. Diamantopoulos, A., and Winklhofer, H. M. 2001. “Index Construction with Formative Indicators: An Alternative to Scale Development,” Journal of Marketing Research (38:2), pp. 269–277. Dörr, S., Walther, S., and Eymann, T. 2013. “Information Systems Success - A Quantitative Literature Review and Comparison,” In Proceedings of the 11th International Conference on Wirtschaftsinformatik, Leipzig. Furneaux, B., and Wade, M. 2011. “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly (35:3), pp. 573–598. Gable, G. G., Sedera, D., and Chan, T. 2008. “Re-Conceptualizing Information System Success : The IS-Impact Measurement Model,” Journal of the AIS (9:7), pp. 377–408. Gartner. 2012. “Forecast: Software as a Service, All Regions, 2010-2015, 1H12 Update,”Stamford. Gefen, D., Rigdon, E. E., and Straub, D. W. 2011. “An Update and Extension to SEM Guidelines for Administrative and Social Science Research,” MIS Quarterly (35:2), pp. iii–xiv. Janssen, M., and Joha, A. 2011. “Challenges for Adopting Cloud-Based Software as a Service (SAAS) in the Public Sector,” In ECIS 2011 Proceedings. Klaus, H., Rosemann, M., and Gable, G. G. 2000. “What is ERP?,” Information Systems Frontiers (2:2), pp. 141–162. Mathieson, K., Peacock, E., and Chin, W. W. 2001. “Extending the Technology Acceptance Model: The Influence of Perceived User Resources,” ACM SIGMIS Database (32:3), pp. 86. Mell, P., and Grance, T. 2009. “The NIST Defintion of Cloud Computing,”. Miles, M. B., and Huberman, A. M. 1994. Qualitative Data Analysis: An Expanded Sourcebook, (2nd ed, )Thousand Oaks: Sage. Moore, G. C., and Benbasat, I. 1991. “Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation,” Information Systems Research (2:3), pp. 192–222. Petter, S., DeLone, W., and McLean, E. 2008. “Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships,” European Journal of Information Systems (17), pp. 236–263. Petter, S., Straub, D., and Rai, A. 2007. “Specifying Formative Constructs in Information Systems Research,” MIS Quarterly (31:4), pp. 623–656. Ringle, C., Wende, S., and Will, A. 2005. “SmartPLS 2.0 M3,”. Rousseau, D. 1985. “Issues of Level in Organizational Research: Multi-Level and Cross INTRODUCTION 16 Level Perspectives,” Research in Organizational Behaviour (7), pp. 1–37. Salleh, S., Teoh, S., and Chan, C. 2012. “Cloud Enterprise Systems: A Review of Literature and Adoption,” In Proceedings of the 16th Pacific Asia Conference on Information Systems. Straub, D., Boudreau, M.-C., and Gefen, D. 2004. “Validation Guidelines for IS Positivist Research,” Communications of the AIS (13:24), pp. 380–427. Susarla, A., Barua, A., and Whinston, A. B. 2009. “A Transaction Cost Perspective of the ‘Software as a Service’ Business Model,” Journal of Management Information Systems (26:2), pp. 205–240. Urbach, N., Smolnik, S., and Riempp, G. 2009. “The State of Research on Information Systems Success – A Review of Existing Multidimensional Approaches,” Business & Information Systems Engineering (1:4), pp. 315 – 325. Venkatesh, V., Brown, S., and Bala, H. 2013. “Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems,” MIS Quarterly (37:1), pp. 21–54. Walther, S., Plank, A., Eymann, T., Singh, N., and Phadke, G. 2012. “Success Factors and Value Propositions of Software as a Service Providers - A Literature Review and Classification,” In Proceedings of the 18th Americas’ Conference on Information Systems. Wieneke, A., Walther, S., Eichin, R., and Eymann, T. 2013. “Erfolgsfaktoren von OnDemand-Enterprise-Systemen aus der Sicht des Anbieters - Eine explorative Studie,” In Proceedings of the 11th International Conference on Wirtschaftsinformatik, Leipzig. Wixom, B. H., and Todd, P. A. 2005. “A Theoretical Integration of User Satisfaction and Technology Acceptance,” Information Systems Research (16:1), pp. 85–102. Wunderlich, P., Kranz, J., Totzek, D., Veit, D., and Picot, A. 2013. “The Impact of Endogenous Motivations on Adoption of IT-Enabled Services: The Case of Transformative Services in the Energy Sector,” Journal of Service Research. CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 17 CHAPTER II: CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION The Role of Confirmation on IS Continuance Intention in the Context of On-Demand Enterprise Systems in the Post-Acceptance Phase Authors: Walther, Sebastian, University of Bayreuth, Germany Eymann, Torsten, University of Bayreuth, Germany Published in: Proceedings of the Eighteenth Americas Conference on Information Systems 1 (AMCIS), Seattle, United States, August 9-12, 2012 ABSTRACT The research project examines expectations as well as organizational and technological cognitive beliefs influencing a company’s intention to continue using on-demand enterprise systems in the post-acceptance phase. Expectation-confirmation theory from behavior literature is integrated with Delone and McLean’s model of IS success to theorize a model of IS continuance on company level. The decision making process to continue using an information system in small and middle enterprises as main target customer group of cloudbased enterprise systems is modeled by re-introducing the attitude construct from adoption literature. Additionally, post-purchase expectations are included as influence factor of attitude and intention in the continuance context. To prevent cloud-washing, attention is drawn to the substantive differences between service and application quality of on-demand enterprise systems. Keywords: Software as a Service, SaaS, Cloud-Computing, Expectancy-Confirmation Theory, IS Continuance, Delone and McLean. 1 The conference proceedings are ranked B in the WI-Orientierungsliste and D in the VHB-Jourqual ranking. CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 24 Figure 1. A Post-Acceptance Model of IS Continuance (Bhattacherjee 2001) 2.3. DeLone and McLean Model of IS Success The D&M IS success model (Delone and McLean 2003) is the most frequently used framework to structure IS success in the IS discipline (Urbach et al. 2009). The D&M model is a process model, which explains IS success starting from technical delivery to concepts focusing on individual and organizational benefits. It includes no overarching measure of success. Instead it provides a set of success categories and interdependencies between each. The six core components are information, system and service quality, intention to use, user satisfaction and net benefits (see Figure 2). In the following the relevant will be shortly introduced (Petter et al. 2008). System quality is the “desirable characteristics of an information system” like ease of use, system flexibility and system reliability. Information quality is the “desirable characteristics of the system outputs” like relevance, understandability and accuracy. Service quality is the helpdesk quality. Net benefits is the degree to which IS contributes to the success of the stakeholders like cost savings and productivity improvements. Continuation is not included into the D&M model. The reason for this is the conceptual gap which can be found in the subsequent differentiation between IS success from a customer’s and vendor’s perspective. While the D&M model of IS success focuses on the customer perspective on individual and organizational level, IS continuance is of importance from a vendor’s perspective. Perceived usefulness Satisfaction Confirmation IS continuance intention + + + + + Figure 1. A Post-Acceptance Model of IS Continuance CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 25 Figure 2. Updated D&M IS Success Model (Delone and McLean 2003) INFORMATION QUALITY SYSTEM QUALITY SERVICE QUALITY INTENTION TO USE USER SATISFACTION NET BENEFITS USE Figure 2. Updated D&M IS Success Model CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 26 3. RESEARCH MODEL The research question imposed was as follows: “Which role plays confirmation in the continuation of an on-demand enterprise system in the post-acceptance phase”? According to the requirements of an on-demand specific continuation framework, which were sketched in the motivational chapter, the post-acceptance model of IS continuance (Bhattacherjee 2001) was selected as fitting best. However, several modifications have to be conducted. Beneath the “model fit” the selected model offers additional benefits. First, it captures initial expectations indirectly; therefore a temporarily divided surveying process is obsolete. Secondly, it introduces cognitive beliefs and therefore integrates TAM and ECT. Third, the satisfaction construct 2 captures the unique decision making process in small and middle enterprises (SME) (Haddara and Zach 2011). SME can be seen as the primary customer group of on-demand enterprise applications. In the case of SME usually a small number of executives decide to continue or discontinue the use of an enterprise application. This leads to a decision making process which is more dependent on the individual. As previously stated, a continuance model for SaaS has to capture several on-demand specific considerations which are not captured by the initial framework: First, exploratory interviews with senior executives from software vendor SAP and a literature review on on-demand application success showed that the success factors of ondemand applications could be categorized according to the D&M success dimensions system, information and service quality, as well as net benefits. Hence these constructs are introduced and modified. The net benefits construct is re-named to organizational benefits to highlight the importance on company level. Additionally, the system, information and service quality are subsumed in a higher-order construct “technological quality”. In TAM, technological quality (analogous output quality) can be seen as a cognitive belief-influencing attitude and perceived usefulness (Venkatesh et al. 2003). Second, the shift from on-premise to on-demand has often been called a transformation from product to service. Therefore the term service quality in the D&M model is misleading, as it might be interpreted as the service delivery process of on-demand applications. Accordingly the service quality from the D&M model is renamed to helpdesk quality. Discussion revealed a major confusion about on-demand specific technical benefits. For instance intuitive user 2 Organizational continuance research usually doesn‘t include the satisfaction construct as the decision making process is more complex than in SME. CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 27 interface was categorized as benefit of on-demand applications. However, from a technical viewpoint, the user interface can also be implemented identically in an on-premise solution. To distinguish this clearly, the technological quality is split into service and application quality. Service quality includes all dimensions of the application delivery process, like availability. Contrary, application quality captures factors, which are not cloud-specific. This includes SOA-paradigm based system characteristics like extensibility, which cannot particularly be seen as technical benefit of the on-demand paradigm. Service and application quality are subdivided into the D&M success categories according to the previous point. Third, perceived usefulness is replaced by organizational benefits. Perceived usefulness was defined as cognitive belief salient to IS use. In TAM, perceived usefulness is defined as the belief of the individual user how useful a system is (Davis 1989). For instance: enhancing productivity, improving managing skills and performance. Applied on the organizational context organizational benefit is defined as the belief to which degree the information system supports the organizational goals. This is consistent with the definition of the net benefits on organizational level (Petter et al. 2008). Fourth, the organizational benefits-satisfaction relationship has been empirically tested to be insignificant (Sabherwal et al. 2006) and is therefore removed from the model. Finally, the constructs system investment and technical integration are included as additional continuation inertia (Furneaux and Wade 2011). Technical integration has been empirically shown to influence continuation. System investment had only little influence in the lateadoption phase. However, it might be important in the early adoption phase, as it is more difficult to argument for discontinuation in an early adoption phase if investments were high. Confirmation is defined as the user’s perception of the congruence between expectation and its actual performance (Patterson et al. 1997). We define technological quality as perceived technological performance, which means the different evaluations on the same stimulus (Spreng et al. 1996). As confirmation is defined as degree to which (pre-purchase) expectations are met by actual performance, a higher performance should result in a smaller gap between expectations and performance, followed by a higher confirmation. This leads to the first proposition: P1. Executives’ perceived technological quality is positively associated with their extent of confirmation. Empirical evidence has shown that cognitive beliefs like confirmation and perceived CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 28 usefulness (Bhattacherjee 2001) can be related similarly to ease of use and perceived usefulness (Davis et al. 1989). Theoretical support can be found in cognitive dissonance theory (Festinger 1957) where cognitive dissonance arises, when pre-acceptance usefulness perceptions are disconfirmed. Users might then try to minimize this dissonance by modifying their usefulness perceptions towards reality. Hence, a high confirmation will elevate users’ perceptions of organizational benefits and vice versa: P2. Executives’ extent of confirmation is positively associated with their beliefs about the organizational benefits. There is moderate empirical evidence that the dimensions of technological quality are positively related to the organizational benefits construct (Petter et al. 2008). Explanation for this relationship can be found in the D&M success model (Delone and McLean 2003), which describes IS success as process where the technological quality represents the foundation on which organizational value can be realized. This leads to following proposition: P3. Executives’ perceived technological quality is positively associated with their beliefs about the organizational benefits. Satisfaction is defined as an affective state that is emotional reaction to a product or service experience (Oliver 1980; Spreng et al. 1996). Per ECT, confirmation is an antecedent of satisfaction. From a pre-purchase perspective high confirmation is associated with the realization of benefits, which were expected. Contrary, the lack of confirmation is associated with failure of the consumed service or product. The confirmation-satisfaction has been empirically validated in IS and other industries. Hence: P4. Executives’ extent of confirmation is positively associated with their satisfaction. Continuance intention is defined as the intention to continue using the enterprise application (Bhattacherjee 2001; Mathieson 1991). Per TAM (Davis 1989) beliefs are direct and indirect predictors of intentions as enhanced organizational performance is coupled to several extrinsic and intrinsic rewards for the responsible IS executive like promotions, monetary gains and reputations (Vroom 1995). Therefore, IS being an instrument to support these goals high organizational benefits are likely to strengthen continuation intention. The organizational benefits-continuation context has been empirically validated in IS showing a significant correlation (Sabherwal et al. 2006). Hence: P5. Executives’ beliefs about the organizational benefits are positively associated with CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 29 their continuation intentions. Satisfaction is an emotional state, which is related to a perceived product or service quality. Therefore a better technological quality is likely to raise satisfaction. There is strong empirical support for following proposition (Petter et al. 2008): P6. Executives’ perceived technological quality is positively associated with their satisfaction. Per ECT, users’ primary predictor of continuation intention is satisfaction. Satisfaction is an affect, which is captured as positive or negative feeling. According to the theory of reasoned action, a positive affect leads to continuation intention while dissatisfaction is followed by discontinuation (Ajzen 1991). This leads to the seventh proposition: P7. Executives’ satisfaction is positively associated with their continuation intentions. System investment is defined as “the financial and other resources committed to the acquisition, implementation and use of an information system” (Furneaux and Wade 2011). System investment is especially important, as the discontinuance of an existing system in an adoption phase would signal a “loss” of sunk costs. This effect is based on the effect of sunk costs, where executives continue making resource commitments even though discontinuance would make sense from a rational viewpoint (Arkes and Blumer 1985). While system investment might have negative impact on discontinuance intention, the theory of sunk costs is also applicable vice versa: P8. Organizations’ system investment is positively associated with their continuation intentions. System embeddedness is defined as technical integration or “the extent to which an information system relies on sophisticated linkages among component elements to deliver needed capabilities”. Substantial integration of information systems into the organization increases the probability of difficulties when switching an information system. This relationship has been empirically validated to have negative influence on discontinuance of information systems (Furneaux and Wade 2011). Hence: P9. Higher levels of technical integration are positively associated with executives’ continuance intentions. Figure 3 summarizes the constructs and hypotheses. CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 30 Figure 3. A Continuance Model for On-Demand Enterprise Systems Organizational Benefits Continuance Intention Continuance Decision Confirmation Satisfaction Figure 3. A Continuance Model for On-Demand Enterprise Applications P2 (+) Dotted Lines –not investigated P1 (+) P3 (+) P4 (+) P5 (+) P6 (+) P7 (+) System Investment Technical Integration P8 (+) P9 (+) Service Quality Application Quality Technological Quality CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 31 4. CONCLUSION The paper summarized the authors’ state of work. This was done in three steps. First, the research question and its relevance to the IS discipline were explained. Therefore, literature in the field of SaaS adoption, continuance, expectations and success was illustrated. Based on the relevant literature, research gaps were highlighted. Second, relevant theories to study the concept of confirmation were introduced. Third, the research model with its hypotheses and constructs was sketched. This was done by linking SaaS-specific considerations with general theory to balance external and internal validity. Next steps in the research project will include operationalization of the relevant constructs and creation of the survey. During the surveying process, customers of SAP By Design will be contacted. According to the gained sample size, the data analysis method eventually has to be modified and small sample strategies have to be applied, including simulation methods. The study has several flaws which are mainly of theoretical nature. The theoretical problems arise when introducing the belief-attitude-intention chain into the organizational decision processes. In big companies, the decision process is highly structured with many cost calculations and strategic considerations. Especially the attitude then recesses as it is formalized in TAM2 and TAM3 (e.g. Venkatesh et al. 2003). The more a cognitive decision process is made consciously, like information-based decisions, the less it is based on attitude. However, as the decision process is made in a SME, it is likely, that attitude might be a significant influence factor of continuation intention. Theory has used both perspectives on decision making, however, data will show if it holds true in the special case of SaaS in the context of SME. In this point it still has to be discussed, if an exploratory-empirical approach would be more adequate. Another theory-driven problem is the conceptualization and operationalization of the perceived usefulness construct which is lifted into an organizational concept in the proposed model. Literature doesn’t provide any hints on the perceived usefulness construct on organizational level. This is an essential problem, as it can lead to wrong implications if continuation is investigated in an organizational context but operationalized from an individual perspective. CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 32 REFERENCES Ajzen, I. 1991. “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes (50:2), pp. 179–211. Ajzen, I., and Fishbein, M. 1980. Understanding Attitudes and Predicting Social Behaviour, Englewood Cliffs, New Jersey: Prentice-Hall. Anderson, E. W., Fornell, C., and Rust, R. T. 1997. “Customer Satisfaction, Productivity, and Profitability: Differences between Goods and Services,” Marketing Science (16:2), pp. 129–145. Anderson, E. W., and Sullivan, M. W. 1993. “The Antecedents and Consequences of Customer Satisfaction for Firms,” Marketing Science (12), pp. 125–143. Arkes, H. R., and Blumer, C. 1985. “The Psychology of Sunk Cost,” Organizational Behavior and Human Decision Processes (35:1), pp. 124–140. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. 2010. “A View of Cloud Computing,” Communications of the ACM (53:4), pp. 50–58. Benlian, A. 2009. “A Transaction Cost Theoretical Analysis of Software-as-a-Service (SAAS)-Based Sourcing in SMBs and Enterprises,” In Proceedings of the 17th European Conference on Information Systems. Benlian, A., and Hess, T. 2010. “The Risks of Sourcing Software as a Service – An Empirical Analysis of Adopters and Non-Adopters,” In Proceedings of the 18th European Confence on Information Systems. Benlian, A., Hess, T., and Buxmann, P. 2009. “Drivers of SaaS-Adoption - An Empirical Study of Different Application Types,” Business & Information Systems Engineering (1:5), pp. 357–369. Benlian, A., Koufaris, M., and Hess, T. 2010. “The Role of SaaS Service Quality for Continued SaaS Use: Empirical Insights From SaaS Using Firms,” In Proceedings of the 31st International Conference on Information Systems. Bhattacherjee, A. 2001. “Understanding Information Systems Continuance: An ExpectationConfirmation Model,” MIS Quarterly (25:3), pp. 351–370. Chau, P. Y. K., and Tam, K. Y. 2000. “Organizational Adoption of Open Systems: A ‘Technology-Push, Need-Pull’ Perspective,” Information & Management (37:5), pp. 229–239. Davis, F. D. 1989. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly (13:3), pp. 319. Davis, F. D., Bagozzi, R. P., and Warshaw, P. R. 1989. “User Acceptance of Computer Technology: A Comparison of Two Theoretical Models,” Management Science (35:8), pp. 982–1003. Delone, W., and McLean, E. 2003. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems (19:4), pp. 9–30. Festinger, L. 1957. A Theory of Cognitive Dissonance, New York, NY, USA: Harper. Furneaux, B., and Wade, M. 2011. “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly (35:3), pp. 573–598. Gable, G. G., Sedera, D., and Chan, T. 2008. “Re-Conceptualizing Information System Success : The IS-Impact Measurement Model,” Journal of the AIS (9:7), pp. 377–408. Haddara, M., and Zach, O. 2011. “ERP Systems in SMEs: A Literature Review,” In Proceedings of the 44th Hawaii International Conference on System Sciences, pp. 1–10. Heart, T. 2010. “Who is out there? Exploring the Effects of Trust and Perceived Risk on SaaS Adoption Intentions,” ACM SIGMIS Database (41:3), pp. 49. CLOUD ENTERPRISE SYSTEMS AND CONFIRMATION 33 Heinle, C., and Strebel, J. 2010. “IaaS Adoption Determinants in Enterprises,” In Economics of Grids, Clouds, Systems, and Services, J. Altmann and O. F. Rana (eds.), Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 93–104. Heskett, J. 1997. The Service Profit Chain :How Leading Companies Link Profit and Growth to Loyalty, Satisfaction, and Value, New York: Free Press. Janssen, M., and Joha, A. 2011. “Challenges for Adopting Cloud-Based Software as a Service (SAAS) in the Public Sector,” In Proceedings of the 19th European Conference on Information Systems. Kim, W., Kim, S. D., Lee, E., and Lee, S. 2009. “Adoption Issues for Cloud Computing,” In ICIIWAS 2009 Proceedings, pp. 3–6. Limam, N., and Boutaba, R. 2010. “Assessing Software Service Quality and Trustworthiness at Selection Time,” IEEE Transactions on Software Engineering (36:4), pp. 559–574. Martens, B., and Teutenberg, F. 2011. “Risk and Compliance Management for Cloud Computing Services: Designing a Reference Model,” In Proceedings of the 17th Americas’ Conference on Information Systems. Mathieson, K. 1991. “Predicting User Intentions: Comparing the Technology Acceptance Model with the Theory of Planned Behavior,” Information Systems Research (2:3), pp. 173–191. Matros, R., Rietze, C., and Eymann, T. 2010. “SaaS und Unternehmenserfolg: Erfolgskategorien für die Praxis,” In Software-as-a-Service, A. Benlian, T. Hess, and P. Buxmann (eds.), Wiesbaden: Gabler, pp. 239–254. Misra, S. C., and Mondal, A. 2011. “Identification of a Company’s Suitability for the Adoption of Cloud Computing and Modelling its Corresponding Return on Investment,” Mathematical and Computer Modelling (53:3-4), pp. 504–521. Oliver, R. L. 1980. “A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions,” Journal of Marketing Research (17:4), pp. 460–469. Patterson, P. G., Johnson, L. W., and Spreng, R. A. 1997. “Modeling the Determinants of Customer Satisfaction for Business-to-Business Professional Services,” Journal of the Academy of Marketing Science (25:1), pp. 4–17. Petter, S., DeLone, W., and McLean, E. 2008. “Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships,” European Journal of Information Systems (17), pp. 236–263. Sabherwal, R., Jeyaraj, A., and Chowa, C. 2006. “Information System Success: Individual and Organizational Determinants,” Management Science (52:12), pp. 1849–1864. Spreng, R. A., MacKenzie, S. B., and Olshavsky, R. W. 1996. “A Reexamination of the Determinants of Consumer Satisfaction,” The Journal of Marketing (60:3), pp. 15–32. Susarla, A., Barua, A., and Whinston, A. B. 2009. “A Transaction Cost Perspective of the ‘Software as a Service’ Business Model,” Journal of Management Information Systems (26:2), pp. 205–240. Taylor, S., and Todd, P. A. 1995. “Understanding Information Technology Usage: A Test of Competing Models,” Information Systems Research (6:2), pp. 144–176. Teo, H. H., Wei, K. K., and Benbasat, I. 2003. “Predicting Intention to Adopt Interorganizational Linkages: An Institutional Perspective,” MIS Quarterly (27:1), pp. 19–49. Urbach, N., Smolnik, S., and Riempp, G. 2009. “The State of Research on Information Systems Success – A Review of Existing Multidimensional Approaches,” Business & Information Systems Engineering (1:4), pp. 315 – 325. Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. 2003. “User Acceptance of Information Technology: Toward a Unified View,” MIS Quarterly (27:3), pp. 425–478. Vroom, V. 1995. Work and Motivation, (1st ed., )San Francisco: Jossey-Bass Publishers. CLOUD ENTERPRISE SYSTEMS SUCCESS 40 3. INSTRUMENT DEVELOPMENT PROCESS The instrument development procedure was conducted in a three stage approach according to Moore and Benbasat (1991) (see figure 1). To account for the unique characteristics of formative constructs, newer scale development elements were integrated (Diamantopoulos and Winklhofer 2001; MacKenzie et al. 2011; Petter et al. 2007), which explicitly focus on formative scale development. This included a clear conceptualization and content specification step, as well as formative-construct-specific scale purification and refinement. Figure 1. Scale Development Procedure 3.1. Conceptualization and Content Specification Diamantopoulos and Winklhofer (2001) explicitly highlight the need for a proper conceptualization concerning formative constructs, as “under formative measurement the latent variable is determined by its indicators rather than vice versa, content specification is inextricably linked with indicator specification”. MacKenzie (2003) further highlights the problems if constructs are not properly conceptualized: 1) confusion about similarities and differences of already existing constructs, 2) contamination in the sense that definitions overlap with other existing construct definitions, 3) invalid conclusions about relationships with other constructs. These problems are addressed by a conceptualization procedure proposed by MacKenzie et al. (2011), which is applied in a reduced form. The conceptual theme of the construct is defined by the dimensions which the construct covers. The dimensional analysis is split into three areas of concern: general IS success (mainly derived from Wixom and Todd (2005)), ES success (mainly derived from Gable et al. (2008)) and I: Item Creation Before Data Collection II: Scale Development III: Instrument Testing Conceptualization Content Specification Item Generation Access Content Validity Pretest and Refinement Evaluation of Formative Measurement Model and Re-Specification Field Test CLOUD ENTERPRISE SYSTEMS SUCCESS 41 SaaS success (newly developed). We did not explicitly study infrastructure as a service (IaaS) and platform as a service (PaaS) specific success factors as both, IaaS and PaaS, are hierarchically integrated into SaaS (Xu et al. 2010). Thus, SaaS usually incorporates components of PaaS and IaaS which are addressed here. However, it has to be noted that there might be differences in the importance of specific success factors for IaaS, PaaS and SaaS. For instance, user requirements might be more important for SaaS than for IaaS. After a thorough literature review 5 on SaaS benefits and success and the inclusion of existing success dimensions from general IS and ES literature, the first set included a total of 39 net benefits, 8 information quality and 21 system quality dimensions. The initial set of dimensions was reduced in a two-step approach by the first author. First, the dimensions were categorized according to IS success categories mostly based on existing literature (Gable et al. 2008; Urbach et al. 2010; Wixom and Todd 2005), and if newly created, according to the construct definitions (Delone and McLean 2003). Second, items were then culled or dropped if they seemed to be ambiguous, too narrow or not significant in the context of investigation. I.e. some of the benefits were interrelated with other success dimensions (e.g. the ubiquity of the system is related to mobility of the employees and to system accessibility) and it was individually decided whether all interrelated dimensions were included, or whether the interrelated dimensions exhaustively cover the content of similar dimensions to be culled or collapsed. Additionally, some SaaS-related benefits were relevant in the pre-adoption phase, however, felt to be non-appropriate in the post-acceptance phase (e.g. ease of implementation) and were therefore not included. Finally, some factors showed to be too narrow (e.g. system accuracy (Gable et al. 2008)) to be relevant from an organizational perspective, even though they could be relevant for other stakeholders. 5 The literature review (1/2000-5/2012) was conducted applying the search string “software as a service“ OR “SaaS“. The source selection was based on the Saunder’s AIS ranking (2012), including conferences like ECIS and ICIS. CLOUD ENTERPRISE SYSTEMS SUCCESS 42 Table 2. Success Dimensions (Final) 3.2. Item Generation After conceptualizing the constructs, an item pool was generated, which captured all aspects of the domain of the construct, while “minimizing the extent to which the items tap concepts outside of the domain of the focal construct” (MacKenzie et al. 2011). Diamantopoulos and Winklhofer (2001) note that for a formative specification, “the indicators must cover the entire scope of the latent variable, as defined in content specification”. MacKenzie et al. (2005) add that “dropping a measure from a formative-indicator model may omit a unique part of the conceptual domain and change the meaning of the variable, because the construct is a composite of all the indicators”. Hence, all dimensions of the formative constructs will influence the meaning of the variable. In contrary, keeping “irrelevant items” will not bias the results as we analyze data using PLS (Mathieson et al. 2001). In the interest of completeness, all identified dimensions were transformed into items. However, it has to be noted that in the Category Dimension ID Domain Source (s) (empirical & theoretical/conceptual) Reliability SQ1 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Flexibility SQ2 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Integration SQ3 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Accessibility SQ4 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Ease of Use SQ5 General e.g. Davis 1989; Wixom and Todd 2005 Response Time SQ6 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Functionalities SQ7 ES e.g. Urbach et al. 2010; Gable et al. 2008 Security SQ8 SaaS e.g. Benlian et al. 2011; Delone and Mclean 2004 Ease of Learning SQ9 ES Gable et al. 2008 User Requirements SQ10 ES Gable et al. 2008 Ease of Update SQ11 SaaS e.g. Campbell-Kelly 2009 Customization SQ12 SaaS/ES e.g. Benlian et al. 2009; Gable et al. 2008 Completeness IQ1 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Accuracy IQ2 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Format IQ3 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Currency IQ4 General e.g. Wixom and Todd 2005; Bailey and Pearson 1983 Relevance IQ5 General/ES e.g. Delone and McLean 2004; Gable et al. 2008 Understandability IQ6 General/ES e.g. Delone and McLean 2004; Gable et al. 2008 Productivity (individual) NB1 General e.g. Davis 1989; Gable et al. 2008 Productivity (organizational) NB2 General e.g. DeLone and McLean 2004, Gable et al. 2008 Decision Making NB3 ES Gable et al. 2008 Cost Savings NB4 All e.g. DeLone and McLean 2004, Gable et al. 2008 Better Planning NB5 SaaS e.g. Benlian et al. 2011 Strategic Flexibility NB6 SaaS e.g. Bibi et al. 2012; Benlian and Hess 2011 Mobility NB7 SaaS e.g. Bibi et al. 2012; Campbell-Kelly 2009 Innovation Ability NB8 SaaS e.g. Benlian et al. 2009; Bibi et al. 2012 Qual. Business Processes NB9 ES e.g. Urbach et al. 2010; Gable et al. 2008 IT-Risk Transfer NB10 SaaS Sarkar and Young 2011; Janssen and Joha 2011 Staff Requirements NB11 ES/SaaS e.g. Gable et al. 2008; Janssen and Joha 2011 Improved Outcomes/Outp. NB12 ES Gable et al. 2008 System Quality Information Quality Net Benefits CLOUD ENTERPRISE SYSTEMS SUCCESS 43 previous step (conceptualization and content specification) dimensions were already culled or collapsed. Items to measure the dimensions were taken from previously tested scales when possible and modified to fit into the context of research. Most items were newly created (see Table 5). This led to 14 net benefits, 12 system quality and 6 information quality indicators. 3.3. Assessing Content Validity Content validity is the “degree to which items in an instrument reflect the content universe to which the instrument will be generalized” (Straub et al. 2004). Examining content validity is a mandatory step for developing formative scales, as the constructs are directly defined by the dimensions. According to Petter et al. (2007) Q-Sorting can be one of the best methods to assure content validity for formative indicators. Consequently, we followed the same in this study. In the first sorting round the list of formative indicators and construct definitions, which were created in stage I, were given to one doctoral student, one regular student, one associate professor and one professor. The participants were told to read the construct definitions carefully, and then to map the items to the given categories. The results showed very heterogeneous results in the system quality and past experience category, which could be seen at the low overall (average) hit ratio of 0.67 and a Cohen’s Kappa (Cohen 1968) of 0.63. While there is no threshold level for appropriate levels of raw agreement and placement ratios, the metric helps to identify problematic areas. As the first round showed several flaws of the newly developed items, several items had to be re-written to achieve a better understandability. Strong analytical functions was culled after the first round as it was noted that it overlaps too strongly with information quality (cause) and better decision making (result). The results are shown in Table 3. In the second round four new judges were identified: one doctoral student, two associate professors and one professional. The procedure was executed analogously to the first time including the transformed items. In the second round, the overall hit ratio rose to 0.85. Cohen’s Kappa was clearly above the recommended threshold level of 0.65 (e.g. Todd and Benbasat). After the second round, two more items had to be changed (e.g. poor wording of risk transfer). CLOUD ENTERPRISE SYSTEMS SUCCESS 44 Table 3. Results of Q-Sorting 3.4. Pre-Test, Refinement, and Field Test The pre-test was an initial test of the overall instrument. The questionnaire was distributed to a heterogeneous sample of participants; including professors, associate professors, doctoral students, and employees of cloud ES providers, including sales and consulting which have direct customer access, and customers. The questionnaire was distributed as online-survey, of which 19 surveys were completed. Beneath each page a textbox was given, where the participants could comment on the questionnaire. Further, the participants were told to take the view of a customer when filling out the questionnaire. The goal of this test was to have a first feedback on wording, length, and instructions (Moore and Benbasat 1991). No quantitative pre-assessment was done to retain all dimensions within the field test. Questionnaire length was felt to be appropriate. A few changes were made on wording of the items and on the introductory text: for instance the introductory text was shortened and written more neutrally to limit priming effects, as well as “my cloud enterprise system” was re-worded to “our cloud enterprise system” to clarify the organizational character of the study. The second survey 6 was a “full scale” field test with the proposed target population of the study. The indicators were measured using Likert scales from “strongly disagree” (1) to “strongly agree (7) with the option to refuse to answer on single questions. The test was available as online questionnaire and as offline version. It was then distributed via several 6 “Low investment risk” was included within the full scale field test questionnaire, but excluded in the quantitative assessment, as it clearly belongs to the pre-adoption phase, leading to a total of 12 NB indicators. Agreement Measure Judges First Round Second Round Agreement Measure Judges First Round Second Round Agreement Measure Construct First Round Second Round Inter-Judge Agreement (Cohen's Kappa) A+B 0.56 0.84 Raw Agreement A+B 0.67 0.89 Placement Ratio Summary SQ 0.58 0.83 A+C 0.72 0.94 A+C 0.81 0.93 IQ 0.88 0.83 A+D 0.53 0.84 A+D 0.57 0.84 NB 0.7 0.85 B+C 0.72 0.88 B+C 0.78 0.87 PE 0.56 1.00 B+D 0.47 0.84 B+D 0.54 0.79 C+D 0.75 0.78 C+D 0.68 0.83 Ave. 0.63 0.85 Ave. 0.67 0.86 CLOUD ENTERPRISE SYSTEMS SUCCESS 45 media channels and by writing messages and E-Mails to top management, IT executives, line of business managers (LoB) (Martrain 2011), senior IT personnel and others (e.g. IT Strategy) with a total of 119 responses, of which 16 had to be dropped due to missing values (n=103). To cope with the problem that individuals report about company properties we applied the “key informant approach” (Segars and Grover 1998) by explicitly stating that the survey is focused on IT decision makers and by including a question into the survey whether the respondent is involved into the IT decision making process (i.e. which system is used). The sample characteristics are shown in Table 4. Due to the distribution via internet, no reliable estimations can be made how high the response rate was. To address the issue of response rate bias, we used a stratified sample of IT decision makers therefore limiting the possibility of non-response bias. Table 4. Sample Characteristics Position in Company % # Employees % System Age % Top Managememt 44 1-99 28 1-6 months 22 IT Executive 33 100-249 16 7-12 months 25 Line of Business Manager 15 250-499 28 13-18 months 33 Senior IS Personnel 5 500-999 14 18+ months 20 Others 3 1000+ 14 3.5. Quantitative Assessment of Measurement Instrument The evaluation of the formative measurement model was done in a three step procedure as proposed by Hair et al. (2013): 1) assess convergent validity of formative measurement model, 2) assess formative measurement model for collinearity issues, and 3) assess the significance and relevance of the formative indicators. Data was analysed using SmartPLS (Ringle et al. 2005) which is well suited for assessing formative indicators (Gefen et al. 2011; Hair et al. 2011) with parameter settings using 103 cases and 5000 samples (Hair et al. 2011). Missing values were replaced using the “mean replacement” algorithm provided by SmartPLS. First, convergent validity of the formative construct was assessed, which is described as “the extent to which a measure correlates positively with other measures of the same construct” (Hair et al. 2013). Convergent validity can be assessed by evaluating whether the formative construct is highly correlated to a reflective measure of the same construct. This method is known as redundancy analysis (Chin 1998). Information quality and net benefits showed an CLOUD ENTERPRISE SYSTEMS SUCCESS 46 adequate convergent validity with path strengths of 0.873 and 0.837 between the formative and reflective construct. System quality was slightly below the threshold level 0.8 (Chin 1998) with a value of 0.792. The reflective set of all constructs showed adequate convergent validity with loadings over 0.89 (Nunnally and Bernstein 1994). Second, multicollinearity of the formative indicators was assessed. This was done by using the SPSS software package within the linear regression module calculating the variance inflation factor (VIF). All VIFs were below the recommended threshold value of 5 (Hair et al. 2011) (see Table 5). Finally, the indicators were assessed for significance and relevance (see Table 5). Information quality showed 5 of 6 indicators to be significant at the p=0.1 level. Additionally, no sign changes of the weights were observed. Information quality showed desirable statistical properties and was therefore not re-specified. The higher the number of indicators within a formative construct, the more likely it is that indicators will be non-significant (Cenfetelli and Bassellier 2009). I.e. Mathieson et al. (2001) employ 7 formative indicators to measure perceived resources of which 4 are non-significant. As net benefits and system quality have 12 indicators each, it is not surprising that a large amount of indicators within these constructs were insignificant. Cenfetelli and Bassellier (2009) however note that this should not lead to the misinterpretation that the indicator is irrelevant. Rather it can only be interpreted that the indicator has a smaller influence than other indicators (weight). Another problem is the “cooccurrence of negative and positive indicator weights” (Cenfetelli and Bassellier 2009). This can happen when single indicators are more strongly correlated to other indicators than to the construct they measure. We addressed the problem of insignificant items and sign changes of system quality and net benefits in two steps. First we applied the procedure proposed by Hair et al. (2013) to drop insignificant indicators which do not contribute either relatively (significant weights) nor absolutely (outer loads) (Cenfetelli and Bassellier 2009). If the outer loading was greater than 0.5, the indicator was retained. If an outer loading was below 0.5, but was significant, it was individually evaluated whether an indicator would be dropped or not. In the case of insignificant outer loadings <0.5 the indicators were dropped. Second, we created second-order constructs of theoretical meaningful sub-categories (Cenfetelli and Bassellier 2009; Hair et al. 2011; Jarvis et al. 2003). After careful evaluation based on theoretical considerations NB5 (loading<0.5; significant) was dropped, as we felt that a good plan-ability of IT costs will only influence a very low number of people within the company (CFO, CIO) with limited overall impact on the organization, and will therefore remain insignificant across studies. Additionally, NB10 was dropped (loading<0.5; significant) as the CLOUD ENTERPRISE SYSTEMS SUCCESS 47 transfer of risk to the provider cannot be seen as a clear success measure itself, it can rather be seen as a reason why organizations use a system (i.e. the transference of operational and financial risk due to IT failures to a service provider). Hence, we argue that using this indicator to evaluate success can be misleading and therefore this measure was dropped. No system quality measures were dropped. IQ3 was not dropped, as the absolute contribution (loading) was greater than 0.5. Table 5. Test for Multicollinearity, Significance, and Contribution Step 2 and 3: Assessing Multicollinearity, Significance and Contribution VIF t-value weights loadings Our cloud enterprise system… NB1 … increases the productivity of end-users. 3.860 1.275 -0.181 0.498 NB2* … increases the overall productivity of the company. 3.284 1.949 0.253 0.618 NB3 … enables individual users to make better decisions. 2.442 0.133 -0.013 0.447 NB4* … helps to save IT-related costs. 2.049 1.811 0.242 0.663 NB5 … makes it easier to plan the IT costs of the company. 2.179 0.412 -0.041 0.359 NB6* … enhances our strategic flexibility. 3.622 2.107 0.278 0.854 NB7 … enhances the ability of the company to innovate. 3.466 1.591 0.219 0.802 NB8 … enhances the mobility of the company's employees. 2.314 0.744 -0.101 0.466 NB9* … improves the quality of the company's business processes. 2.059 2.295 0.267 0.612 NB10 … shifts the risks of IT failures from my company to the provider. 1.736 0.768 0.075 0.343 NB11 … lower the IT staff requirements within the company to keep the system running. 1.769 0.854 0.113 0.381 NB12* … improves outcomes/outputs of my company. 1.841 1.982 0.254 0.774 Net Benefits (reflective) (Adapted from Wixom and Watson (2001)) loadings NB13 … has changed my company significantly. 23.908 0.898 NB14 … has brought significant benefits to the company. 49.023 0.921 VIF t-value weights loadings Our cloud enterprise system… SQ1#* … operates reliabliy and stable. 1.501 2.991 0.339 0.569 SQ2# … can be flexibly adjusted to new demands or conditions. 3.051 1.645 -0.283 0.562 SQ3# … effectively integrates data from different areas of the company. 2.426 0.386 0.052 0.584 SQ4# … makes information easy to access (system accessibility). 1.976 0.061 -0.009 0.595 SQ5 … is easy to use. 2.325 0.923 0.116 0.646 SQ6# … provides information in a timely fashion (response time). 1.827 0.411 -0.041 0.515 SQ7* … provides key features and functionalities that meet the business requirements. 2.244 2.952 0.461 0.799 SQ8 … is secure. 1.274 0.111 -0.015 0.457 SQ9 … is easy to learn. 2.496 0.537 -0.071 0.448 SQ10 … meets different user requirements within the company. 2.143 1.539 0.185 0.607 SQ11 … is easy to upgrade from an older to a newer version. 1.569 1.274 0.148 0.622 SQ12* … is easy to customize (after implementation, e.g. user interface). 2.052 2.849 0.472 0.748 System Quality (reflective) (Adapted from Wixom and Todd (2005)) loadings SQ13 In terms of system quality, I would rate our cloud enterprise system highly. 134.603 0.969 SQ14 Overall, our cloud enterprise system is of high quality. 109.413 0.967 VIF t-value weights loadings Our cloud enterprise system… IQ1#* … provides a complete set of information. 2.765 3.168 0.329 0.873 IQ2#* … produces correct information. 2.207 1.782 0.16 0.758 IQ3# … provides information which is well formatted. 2.804 0.394 0.038 0.735 IQ4#* … provides me with the most recent information. 2.745 2.671 0.266 0.858 IQ5* … produces relevant information with limited unnecessary elements. 3.236 2.019 0.235 0.838 IQ6* … produces information which is easy to understand. 3.891 1.656 0.168 0.822 Information Quality (reflective) (Adapted from Wixom and Todd (2005)) loadings IQ7 Overall, I would give the information from our cloud enterprise system high marks. 66.711 0.953 IQ8 In general, our cloud enterprise system provides me with high-quality information. 53.37 0.947 # Wixom and Todd (2005); * significant at least at the p=0.1 level Net Benefits (formative) System Quality (formative) Information Quality (formative) CLOUD ENTERPRISE SYSTEMS SUCCESS 48 3.6. Re-Specification and Final Measurement Instrument The second step of managing the insignificant indicators and sign changes was to model net benefits and system quality as second-order constructs (Type III, see Ringle et al. 2012, Appendix B). Net benefits was re-modelled according to the original IS success model (DeLone and McLean 1992) where net benefits includes individual and organizational impact. Figure 2. Re-Specified Net Benefits System quality was split into five categories. Architecture agility captures the ease of interoperability and flexibility of the system. Architecture agility is one of the main goals of SOAs (Ren and Lyytinen 2008) and is therefore well suited to represent cloud system requirements. System performance captures the raw processing capacity and stability. Especially IT failures due to internet connectivity problems or general performance failures (Benlian and Hess 2011) are a main concern connected to cloud computing. Business requirements represent the requirements which are posed to a system to support the company in its business processes. Business requirements have been studied in different ESand SaaS-specific contexts (Benlian et al. 2011; Urbach et al. 2010). Ease of utilization represents the effort which has to be invested to learn, use and maintain the system. Due to the web-based interfaces, customercentric user scenarios and the automatic updates by the service providers, ease of utilization has been discussed as one of the main system benefits of SaaS. Finally, security, which represents the degree to which the system is protected against attacks (e.g. antivirus software) Net Benefits (2nd Order) Organizational Impact Individual Impact NB2: Productivity (organ.) NB9: Business Processes NB11: IT Staff Requirements NB4: Cost Savings NB6: Strategic Flexibility NB7: Innovation Ability NB12: Improved Outputs NB1: Productivity (indiv.) NB3: Better Decisions NB8: Mobility .242* 2.072 .515* 3.016 .393* 1.847 .773* 23.107 .266* 6.969 NB13 NB14 .870* 13.696 .943* 100.874 .277* 2.746 .097 0.938 .078 0.976 .121* 1.814 .171* 2.038 .472* 5.613 .181* 1.832 CLOUD ENTERPRISE SYSTEMS SUCCESS 49 has been shown to range among the main concerns in the context of SaaS (Benlian and Hess 2011). Figure 3. Re-Specified System Quality After the models were re-specified, SmartPLS was applied using the same parameter settings as in the previous chapter. In the final net benefits model two indicator’s weights were still insignificant (see Figure 2) with outer loadings above 0.5. Therefore, they were retained. No sign changes occurred. System quality showed desirable statistical properties with all indicators being significant at the p=0.1 level (see Figure 3), except for ease of learning, which was retained. Business Requirements System Quality (2nd Order) System Performance Architecture Agility Ease of Utilization SQ2: Flexibility SQ3: Integration SQ12: Customization SQ1: Reliability SQ11: Ease of Upgrade SQ6: Response Time SQ7: Functionalities SQ10: User Requirements SQ5: Ease of Use SQ9: Ease of Learning .422 2.441 .332 2.221 .405 3.168 .541 3.810 .660 4.815 .266 8.647 .234 7.350 .316 8.873 .285 9.458 SQ13 SQ14 .969 127.804 .967 117.547 SQ8: Security SQ4: Accessibility .714 7.118 .346 2.651 .435 4.082 .397 3.125 .151 0.995 .422 3.174 Security .170 4.129 1.000 0.000 SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 56 1. INTRODUCTION The emergence of cloud computing has significantly transformed the application of enterprise systems (ES) within organizations (Raihana 2012). Historically, ES have been implemented by large enterprises integrating different functional areas with the goal to provide a competitive advantage to the adopting organizations (Klaus et al. 2000) with only few small and medium enterprises (SMEs) being able to afford them (Raihana 2012). However, cloud computing has revolutionized how IT is used in enterprises, leading to the emergence of cloud enterprise systems (CES) like SAP By Design, Microsoft Netsuite or Salesforce.com. CES are a special form of software as a service (SaaS), which allow traditional ES to be presented in the cloud, making it an affordable, easy to implement and flexible software solution (Salleh et al. 2012). According to Gartner, in 2009, SaaS sales had reached $7.9 billion dollars, with approximately 65% of the sales attributed to CES. It is projected that by 2015 SaaS sales will reach the $21 billion mark (Gartner 2012). However, despite the economic relevance of CES and SaaS, there are innumerable stories about the difficulties that companies (e.g. Salesforce) have been facing in retaining their customers, and online tech bloggers have repeatedly emphasized the criticality of customer retention in the context of SaaS. Despite this acknowledgement within practice, only limited research has been done on the examination of the key antecedents of subscription renewal (in other words, customer continuance and retention) after the system has been implemented. This lack of research concerning the central concept of subscription renewal is even more surprising, as cloud computing has been labelled as “utility computing” on a commercial basis (Armbrust et al. 2010), where resources can be consumed “on-demand” with the (theoretical) possibility to immediately cancel the subscription if the service is erroneous (in contrast to classical IT outsourcing or licencebased on-premise ES). While this vision of “computing as a commodity” might already have become reality concerning infrastructure-services, IT decision makers might face severe problems when discontinuing or switching their CES, i.e., due to the large implementation costs or a SaaS vendor lock-in, which can apply when introducing a CES. Therefore in this study we investigate the following research question: “What factors influence the subscription renewal intention of CES adopters?” SaaS has seen a steadily growing body of research between 2007-2011 (Walther et al. 2012), with several theoretical and conceptual contributions concerning success, chances, risks and the adoption of SaaS. Many of the contributions are built on existing IT ousourcing literature, SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 57 where cloud computing can be seen as a special case of IT ousourcing. Prior to model development we conducted a thorough literature review based on Webster and Watson (2002) 2 (1/2000-5/2012) applying the search string “Software as a Service” OR “SaaS”. Results showed only a limited amount of research in and adjacent to SaaS continuation and adoption (see Table 1 3 ). Only one study was found empirically investigating the organizational continuation of SaaS (marked grey in Table 1) with focus on behavioural factors and the (technical) service quality. Overall, only two studies focused on continued SaaS use. In addition, whilst traditional ES have been explored to a large extent (Esteves and Bohoquez 2007; Esteves and Pastor 2001) cloud-based ES systems in general have received minimal attention. Existing ES publications have predominately focused on overall system success for traditional on-premise ES systems (e.g. Gable et al. 2008; Sedera and Gable 2010). In examining the research question, we focus on the beliefs, perceptions, and attitudes of the IT decision makers and their role towards subscription renewal intention. It has been widely acknowledged that IT-related decisions in an organization are typically made by the IT managers. This is especially true in the context of CES, whose primary customers are SMEs, and where typically a small group of executives has a decent power to decide whether an information system within their company is continued or discontinued (Premkumar 2003). Further evidence supporting this view is provided by Dibbern (2004), which found that sourcing and adoption decisions are mainly based on individuals and not made by organizations. In addition, tech blogs have also argued that decisions about whether or not to continue cloud solutions are made by single decision makers, such as line of business managers (Martrain 2011). Given the previously described organizational environment, where decisions about organizational IT artifacts are made by individual decision makers, we argue that the strongest way to predict the continuation intention of an organizational IT artefact is to build on cognitive and behavioral processes of individuals, more specifically, the IT decision makers. Hence, we assume that organizational change results out of the urge of the IT decision maker to stay consistent in his beliefs, attitudes and intentions (Fishbein and Ajzen 1975). Building on prior work which has made efforts to explain organizational adoption and continuance by focusing on the decision makers’ viewpoint (Benlian et al. 2009, 2 The selection of the sources was based on the Saunder’s AIS ranking (2012) up to position 25, including the AIS basket of 8 and major conferences like ICIS and ECIS. Additionally, reference lists of the extracted articles were screened. 3 Papers were only mentioned once if they were transitioned from conference to journal article. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 58 2011), we theorize and test a conceptual model, which takes a socio-technical approach (e.g. Bostrom and Heinen 1977), examining the effect of both - social-related and technologyspecific factors. Consequently, we have drawn on the IS success model for technologyspecific variables and its interaction with social elements, the expectation-confirmation theory (ECT) (Oliver 1980) from consumer satisfaction literature to represent social variables, as well as continuation inertia drawn from the existing literature on organizational system continuance (Furneaux and Wade 2011), which are social-related and technological variables. The rest of the manuscript is as follows. First, the theoretical framing is given. Second, the research model of subscription renewal of CES is presented. Third, the methodology is described. Fourth, the results are presented and subsequently discussed. x) Table 1. SaaS-Related Client-Side Adoption and Continuation Literature Authors/Paper IND ORGA ADOPT CONT CR TECH NB CI THEO EMP Xin and Levina 2008 X X X X X Benlian et al. 2009 X X X X X Susarla et al. 2009 X X X X X Heart 2010 X X X X X Yao et al. 2010 X X X X X X Benlian et al. 2011 X X X X X Benlian and Hess 2011* X X X X X X Janssen and Joha 2011 X X X X X Misra and Mondal 2011 X X X X Wang 2011 X X X X X Wu et al. 2011 X X X X X X SUM 011 10 2411 725 6 This Article X X X X X X X Type Legend: IND=Individual Level; ORGA=Organizational Level; ADOPT=Adoption; CONT= Continuation; CR=Continuance-related; TECH=Technological Quality; NB=Net Benefits; CI=Continuation Inertia; THEO=Theoretical/Conceptual; EMP=Empirical *Study investigates adopter's and non-adopter's intention to increase the level of sourcing, therefore it is categorized as adoption. Level of Analysis Adoption Phase Nature of Investigated Factors SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 59 2. THEORETICAL FRAMING In the following paragraphs, we outline different conceptual areas and their conceptual relevance in the context of CES. As outlined earlier, in examining our research question, we examine the role of the beliefs, attitudes, etc., of the individual decision makers with respect to CES continuation intention. 2.1. The Dependent Variable: System Continuation Literature on system continuance of individuals is mainly based on theories drawn from social psychology like the theory of reasoned action (TRA) (Fishbein and Ajzen 1975) or the theory of planned behavior (TPB) (Ajzen 1991) which focus on the prediction of human behavior. TRA has taken shape in IS in the form of the technology acceptance model (TAM) (Davis 1989). System continuance has often been studied in the context of adoption, but is not limited to it. For instance continuation has been studied to evaluate the post-implementation phase (Benlian et al. 2011), to evaluate the success of e-commerce systems (Wang 2008) and at the end of the lifecycle as discontinuance intention (Furneaux and Wade 2011). From marketing or business perspective, continuation is an indicator for customer retention. Complementary research has investigated the continuation of IS on organizational level, which has been mostly guided by the technology-organization-environment-framework (TOE) (Tornatzky and Fleischer 1990), and the diffusion of innovation theory (DOI) (Rogers 1983). In contrary to the system continuance of individuals, organizational adoption and continuation literature has focused on macro-factors like perceived benefits (Lee and Shim 2007), system performance shortcomings or environmental pressure (Chau and Tam 2000; Furneaux and Wade 2011; Teo et al. 2003), ignoring individual attitudes and cognitive effects (Premkumar 2003). 2.2. The Information Systems Success Model Even though research on IS success is a mature research stream, only a small number of studies have investigated the role of IS success on the continuation 4 of IS (Petter et al. 2008). We use the IS success model for four reasons. First, the success categories have been shown to adequately represent IS success in a variety of contexts such as e-Commerce (Wang 2008) or employee portals (Urbach et al. 2010). Second, the categories are comprehensive and easy to communicate. Third, it is the most widely used success measurement model (Urbach et al. 4 Several studies (Petter et al. 2008; Rai et al. 2002) have used the term use from the IS success model and continuation from ECT/TRA synonymously. However, to be consistent, we refer to continuation. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 60 2009), and therefore its application allows comparability and external validity among studies. Fourth, prior work has shown that the IS success model exhaustively captures SaaS-specific (Walther et al. 2012) and ES-specific (Gable et al. 2008) success factors. The revised IS success model (Delone and McLean 2003) consists of six interlinked success categories: system quality, information quality, service quality, user satisfaction, use and net benefits. In our study, in an effort to keep our model parsimonious, and in line with our socio-technical approach, we focus on the IS success variables which are primarily technology-focused; in other words, we examine the effect of system quality and information quality. In addition, we also examine the effect of using the CES in its organizational environment, which is represented by net benefits. 2.3. The Decision makers’ Cognitive and Affective Responses Premkumar (2003) has highlighted the lack of studies on factors specific to individual decision makers, especially in the context of small enterprises. The expectation confirmation theory (ECT) (Oliver 1980) is one of the predominant concepts in marketing and IS to study consumer satisfaction and customer loyalty, and is therefore well suited to study the effect of individuals’ perceptions in the post-acceptance phase. It has been empirically validated in several product and service continuance contexts (e.g. Patterson et al. 1997). The process by which consumers build repurchase intentions is as follows (Oliver 1980). Customers have (pre-purchase) expectations before consuming the service or product. Temporarily shifted, there is an initial consumption, which leads to a perception of the performance. This performance is then evaluated against the original expectations (confirmation). Based on their extent of confirmation, consumers form an attitude which then influences repurchase intentions. The expectation confirmation model (ECM) (Bhattacherjee 2001) focuses on postacceptance variables and modifies ECT in two dimensions. First, pre-purchase expectations are not included, as satisfaction and confirmation capture all effects of pre-acceptance variables. Second, perceived usefulness is introduced as post-consumption variable. It is noteworthy that the prominent extension of ECM by Bhattacherjee et al. (2008) replaces perceived usefulness by post-usage usefulness and introduces self-efficacy as antecedent of system continuance. According to Hossain and Quaddus (2012), recent research on system continuance in the context of ECM has focused on finding new independent variables influencing continuation intention. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 61 2.4. Social and Technological Commitment: Continuation Inertia In addition to socio-centric variables from marketing and social psychology, we also look at socio-centric and technological variables from organizational system continuance literature. Similar to perceived behavioral control from TPB, continuation inertia enforce behavioral persistence. In our model we focus on system investment and technical integration as organizational and technological commitment (Furneaux and Wade 2011). Both concepts are especially interesting in the context of CES for two reasons. First, flexibility has often been named as one of the major advantages of cloud computing (Armbrust et al. 2010). This includes technological flexibility, where the usage of service oriented architectures should enable a seamless integration and transfer of cloud services, reducing the technological complexity and sophistication of the ES. In contrary, ES are generally very complex IS, where, i.e., vendor lock-in can apply. Second, it has often been stated that one of the value propositions of cloud computing are “low up-front costs”. However, ES research has shown the implementation is one of the major cost drivers of ES, leading to the conclusion that system investments might also play a role in the context of CES. Therefore the exploratory result of both hypotheses can give further insights if cloud computing can generally be labelled as “utility computing”. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 62 3. HYPOTHESES DEVELOPMENT 3.1. IS Success and Subscription Renewal Intention Subscription renewal intention is defined as the intention to continue using the ES (Bhattacherjee 2001), where net benefits is the extent to which an IS is beneficial to the individuals, groups and organizations (Delone and McLean 2003). As Davis et al. (1989) note: “people form intentions towards behavior if they believe it will increase their job performance”. Therefore enhanced organizational performance enabled by the CES is coupled to several extrinsic and intrinsic rewards for the responsible IS executive like promotions, monetary gains and reputation (Vroom 1995). Hence, CES being an instrument to support these goals, high net benefits of the CES are likely to strengthen subscription renewal intention. The net benefits-continuation relationship has been empirically validated in the organizational IS context showing a positive correlation (Petter et al. 2008), but has not been studied in the context of SaaS. H1. IT decision makers’ beliefs about the net benefits are positively associated with CES subscription renewal intention. We define system quality as the degree, to which the system has desirable characteristics, whereas information quality is the desirable characteristic of system output (Delone and McLean 2003). As the system in the context of ES is usually designed to support business processes and organizational goals, analogously to H1, supporting these tasks by high system and information quality will lead to several extrinsic and intrinsic rewards, strengthening the intention to renew the subscription. The relationship between system quality and continuation has been theorized in the IS success model and has gathered mixed empirical support (Petter et al. 2008). Information quality and continuation have been tested to be positively correlated in an organizational context (Fitzgerald and Russo 2005), however the hypothesis still lacks further empirical support. Both hypotheses haven’t been tested in the context of SaaS. H2. IT decision makers’ perceived system quality is positively associated with CES subscription renewal intention. H3. IT decision makers’ perceived information quality is positively associated with CES subscription renewal intention. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 63 3.2. Cognitive and Affective Responses and Subscription Renewal Intention Conflicting conceptualizations of the satisfaction construct (Yi 1989) have made it difficult to compare results of user satisfaction and behavioral literature. Satisfaction was initially defined as “a pleasurable or positive emotional state resulting from the appraisal of one’s job” (Locke 1976). Even though attitude and satisfaction have been used synonymously in literature (LaTour and Peat 1979), both have to be seen as different concepts. Oliver (1980) argues that attitude is a more enduring affect incorporating all prior experiences, while satisfaction is a transient and experience-specific affect. Based on this, we use attitude instead of satisfaction, as it better suited for the research setting, as is not likely that an IT executive bases his decision on a “transient emotional state”. Therefore affect is conceptualized as attitude. From a pre-purchase perspective high confirmation is associated with the realization of a good performance. In contrary, the lack of confirmation is often associated with failure of the product or consumed service. There is strong evidence that attitude is a function of (dis-) confirmation (Oliver 1980). The relationship has been positively tested in an organizational SaaS context as affective response (Benlian et al. 2011). H4. IT decision makers’ extent of confirmation is positively associated with their attitude. Per expectancy-value theory (Fishbein and Ajzen 1975), external variables like system characteristics impact behavioral beliefs, which in turn influence the attitude towards performing the behavior. This attitude then affects the behavioral intention, which then ultimately impacts the behavior itself. Therefore a positive attitude towards using the CES will have positive influence on subscription renewal intention. The affect-continuation relationship has been positively validated in an organizational SaaS context (Benlian et al. 2011) and specifically as attitude in an organizational SaaS adoption context (Benlian et al. 2009). H5. IT decision makers’ attitude is positively associated with CES subscription renewal intention. Theoretical support for the relationship between confirmation and net benefits is found in cognitive dissonance theory (Festinger 1957), where cognitive dissonance arises, when two cognitions are contradictory. The executives might then try to reduce this dissonance by changing their net benefits perceptions towards conflicting cognitions like confirmation. The hypothesis has been empirically validated in the organizational SaaS context as perceived usefulness (Benlian et al. 2011), which can be interpreted as “individual impact” (Rai et al. 2002), hence a part of net benefits, but not specifically as net benefits-confirmation. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 64 H6. IT decision makers’ extent of confirmation is positively associated with their beliefs about the net benefits. 3.3. Continuation Inertia and Subscription Renewal Intention We define system investment as “the financial and other resources committed to the acquisition, implementation and use of an information system” (Furneaux and Wade 2011). System investment can be relevant, as the discontinuance of a running system in a postadoption phase would mark a “loss” aka sunk costs. The sunk cost effect has been thoroughly studied and describes the situation, where executives continue to make commitments of resources despite the fact that rationally seen discontinuance would make sense (Arkes and Blumer 1985). The relationship has been studied in the context of organizational replacement intention (which is the opposite of continuation intention) by Furneaux and Wade (2011), where it was insignificant, but not in the context of SaaS: H7. Higher system investments are positively associated with CES subscription renewal intention. Technical integration is defined as “the extent to which an information system relies on sophisticated linkages among component elements to deliver needed capabilities” (Furneaux and Wade 2011). Sophisticated integration of IS within the organization increases the probability of system shortcomings when switching an information system. Therefore, the executive might not discontinue the usage of a system due to the associated difficulties. This relationship has been empirically validated to have positive influence on continuation intention of IS (Furneaux and Wade 2011), but has not been empirically validated in the context of SaaS. H8. Higher extent of technical integration is positively associated with CES subscription renewal intention. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 65 Figure 1. Research Model Technical Information Quality (IS Success) System Quality (IS Success) Socio-Centric Confirmation (ECT) Attitude (TRA/ECT) System Investment (Continuation Inertia) Technical Integration (Continuation Inertia) Organizational Action Net Benefits (IS Success) t=0 t=1 SUBSCRIPTION RENEWAL INTENTION not tested H1 (+) H4 (+) H6 (+) H5 (+) H7 (+) H3 (+) H2 (+) H8 (+) SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 72 creating brand awareness or a well-managed customer relationship management. Additionally, the strong impact of confirmation shows that expectations might not be set too high, as they might then be disconfirmed. From a theoretical viewpoint, our study suggests that attitude is a significant predictor of subscription renewal intention making it necessary to re-think organizational system continuance in the context of CES (or generally). Continuation inertia showed to significantly influence subscription renewal intention. This is especially interesting in the context of cloud computing, as cloud computing has seen a strong labeling towards low up-front investments, system flexibility, low entrance barrier, etc. Our study opposes the generalizability of this view in the context of CES. Especially implementation and personnel training costs of CES are still substantial investments, posing severe barriers on changing or discontinuing a cloud service. Contrary to Furneaux and Wade (2011) we find a significant relationship of system investment, which might be due to the fact that we are looking at an earlier stage of the lifecycle. Cloud service providers should clarify the amount of implementation costs which are to be expected within the implementation phase to reduce frustration. Technical integration showed to have a negative impact on subscription renewal intention, contrary to our prediction. The reason for this can be that technical integration is no direct predictor of behavioral intention, but influences system satisfaction negatively as conceptualized by Wixom and Todd (2005). As we used PLS, another reason could be that other influences are relatively stronger. This work has several limitations which need to be discussed. First, it is important to highlight that our measurement was based on the view of individuals reporting about organizational properties and their affective and cognitive responses. It may thus be argued that the dependent variable in our model might be biased given that it reflects an individual perspective rather than a shared opinion within the organization. This problem has been highlighted by several prior studies (e.g. Benlian et al. 2011; Furneaux and Wade 2011) studying organizational system continuance. However, we believe that this problem is less severe in our study, as it is likely that in the context of CES, organizational system continuance is typically decided by an individual or a small number of individuals. Second, even though we were able to explain a decent portion of variance in the target construct, there might be factors which we did not include but are relevant (e.g. internal pressures). Even though risks have been often studied in the adoption phase, the novelty of cloud computing might also raise awareness after the system has been adopted (Benlian and Hess 2011). Therefore future research might draw on these variables. Third, as we conducted a cross-sectional study, we are not able to see how good our model tests actual behavior. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 73 Additionally, we draw the directions of our causalities from theoretical assumptions, which cannot be empirically validated. Therefore future research should include “hard data” to limit biases which are connected to survey methods. Finally, our sub-samples should consider different ES, firm sizes, implementation times and industries. Clustering these sub-samples might lead to more specific insights about specific industries or ES, i.e., the role of system investment for less complex ES. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 74 REFERENCES Ajzen, I. 1991. “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes (50:2), pp. 179–211. Arkes, H. R., and Blumer, C. 1985. “The Psychology of Sunk Cost,” Organizational Behavior and Human Decision Processes (35:1), pp. 124–140. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. 2010. “A View of Cloud Computing,” Communications of the ACM (53:4), pp. 50–58. Benlian, A., and Hess, T. 2011. “Opportunities and Risks of Software-as-a-Service: Findings from a Survey of IT Executives,” Decision Support Systems (52:1), pp. 232–246. Benlian, A., Hess, T., and Buxmann, P. 2009. “Drivers of SaaS-Adoption - An Empirical Study of Different Application Types,” Business & Information Systems Engineering (1:5), pp. 357–369. Benlian, A., Koufaris, M., and Hess, T. 2011. “Service Quality in Software-as-a-Service: Developing the SaaS-Qual Measure and Examining Its Role in Usage Continuance,” Journal of Management Information Systems (28:3), pp. 85–126. Bhattacherjee, A. 2001. “Understanding Information Systems Continuance: An ExpectationConfirmation Model,” MIS Quarterly (25:3), pp. 351–370. Bhattacherjee, A., Perols, J., and Sanford, C. 2008. “Information Technology Continuance: A Theoretical Extension and Empirical Test,” Journal of Computer Information Systems (49:1), pp. 17–26. Bostrom, R. P., and Heinen, J. S. 1977. “MIS Problems and Failures: A Socio-Technical Perspective, Part II: The Application of Socio-Technical Theory,” MIS Quarterly (1:4), pp. 11–28. Chau, P. Y. K., and Tam, K. Y. 2000. “Organizational Adoption of Open Systems: A ‘Technology-Push, Need-Pull’ Perspective,” Information & Management (37:5), pp. 229–239. Chin, W. W. 1998. “The Partial Least Squares Approach to Structural Equation Modeling,” In Modern Methods for Business Research, G. A. Marcoulides (ed.), Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 294–336. Chin, W. W. 2010. “How to Write Up and Report PLS Analyses,” In Handbook of Partial Least Squares, V. Esposito Vinzi, W. W. Chin, J. Henseler, and H. Wang (eds.), Springer Berlin Heidelberg, pp. 655–690. Chin, W. W., Marcolin, B. L., and Newsted, P. R. 2003. “A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic - Mail Emotion/Adoption Study,” Information Systems Research (14:2), pp. 189–217. Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences, Routledge Academic, pp. 567. Davis, F. D. 1989. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly (13:3), pp. 319. Davis, F. D., Bagozzi, R. P., and Warshaw, P. R. 1989. “User Acceptance of Computer Technology: A Comparison of Two Theoretical Models,” Management Science (35:8), pp. 982–1003. Delone, W., and McLean, E. 2003. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems (19:4), pp. 9–30. Dibbern, J. 2004. Sourcing of Application Software Services. Empirical Evidence of Cultural, Industry and Functional Differences, Heidelberg: Physica. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 75 Esteves, J. M., and Bohoquez, V. 2007. “An Updated ERP Systems Annotated Bibliography: 2001-2005,” Communications of the AIS (19:18), pp. 386–446. Esteves, J., and Pastor, J. 2001. “Enterprise Resource Planning Systems Research : An Annotated Bibliography,” Communications of the AIS (7:8). Festinger, L. 1957. A Theory of Cognitive Dissonance, New York, NY, USA: Harper. Fishbein, M., and Ajzen, I. 1975. Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research, Reading, MA: Addison-Wesley. Fitzgerald, G., and Russo, N. L. 2005. “The Turnaround of the London Ambulance Service Computer-Aided Despatch system (LASCAD),” European Journal of Information Systems (14:3), pp. 244–257. Fornell, C., and Larcker, D. F. 1981. “Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,” Journal of Marketing Research (18:1), pp. 39–50. Furneaux, B., and Wade, M. 2011. “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly (35:3), pp. 573–598. Gable, G. G., Sedera, D., and Chan, T. 2008. “Re-Conceptualizing Information System Success : The IS-Impact Measurement Model,” Journal of the AIS (9:7), pp. 377–408. Gartner. 2012. “Forecast: Software as a Service, All Regions, 2010-2015, 1H12 Update,”Stamford. Gefen, D., Rigdon, E. E., and Straub, D. W. 2011. “An Update and Extension to SEM Guidelines for Administrative and Social Science Research,” MIS Quarterly (35:2), pp. iii–xiv. Geisser, S. 1975. “The Predictive Sample Reuse Method with Application,” Journal of the American Statistical Association (70:350), pp. 320–328. Hair, J. F., Ringle, C. M., and Sarstedt, M. 2011. “PLS-SEM: Indeed a Silver Bullet,” The Journal of Marketing Theory and Practice (19:2), pp. 139–152. Heart, T. 2010. “Who is out there? Exploring the Effects of Trust and Perceived Risk on SaaS Adoption Intentions,” ACM SIGMIS Database (41:3), pp. 49. Hossain, M. A., and Quaddus, M. 2012. “Expectation–Confirmation Theory in Information System Research: A Review and Analysis,” In Information Systems Theory, Y. K. Dwivedi, M. R. Wade, and S. L. Schneberger (eds.), (Vol. 28)New York, NY: Springer New York, pp. 441–469. Hulland, J. 1999. “Use of Partial Least Squares (PLS) in Strategic Management Research: a Review of Four Recent Studies,” Strategic Management Journal (20:2), pp. 195–204. Janssen, M., and Joha, A. 2011. “Challenges For Adopting Cloud-Based Software as a Service (SaaS) in the Public Sector,” In Proceedings of the 19th European Conference on Information Systems. Klaus, H., Rosemann, M., and Gable, G. G. 2000. “What is ERP?,” Information Systems Frontiers (2:2), pp. 141–162. LaTour, S. A., and Peat, N. C. 1979. “Conceptual and Methodological Issues in Consumer Satisfaction Research,” Advances in Consumer Research, pp. 431–437. Lee, C.-P., and Shim, J. P. 2007. “An Exploratory Study of Radio Frequency Identification (RFID) Adoption in the Healthcare Industry,” European Journal of Information Systems (16:6), pp. 712–724. Locke, E. A. 1976. “The Nature and Causes of Job Satisfaction,” In Handbook of Industrial and Organizational PsychologyPalo Alto: Consulting Psychologists Press, pp. 1319– 1328. Martrain. 2011. “B2B Tech Marketing Shifts from IT to Line of Business Leaving Huge Data Gaps,”. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 76 Misra, S. C., and Mondal, A. 2011. “Identification of a Company’s Suitability for the Adoption of Cloud Computing and Modelling its Corresponding Return on Investment,” Mathematical and Computer Modelling (53:3-4), pp. 504–521. Nunnally, J. C., and Bernstein, I. H. 1994. Psychometric Theory, New York: McGraw-Hill. Oliver, R. L. 1980. “A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions,” Journal of Marketing Research (17:4), pp. 460–469. Patterson, P. G., Johnson, L. W., and Spreng, R. A. 1997. “Modeling the Determinants of Customer Satisfaction for Business-to-Business Professional Services,” Journal of the Academy of Marketing Science (25:1), pp. 4–17. Petter, S., DeLone, W., and McLean, E. 2008. “Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships,” European Journal of Information Systems (17), pp. 236–263. Premkumar, G. 2003. “A Meta-Analysis of Research on Information Technology Implementation in Small Business,” Journal of Organizational Computing and Electronic Commerce (13:2)Taylor & Francis, pp. 91–121. Rai, A., Lang, S. S., and Welker, R. B. 2002. “Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis,” Information Systems Research (13:1), pp. 50–69. Raihana, G. 2012. “Cloud ERP: A Solution Model,” International Journal of Computer Science and Information Technology & Security (2:1), pp. 76–79. Ringle, C., Wende, S., and Will, A. 2005. “SmartPLS 2.0 M3,”. Rogers, E. 1983. Diffusion of Innovations, (5 th, )Free Press. Salleh, S., Teoh, S., and Chan, C. 2012. “Cloud Enterprise Systems: A Review of Literature and Adoption,” In Proceedings of the 16th Pacific Asia Conference on Information Systems. Sedera, D., and Gable, G. 2010. “Knowledge Management Competence for Enterprise System Success,” The Journal of Strategic Information Systems (19:4), pp. 296–306. Segars, A. H., and Grover, V. 1998. “Strategic Information Systems Planning Success: An Investigation of the Construct and Its Measurement,” MIS Quarterly (22:2), pp. 139– 163. Susarla, A., Barua, A., and Whinston, A. B. 2009. “A Transaction Cost Perspective of the ‘Software as a Service’ Business Model,” Journal of Management Information Systems (26:2), pp. 205–240. Teo, H. H., Wei, K. K., and Benbasat, I. 2003. “Predicting Intention to Adopt Interorganizational Linkages: An Institutional Perspective,” MIS Quarterly (27:1), pp. 19–49. Tornatzky, L., and Fleischer, M. 1990. The Processes of Technological Innovation, Lexington, MA: Lexington Books. Urbach, N., Smolnik, S., and Riempp, G. 2009. “The State of Research on Information Systems Success – A Review of Existing Multidimensional Approaches,” Business & Information Systems Engineering (1:4), pp. 315 – 325. Urbach, N., Smolnik, S., and Riempp, G. 2010. “An Empirical Investigation of Employee Portal Success,” The Journal of Strategic Information Systems (19:3), pp. 184–206. Vroom, V. 1995. Work and Motivation, (1st ed., )San Francisco: Jossey-Bass Publishers. Walther, S., Plank, A., Eymann, T., Singh, N., and Phadke, G. 2012. “Success Factors and Value Propositions of Software as a Service Providers - A Literature Review and Classification,” In Proceedings of the 18th Americas’ Conference on Information Systems. Wang, Y.-H. 2011. “The Role of SaaS Privacy and Security Compliance for Continued SaaS Use,” In NCM 2011 Proceedings, pp. 303–306. SUBSCRIPTION RENEWAL OF CLOUD ENTERPRISE SYSTEMS 77 Wang, Y.-S. 2008. “Assessing e-Commerce Systems Success: a Respecification and Validation of the DeLone and McLean Model of IS success,” Information Systems Journal (18:5), pp. 529–557. Webster, J., and Watson, R. T. 2002. “Analyzing the Past to Prepare for the Future: Writing a Literature Review,” MIS Quarterly (26:2), pp. xiii–xxiii. Wixom, B. H., and Todd, P. A. 2005. “A Theoretical Integration of User Satisfaction and Technology Acceptance,” Information Systems Research (16:1), pp. 85–102. Wixom, B. H., and Watson, H. J. 2001. “An Empirical Investigation of the Factors Affecting Data Warehousing Success,” MIS Quarterly (25:1), pp. 17. Wu, W.-W., Lan, L. W., and Lee, Y.-T. 2011. “Exploring Decisive Factors Affecting an Organization’s SaaS Adoption: A Case Study,” International Journal of Information Management (31:6), pp. 556–563. Xin, M., and Levina, N. 2008. “Software-as-a-Service Model: Elaborating Client-Side Adoption Factors,” Proceedings of the 29th International Conference on Information Systems. Yao, Y., Watson, E., and Kahn, B. K. 2010. “Application Service Providers: Market and Adoption Decisions,” Communications of the ACM (53:7), pp. 113. Yi, Y. 1989. “A Critical Review of Consumer Satisfaction,”. CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 78 CHAPTER V: CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES Exploring Subscription Renewal Intention of Operational Cloud Enterprise Systems – A Stakeholder Perspective Authors: Walther, Sebastian, University of Bayreuth, Germany Sarker, Saonee, Washington State University, United States Sedera, Darshana, Queensland University of Technology, Australia Otto, Boris, University of St. Gallen, Switzerland Wunderlich, Philipp, University of Mannheim, Germany Published In: Proceedings of the Nineteenth Americas Conference on Information Systems 1 (AMCIS), Chicago, Illinois, August 15-17, 2013 ABSTRACT Retaining customers is a relevant topic throughout all service industries. However, only limited attention has been directed towards studying the antecedents of subscription renewal in the context of operational cloud enterprise systems. Cloud services have historically been offered as subscription-based services with the (theoretical) possibility of seamless service cancellation, in contrast to classical IT ousourcing contracts or license-based software installations of on-premise enterprise systems. In this work, we investigate the central concept of subscription renewal by focusing on different facets of IS success and their relevance for distinct employee cohorts. Analyzing inter-cohort differences has strong practical implications, as it helps IT vendors to focus on specific IT-related factors when trying to retain customers. Therefore an empirical study was undertaken. The hypotheses were developed on an individual level and tested using survey responses of IT decision makers within companies which adopted cloud enterprise systems. Gathered data was then analyzed using PLS. The results show that subscription renewal intention of the strategic cohort is mainly based on perceived system quality, whereas information quality explains most of the variance of subscription renewal in the management cohort. Beneath the cloud enterprise systems specific contributions, the work adds to the theoretical body of research related to IS success and IS continuation, as well as stakeholder perspectives. Keywords: Cloud computing, software as a service, SaaS, IS Success, IS continuance. 1 The conference proceedings are ranked B in the WI-Orientierungsliste and D in the VHB-Jourqual ranking. CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 79 1. INTRODUCTION Software as a service (SaaS) is a topic of rising importance in the enterprise applications market with a projected market volume of $21 billion in 2015 (Gartner 2012). In addition, a steady rise of SaaS-related academic literature can be observed (Walther et al. 2012). Therefore SaaS is both, a topic of economic and academic relevance. Historically, the main customer groups of enterprise systems (ES) have been large enterprises, where the implementation of an ES provided a competitive advantage to the adopting organization (Klaus et al. 2000). However, the emergence of cloud computing has dramatically changed the way ES are used in organizations, providing affordable and easy to implement software solutions (Salleh et al. 2012), which explicitly focus on SMEs, like Salesforce.com or SAP Business ByDesign. Despite the strong growth of SaaS, there are various stories of tech bloggers emphasizing the difficulties established cloud players (e.g. Salesforce.com) are facing in retaining their customers. Therefore our work aims to understand the central concept of subscription renewal of operational cloud ES. More specifically, we focus on exploring which facets of “IS success” influence subscription renewal, and whether there are significant differences in the importance of specific factors between the strategic and management cohort. In accordance with Sedera et al. (2006), we look at two different types of cohorts, namely the strategic and management cohorts. To investigate this topic, we follow the ideas of the IS success model. Theoretically, the IS success model is derived from the mathematical theory of communication (Shannon and Weaver 1949), where system quality is described as accuracy and efficiency of the IS producing a specific output, information quality, which is the degree to which the information conveys the intended meaning, and the influence or effectiveness level (Mason 1978), which depicts the effect of the information on the receiver. We argue that each step of “success” will then influence the behavior, however, to varying degrees dependent on the cohort. Different positions in companies are usually associated with varying incentive schemes, encouraging beneficial behavior in organizations. While strategic cohort’s job performance is usually measured according to the overall company success as its tasks are more globally, the management cohort in the context of IT (e.g. IT executives), are usually concerned to keep the system running to support the relevant company stakeholders. Therefore, this paper argues that there might be significant differences in the predictive quality of specific variables in varying stakeholder groups. The inter-cohort differences have both important practical and theoretical implications. From a practical viewpoint, the findings are valuable, as they provide IT sales personnel with empirical data which IS success CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 80 measures they should emphasize to make customers renew their subscription. This is especially relevant, as IT decision makers have little time to spare, and focusing on the most important factors might then be crucial for customer retention. From a theoretical viewpoint, our research contributes to linking the IS success model with technology continuation, where only a limited amount of research exists (Urbach et al. 2009). In addition, our research is also interesting from a behavioral viewpoint, as the results show that defining “behavioral belief” (e.g. individual impact, which is a part of net benefits, has been equated with perceived usefulness (Rai et al. 2002), which is a behavioral belief) and “external variable” (e.g. system and information quality (Wixom and Todd 2005)) is not per se clear. For instance, information quality might be a behavioral belief in certain situations, depending on how the job performance is measured in distinct cohorts, clarifying the urgent need for stakeholder separation in behavioral research (to the cost of external validity). The relevance of differing stakeholder perceptions in the context of IS success has been highlighted by several prior studies (e.g. Cameron and Whetten 1983; Sedera et al. 2006; Tallon and Kraemer 2000). However, in contrast to the perceptual focus emphasized in the previously mentioned studies, our focus lies in studying the role of the IS success facets in a behavioral context, namely IS continuance. In addition to the theoretical insights, our work also contributes to the context-specific body of research on SaaS in the post-acceptance phase, where only limited empirical research has been conducted 2 . The lack of research on SaaS continuation is surprising, as cloud computing has been labeled as “utility computing” on a commercial basis (Armbrust et al. 2010) or as “easy in, easy out” concept”, therefore strongly opposing “license-based” contract schemes usually found in classical on-premise ES solutions. Hence, while license-based continuation can be seen as “mandatory” to a certain degree, cloud services offer the (theoretical) possibility to quit the cancellation immediately and without financial penalties. Therefore, cloud computing can be seen as an ideal scenario to study IS continuation, especially concerning organizational level artifacts. The rest of the paper is built as follows. First, the theoretical background is given shortly introducing the concepts of IS continuance and IS success. Second, the research hypotheses are developed. Third, research methodology and results are presented and subsequently discussed. 2 A thorough literature review based on Webster and Watson (2002) (1/2000-5/2012) including the AIS basket of 8 and major conferences like ECIS and ICIS only revealed one empirical paper on SaaS continuation (Benlian et al. 2011). CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 81 2. THEORETICAL BACKGROUND 2.1. Information Systems Continuance IS continuance research is mainly based on theories drawn from social psychology, such as expectancy-value theory (Ajzen and Fishbein 1980). Per expectancy-value theory, external variables like system characteristics impact behavioral beliefs, which in turn influence the attitude towards performing the behavior. This attitude then affects the behavioral intention, which then ultimately impacts the behavior itself. According to the theory of planned behavior (TPB) (Ajzen 1991), behavioral beliefs are the subjective expectations that the behavior will produce a specific outcome, whereas attitude toward the behavior is the degree to which the performance of the behavior is positively or negatively evaluated (Ajzen 1991). Intention, in contrast, is the person’s readiness to perform a specific behavior. As postulated in the theory of reasoned action (TRA) (Ajzen and Fishbein 1980), these relationships will be predictive of behavior, if time target and context are consistently specified between belief factors, attitude and the behavior to be investigated. The relationship between behavioral intention and actual use has been validated in IS and related disciplines (Ajzen 1991). The mostly cited work in the context of IS continuance is the expectancy-confirmation model (ECM) (Bhattacherjee 2001), which has been extended in several different ways (e.g. Bhattacherjee et al. 2008). Research in the context of IS continuance has been mostly concerned on extending the ECM with factors influencing the confirmation construct, as well as finding direct antecedents influencing behavioral intention. The concept of continuation intention has been used to evaluate several scenarios, and among them are the post-adoption phase (Benlian et al. 2011), success of web-technology based business models (Wang 2008) or the end of the lifecycle (Furneaux and Wade 2011) as discontinuance intention. 2.2. Information Systems Success The IS Success Model (Delone and McLean 2003) is the most frequently used framework to structure IS success (Urbach et al. 2009). DeLone and McLean’s work was based on a literature review, which aggregated the single success measures used in prior IS research. These success measures where then categorized according to the mathematical theory of communication proposed by Shannon and Weaver (1949) and its expansion proposed by Mason (1978). The categories of IS Success identified by DeLone and McLean (1992) were analogously defined to the theory of communication. In 2003, Delone and McLean provided a ten-year update which, is subsequently referred to as revised IS Success Model. The revised CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 88 Table 3. Sample Characteristics Management Cohort Management Cohort (n=33) # Employees # System Age # 1-99 3 1-6 months 8 100-249 5 7-12 months 8 250-499 14 13-18 months 13 500-999 4 18+ months 4 1000+ 7 To handle the problem that individuals report about organizational properties, the key informant approach was used (Segars and Grover 1998), as it can lead to wrong conclusions if the survey participants report about subscription renewal, however, they have no insights into company strategy. We coped with this problem by especially asking if the participant is involved into the IS continuation process at the very beginning of the questionnaire, as well as highlighted a note in the introduction that the study is solely for stakeholders who decide about the IS. 4.2. Data Analysis Data was analyzed using SmartPLS (Ringle et al. 2005). This was done for three reasons. First, PLS supports small and medium sample sizes well (Chin et al. 2003; Hulland 1999). The “rule of thumb” for minimum sample sizes was met (Hair et al. 2011). Second, PLS is better suited for exploratory setups (Gefen et al. 2011), where new structural paths are developed building on prior model considerations (Chin 2010). Third, PLS-SEM is better suited for predictive applications (Hair et al. 2011) due to its variance-based approach. Hence, we are testing different categories of success and its predictive relevance according to distinct stakeholder groups (in comparison to testing a new behavioral model), PLS-SEM is more suited for this application. CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 89 5. RESULTS PLS estimates were reported and evaluated according to Hair et al. (2011) (see Tables 4 and 5) in a 2-step approach suggested by Chin (2010). 5.1. Measurement Model The measurement model was assessed by estimating the internal consistency, as well assuring discriminant and convergent validity. The measurement instrument showed desirable reliability with all reflective factor loadings above 0.6 which is clearly over the proposed threshold level of 0.5 (Hulland 1999). Composite reliability showed necessary level for most constructs except for subscription renewal intention, which was slightly below the threshold level of 0.8 (Nunnally and Bernstein 1994). Average variance extracted (AVE) of all latent constructs was above the recommended threshold level of 0.5 (Fornell and Larcker 1981), showing the necessary convergent validity. Discriminant validity of all latent constructs was established as the square root of each construct’s AVE was greater than the latent-variable correlation between each construct and its comparing construct (Hair et al. 2011) (see Tables 6 and 7). (Wixom and Watson 2001) Table 4. Strategic Cohort Instrument Assessment ID Item Reflective Measures Outer Loadings t-value Composite Reliability AVE Net Benefits (Adapted from Wixom and Watson 2001) 0.91 0.83 NB1 Our CES has brought significant benefits to the company. 0.9 4.81 NB2* Overall, my CES is beneficial for the company. 0.93 4.96 Subscription Renewal Intention** (Adapted from Bhattacharjee 2001) 0.78 0.64 SRI1 We intend to continue the subscription of our CES rather than discontinue ist subscription 0.69 3.35 SRI2 We intend to continue the subscription of our CES than to subscribe to any alternative means. 0.9 14.00 Information Quality (Adapted from Wixom and Todd 2005) 0.97 0.94 IQ1 Overall, I would give the inforamtion from our CES high marks. 0.98 4.54 IQ2 In general, our CES provides me with high-quality information. 0.96 4.52 System Quality (Adapted from Wixom and Todd 2005) 0.97 0.94 SQ1 In terms of system quality, I would rate our CES highly. 0.98 16.09 SQ2 Overall, our CES is of high quality. 0.97 14.85 * Newly created ** One item was dropped due to poor psychometric properties. Items with Loadings and Weights Quality Criteria CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 90 Table 5. Management Cohort Instrument Assessment Table 6. Strategic Cohort Discriminant Validity Table 7. Management Cohort Discriminant Validity 5.2. Structural Model To test the significance of the paths and to calculate t-values, the bootstrap algorithm was applied with 43 and 33 cases and 5000 subsamples each (Hair et al. 2011). The results indicate that the constructs accounted for 44.8% (strategic cohort) and 67.1% (management ID Item Reflective Measures Outer Loadings t-value Composite Reliability AVE Net Benefits (Adapted from Wixom and Watson 2001) 0.97 0.95 NB1 Our CES has brought significant benefits to the company. 0.97 41.31 NB2* Overall, my CES is beneficial for the company. 0.98 48.07 Subscription Renewal Intention** (Adapted from Bhattacharjee 2001) 0.78 0.65 SRI1 We intend to continue the subscription of our CES rather than discontinue ist subscription 0.95 27.07 SRI2 We intend to continue the subscription of our CES than to subscribe to any alternative means. 0.63 2.01 Information Quality (Adapted from Wixom and Todd 2005) 0.93 0.87 IQ1 Overall, I would give the inforamtion from our CES high marks. 0.93 10.46 IQ2 In general, our CES provides me with high-quality information. 0.93 15.05 System Quality (Adapted from Wixom and Todd 2005) 0.97 0.93 SQ1 In terms of system quality, I would rate our CES highly. 0.96 13.60 SQ2 Overall, our CES is of high quality. 0.97 14.48 * Newly created ** One item was dropped due to poor psychometric properties. Items with Loadings and Weights Quality Criteria Latent Construct 1 2 3 4 1. Net Benefits 0.91 2. Subscription Renewal Intention 0.4 0.8 3. System Quality 0.31 0.58 0.97 4. Information Quality 0.31 0.32 0.77 0.97 Note: The diagonal (bold) shows the construct's square root of AVE Latent Construct 1 2 3 4 1. Net Benefits 0.97 2. Subscription Renewal Intention 0.62 0.81 3. System Quality 0.56 0.67 0.96 4. Information Quality 0.47 0.75 0.65 0.93 Note: The diagonal (bold) shows the construct's square root of AVE CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 91 cohort) of the variance in subscription renewal intention. System quality did not significantly contribute to the explanation of subscription renewal intention in the management cohort. The highest effect size for the strategic cohort was observed as system quality, with a negative effect size of information quality. For the management cohort, information quality contributed most to the prediction of subscription renewal intention. In addition to R² values, predictive relevance was assessed using the blindfolding procedures to obtain cross-validity redundancy (Chin 1998). Results showed good predictive relevance, with all Q²>0 (Geisser 1975). Omission distance was iterated between 5 and 10, showing consistent results (Hair et al. 2011). Figure 3. Strategic Cohort Path model Figure 4. Management Cohort Path Model Subscription Renewal Intention R²=.448 Net Benefits .281* (1.606) .782*** (3.422) Information Quality -.373 (1.331) System Quality * p = .1 ** p < .05 *** p < .01 Subscription Renewal Intention R²=.671 Net Benefits .283 (1.314) .180 (0.851) Information Quality .501*** (2.719) * p < .1 ** p < .05 *** p < .01 System Quality CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 92 6. GROUP COMPARISON As outlined in the introduction, another important and seldom considered factor to further explore fundamental differences in IT decision makers’ behavioral intention between cohorts, we conducted a group comparison between the strategic and management cohort. Differences between the management and strategic cohorts were i.e., identified by Sedera et al. (2006) in the context of ES success. To test whether significant differences between the two samples exist, the t-test suggested by Chin (2004) was applied, with SE as standard error, m as sample size of the strategic cohort and n as sample size of the management cohort. Formula 1. T-Value Calculation The results show that only information quality shows (a weak) significant difference between the two stakeholders groups. Especially net benefits is difficult to discriminate between the stakeholder groups. The non-significance of system quality  subscription renewal intention between the two cohorts has to be further investigated, as the t-value (see formula) rapidly rises with larger sample sizes, thus making significant differences more likely, especially as the effect sizes strongly differ between the two cohorts. Table 8. T-Values of Inter-Cohort Differences Path t-value Sig (2-tailed) Net Benefits  Subscription Renewal Intention 0.0037 p > 0.1 (ns) Information Quality  Subscription Renewal Intention 1.6163 p = 0.1 (s) System Quality  Subscription Renewal Intention 0.4725 p > 0.1 (ns) 𝑡=𝑃𝑎𝑡ℎ𝑆𝑎𝑚𝑝𝑙𝑒 1− 𝑃𝑎𝑡ℎ𝑆𝑎𝑚𝑝𝑙𝑒 2 (𝑚−1)² (𝑚+𝑛−2)×𝑆.𝐸.𝑆𝑎𝑚𝑝𝑙𝑒 1 2+(𝑛−1)² (𝑚+𝑛−2)×𝑆.𝐸.𝑆𝑎𝑚𝑝𝑙𝑒 2 2 × 1 𝑚+1 𝑛 CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 93 7. DISCUSSION, CONCLUSION, AND LIMITATIONS Our paper yielded interesting results by including distinct stakeholder perspectives into the investigation of the central concept subscription renewal intention. System quality contributed most to the prediction of subscription renewal intention of the strategic cohort. This result was unexpected, as we argued that the strategic cohort’s job performance is mainly measured by the overall performance of the company, which can be represented more accurately by net benefits. Hence, the way we developed the hypotheses (i.e. job performance, where the IS is a means to that end), this is an unexpected result. It is hence possible to argue in various directions, such as that due to the high amount of information the strategic cohort is presented (Sparrow 2000) from various parts of the firm, that they highly focus on raw system data to reduce the complexity of their decision process. However, the focus on system quality (and not the influence of the system on the company) is partly alerting, as the system itself is only a means to an end (i.e. company performance). Therefore a more holistic view on the company might be beneficial. From marketing perspective this also has interesting implications, such as that the top management has to be approached by discussing in favor of system quality, more than on net benefits or even information quality. Concretely, this means that sales managers should emphasize the reliability, integration ability or other important characteristics of the system. Information quality contributes most to subscription renewal of the management cohort. This result is less surprising, as the management cohort (i.e., IT executives) are more integrated into the daily operations, thus have to deal with the task specific, real-time data needs (Anthony 1965) of the operational cohort. If one thinks of the dimensions, which information quality has been modeled as, such as “well formatted” or “ease of understanding”, the direct needs of the operational cohort might influence the considerations and intention to continue the subscription or discontinue the information system. From a behavioral perspective, these are interesting results, as it might show that the development of the hypotheses via “job performance” might not be universally applicable on each cohort, and an “organizational” hypotheses development might be more adequate. For instance, pressure between different organizational units might be a better or more accurate way to develop the hypotheses, yielding higher predictive power for distinct cohorts. In contrary to our prediction, system quality did not contribute to the prediction of subscription renewal intention of the management cohort. This is a rather surprising finding, as one would assume that dimensions like reliability or timeliness are of utmost importance for IT executives. Further research should tackle this finding and try to explain why the management CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 94 cohort focuses on information quality, and not system quality using qualitative methods. This study’s results have to be interpreted in the light of its limitations. First of all, the small sample sizes have to be noted. Even though the “rule of thumb” for minimum sample sizes was met, non-significant paths can turn significant if the sample size (in PLS: cases) rises. Therefore, future research should not dismiss single paths and further investigate the role of IS success in IS continuation from various stakeholder perspectives. In addition, there is also the problem that individuals report about group properties. This is especially important, as the hypotheses are developed by taking an individual perspective acting as a company stakeholder with specific tasks within the organization. The development from the individual perspective (incentive through job performance, whereas the specific incentive is coupled to the cohort type) might be insufficient to explain the specific behavioral intention. Further research has to clarify, whether these hypotheses can be better explained (and therefore better predicted!) on an organizational level. Third, we defined “top management” as strategic cohort, and IT executives as management cohort. This is consistent with Sedera et al. (2006), however, we did not assure complete convergence between the two groups due to the research design (e.g. we did not give the cohort definitions to the participants and let them decide whether they are part of the strategic or management cohort). CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 95 REFERENCES Ajzen, I. 1991. “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes (50:2), pp. 179–211. Ajzen, I., and Fishbein, M. 1980. Understanding Attitudes and Predicting Social Behaviour, Englewood Cliffs, New Jersey: Prentice-Hall. Anthony, R. 1965. Planning and Control Systems: A Framework for Analysis, Boston: Harvard University. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. 2010. “A View of Cloud Computing,” Communications of the ACM (53:4), pp. 50–58. Benlian, A., Koufaris, M., and Hess, T. 2011. “Service Quality in Software-as-a-Service: Developing the SaaS-Qual Measure and Examining Its Role in Usage Continuance,” Journal of Management Information Systems (28:3), pp. 85–126. Bhattacherjee, A. 2001. “Understanding Information Systems Continuance: An ExpectationConfirmation Model,” MIS Quarterly (25:3), pp. 351–370. Bhattacherjee, A., Perols, J., and Sanford, C. 2008. “Information Technology Continuance: A Theoretical Extension and Empirical Test,” Journal of Computer Information Systems (49:1), pp. 17–26. Cameron, K., and Whetten, D. 1983. Some Conclusions About Organizational Effectiveness. Organizational Effectiveness: A Comparison of Multiple Models, New York: Academic Press, pp. 261–277. Chin, W. W. 1998. “The Partial Least Squares Approach to Structural Equation Modeling,” In Modern Methods for Business Research, G. A. Marcoulides (ed.), Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 294–336. Chin, W. W. 2004. “Frequently Asked Questions - Partial Least Squares & PLS-Graph,”. Chin, W. W. 2010. “How to Write Up and Report PLS Analyses,” In Handbook of Partial Least Squares, V. Esposito Vinzi, W. W. Chin, J. Henseler, and H. Wang (eds.), Springer Berlin Heidelberg, pp. 655–690. Chin, W. W., Marcolin, B. L., and Newsted, P. R. 2003. “A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic - Mail Emotion/Adoption Study,” Information Systems Research (14:2), pp. 189–217. Davis, F. D. 1989. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly (13:3), pp. 319. Davis, F. D., Bagozzi, R. P., and Warshaw, P. R. 1989. “User Acceptance of Computer Technology: A Comparison of Two Theoretical Models,” Management Science (35:8), pp. 982–1003. DeLone, W., and McLean, E. 1992. “Information Systems Success: The Quest for the Dependent Variable,” Information Systems Research (3:1), pp. 60–95. Delone, W., and McLean, E. 2003. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems (19:4), pp. 9–30. Fishbein, M., and Ajzen, I. 1975. Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research, Reading, MA: Addison-Wesley. Fornell, C., and Larcker, D. F. 1981. “Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,” Journal of Marketing Research (18:1), pp. 39–50. Furneaux, B., and Wade, M. 2011. “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly (35:3), pp. 573–598. CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 96 Gable, G. G., Sedera, D., and Chan, T. 2008. “Re-Conceptualizing Information System Success : The IS-Impact Measurement Model,” Journal of the AIS (9:7), pp. 377–408. Gartner. 2012. “Forecast: Software as a Service, All Regions, 2010-2015, 1H12 Update,”Stamford. Gefen, D., Rigdon, E. E., and Straub, D. W. 2011. “An Update and Extension to SEM Guidelines for Administrative and Social Science Research,” MIS Quarterly (35:2), pp. iii–xiv. Geisser, S. 1975. “The Predictive Sample Reuse Method with Application,” Journal of the American Statistical Association (70:350), pp. 320–328. Hair, J. F., Ringle, C. M., and Sarstedt, M. 2011. “PLS-SEM: Indeed a Silver Bullet,” The Journal of Marketing Theory and Practice (19:2), pp. 139–152. Hulland, J. 1999. “Use of Partial Least Squares (PLS) in Strategic Management Research: a Review of Four Recent Studies,” Strategic Management Journal (20:2), pp. 195–204. Klaus, H., Rosemann, M., and Gable, G. G. 2000. “What is ERP?,” Information Systems Frontiers (2:2), pp. 141–162. Mason, R. O. 1978. “Measuring Information Output: A Communication Systems Approach,” Information & Management (1:4), pp. 219–234. Nunnally, J. C., and Bernstein, I. H. 1994. Psychometric Theory, New York: McGraw-Hill. Petter, S., DeLone, W., and McLean, E. 2008. “Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships,” European Journal of Information Systems (17), pp. 236–263. Rai, A., Lang, S. S., and Welker, R. B. 2002. “Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis,” Information Systems Research (13:1), pp. 50–69. Ringle, C., Wende, S., and Will, A. 2005. “SmartPLS 2.0 M3,”. Salleh, S., Teoh, S., and Chan, C. 2012. “Cloud Enterprise Systems: A Review of Literature and Adoption,” In Proceedings of the 16th Pacific Asia Conference on Information Systems. Sedera, D., Chian, F., and Dey, S. 2006. “Dentifying and Evaluating the Importance of Multiple Stakeholder Perspective in Measuring ES-Success,” In Proceedings of the 14th European Confence on Information Systems. Sedera, D., Gable, G. G., and Chan, T. 2004. “Measuring enterprise systems success: the importance of a multiple stakeholder perspective,” In Proceedings of the 12th European Confence on Information Systems. Segars, A. H., and Grover, V. 1998. “Strategic Information Systems Planning Success: An Investigation of the Construct and Its Measurement,” MIS Quarterly (22:2), pp. 139–163. Shannon, C. E., and Weaver, W. 1949. The Mathematical Theory of Communication, Urbana: University of Illinois Press. Sparrow, P. R. 2000. “New Employee Behaviours, Work Designs and Forms of Work Organization: What is in Store for the Future of Work?,” Journal of Managerial Psychology (15:3), pp. 202–218. Tallon, P., and Kraemer, K. 2000. “Executives’ Perceptions of the Business Value of Information Technology: A Process-Oriented Approach,” Journal of Management Information Systems (4:16), pp. 145–173. Urbach, N., Smolnik, S., and Riempp, G. 2009. “The State of Research on Information Systems Success – A Review of Existing Multidimensional Approaches,” Business & Information Systems Engineering (1:4), pp. 315 – 325. Urbach, N., Smolnik, S., and Riempp, G. 2010. “An Empirical Investigation of Employee Portal Success,” The Journal of Strategic Information Systems (19:3), pp. 184–206. Vroom, V. 1995. Work and Motivation, (1st ed., )San Francisco: Jossey-Bass Publishers. CLOUD ENTERPRISE SYSTEMS – STAKEHOLDER PERSPECTIVES 97 Walther, S., Plank, A., Eymann, T., Singh, N., and Phadke, G. 2012. “Success Factors and Value Propositions of Software as a Service Providers - A Literature Review and Classification,” In Proceedings of the 18th Americas’ Conference on Information Systems. Wang, Y.-S. 2008. “Assessing e-Commerce Systems Success: a Respecification and Validation of the DeLone and McLean Model of IS success,” Information Systems Journal (18:5), pp. 529–557. Webster, J., and Watson, R. T. 2002. “Analyzing the Past to Prepare for the Future: Writing a Literature Review,” MIS Quarterly (26:2), pp. xiii–xxiii. Wieneke, A., Walther, S., Eichin, R., and Eymann, T. 2013. “Erfolgsfaktoren von OnDemand-Enterprise-Systemen aus der Sicht des Anbieters - Eine explorative Studie,” In Proceedings of the 11th International Conference on WirtschaftsinformatikLeipzig. Wixom, B. H., and Todd, P. A. 2005. “A Theoretical Integration of User Satisfaction and Technology Acceptance,” Information Systems Research (16:1), pp. 85–102. Wixom, B. H., and Watson, H. J. 2001. “An Empirical Investigation of the Factors Affecting Data Warehousing Success,” MIS Quarterly (25:1), pp. 17. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 104 is a relevant variable in the context of cloud-based ES. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 105 3. HYPOTHESES DEVELOPMENT 3.1. Continuance Forces We define continuance forces as factors which actively influence the perpetuation of the status quo. In this study, we assume that the strongest argument for continuing a system is its operational success. Hence, in order to keep our model coherent within our socio-technical approach, we investigate two technical success measures (information quality and system quality) and one socio-organizational success (net benefits). System Quality System quality, being the most desirable characteristic of an information system (Delone and McLean 2003), reflects certain system properties, such as processing power, reliability, or ease of use. System quality has a strong impact on the workflows of operational system users, as the input and output of data is interwoven into daily business (i.e. system failure, such as the infamous “blue screen”, might interrupt work in progress or even lead to loss of data). In addition, a system which is difficult to use might use up a significant amount of human resources, which could be better distributed and utilized elsewhere. Hence, poor system quality can lead to consumption of valuable company resources. As the IT function is responsible for problems caused by IT failure, it will try to ensure high system quality. If a system cannot provide these requirements, it is likely to be replaced (Furneaux and Wade 2011). The relationship between continuance and system quality has gathered mixed empirical support (Petter et al. 2008) on an organizational level. However, it has not been tested in the context of SaaS or ES. It was therefore hypothesized that H1: System quality is positively correlated with continuance intentions. Information Quality Information quality is the most desirable characteristic of system output (Delone and McLean 2003), referring to aspects such as format, timeliness, or comprehensibility. One of the main tasks of ES is the provision of information for strategic, management, and operational needs within a company (Anthony 1965). Poor information quality can harm the company on several organizational levels. For instance, operational users of the system are dependent on an adequate format of the data, as transferring data between input interfaces can consume considerable time CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 106 when formats are incompatible. In addition, strategic decisions are often based on an aggregation and analysis of fundamental data, with the quality of the information significantly affecting executives in their organizational behavior. If the system is not capable of providing relevant and properly formatted data, executives might give this pressure down to the IT function, which will be forced to replace the information system. There is no sufficient empirical evidence for the relationship between information quality and continuance intention (Petter et al. 2008). Thus, it was hypothesized that H2: Information quality is positively correlated with continuance intentions. Net Benefits Net benefits is the extent to which an information system is beneficial to individuals, groups, and organizations (Delone and McLean 2003). The main task of an information system is to support the company in its business processes. Hence, an information system is only a means to an end, such as profitability. The failure to support business processes, help to raise productivity or the exposure to risks due to the information system therefore have to be seen as essential parts whether an ES is continuously used. Hence, failure to support company goals on the part of an information system might lead to discontinuance of this system. There is some empirical evidence for the relationship between net benefits and continuance intention (Petter et al. 2008). However, this relationship has not been tested in the context of SaaS. Therefore, it was hypothesized that H3: Net benefits are positively correlated with continuance intentions. 3.2. Continuance Inertia We define continuance inertia as sources which positively influence the continuance of information systems. However, in contrary to IS success, these sources are not related to a positive evaluation of the system. In our study, this is represented by technical integration of the system and system investment as socio-organizational commitment analogously to the work of Furneaux and Wade (2011). Technical Integration Technical integration is defined as the “extent to which an information system relies on sophisticated linkages among component elements to deliver required capabilities” (Furneaux CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 107 and Wade 2011). Despite the vision of seamless service orientation and modern ERP systems, information systems are usually embedded within an interwoven network of information technology. These interrelations between operational systems are often not well documented, leading to unpredictable system performance when a system is replaced. In addition, replacement intentions are usually formed more easily with regard to systems with a low complexity, as high complexity and integration increases the likelihood of difficulties when the system is discontinued (Furneaux and Wade 2011), resulting in performance shortcomings which can severely damage daily business. Thus H4: Technical integration is positively correlated with continuance intentions. System Investment System investment is defined as “the financial and other resources committed to the acquisition, implementation, and use of an information system” (Furneaux and Wade 2011). Implementing and maintaining an information system is usually associated with a variety of investments, such as capital and human resource investments. Therefore, discontinuance of an information system is usually associated with a short-term loss of company resources, which in turn is associated with additional costs for implementing the replacing system. In addition, IT decision makers have expressed their feeling of “wasting” resources (Furneaux and Wade 2011) when discontinuing a system. The relationship between system investment and continuance intention (as negative replacement intention) was insignificant in the initial study (Furneaux and Wade 2011). Despite this fact, we hypothesize that H5: System investment is positively correlated with continuance intentions. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 108 Figure 1. Research Model System Quality* Information Quality* Net Benefits** Continuance Intention System Investment** Technical Integration* H1 H2 H3 H4 H5++ Continuance Forces Continuance Inertia Continuance Decision not investigated * technology-related variable ** socio-organizational variable CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 109 4. METHODOLOGY 4.1. Data Collection The full-scale field test was conducted between August and December 2012. The survey was made available as an online questionnaire, on paper, and as an interactive PDF file. It was then distributed over several distribution channels, such as social media channels of cloud service providers, or it was directly made available to IT decision makers having adequate backgrounds (e.g. via business networks like LinkedIn and XING). After dropping 23 invalid questionnaires, 115 questionnaires were used to test the research model (see Table 1). Due to the methodology of the survey, individuals reported on organizational or group properties. It was therefore important to make sure the participants possessed adequate knowledge. Hence, we applied the key informant approach (Segars and Grover 1998). This included a note in the introduction part of the questionnaire that the study addresses key decision makers, and a specific question at the beginning of the questionnaire asking if the participant is involved in the decision whether or not the ES should be continued. In addition, in an effort to increase content validity, we asked the participants to fill out the questionnaire with regard to one specific type of ES only. Due to the distribution method via social media platforms, the response rates could not be calculated reliably. However, to address the possibility of response rate bias, we used a stratified sample of IT decision makers. Table 1. Sample Characteristics Position in Company # # Employees # System Age # Top Managememt 52 1-99 35 1-6 months 26 IT Executive 34 100-249 14 7-12 months 29 Line of Business Manager 17 250-499 29 13-18 months 36 IT Personnel 10 500-999 16 18+ months 24 Others (e.g. IT strategy) 2 1000+ 21 4.2. Instrument Development To test the research model, we used both formative and reflective measures (see Table 2). The items were measured on a 7-point Likert scale, ranging from “strongly disagree” to “strongly agree”. Continuance forces were measured formatively, as formative measurement provides “specific and actionable attributes” of a concept (Mathieson et al. 2001), which is particularly CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 110 interesting from a practical viewpoint. In formative measurement, the weight of single indicators can be used to draw practical implications on the importance of specific details and therefore guide practical enforcement on these system characteristics (e.g. “overall system quality is high” (reflective) vs. “system is easy to use” (formative)). Another possibility of modeling “actionable attributes” would have been the use of multi-dimensional constructs, where first-order constructs (dimensions) can be measured reflectively (e.g. Wixom and Todd 2005). However, taking the IT decision makers’ time constraints into account, this approach would have been rather impracticable, as it would have raised the number of questions by the number of three (assuming three indicators per first-order construct). Unlike continuance forces, which represent the evaluation of an information system’s success, continuance inertia can be seen as historically given. Measuring these constructs formatively would add little to the practical contribution of the study. Therefore, these constructs were measured using well-validated reflective scales (Furneaux and Wade 2011). The formative instrument was developed according to Moore and Benbasat (1991), with elements of newer scale development procedures (Diamantopoulos and Winklhofer 2001; MacKenzie et al. 2011; Petter et al. 2007) in six steps (see Figure 2). In the following, the process is described in detail. Figure 2. Quantitative Assessment of Formative Instrument In the conceptualization and content specification phase, we clearly defined the constructs and identified SaaS specific success dimensions by conducting a content-based systematic literature review based on Webster and Watson (2002). To these newly identified SaaS specific dimensions, we added existing ES success measures (Gable et al. 2008) and general IS success measures (Wixom and Todd 2005). This led to an initial set of 39 net benefits, 8 information I: Item Creation Before Data Collection II: Scale Development III: Instrument Testing Conceptualization Content Specification Item Generation Access Content Validity Pretest and Refinement Evaluation of Formative Measurement Model and Re-Specification Field Test CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 111 quality dimensions and 21 system quality dimensions. This initial set was then reduced by the first author by culling or dropping items which seemed too narrow or not significant in our context of investigation. Based on this identification of the relevant dimensions, we then generated an item pool which represented all aspects of the construct, while “minimizing the extent to which the items tap concepts outside of the domain of the focal construct” (MacKenzie et al. 2011). As “dropping a measure from a formative-indicator model may omit a unique part of the conceptual domain and change the meaning of the variable, because the construct is a composite of all the indicators” (MacKenzie et al. 2005) and keeping “irrelevant items” will not bias the results when analyzing the data using PLS (Mathieson et al. 2001), all initially identified dimensions were kept and transformed into items. Content validity, which is the “degree to which items in an instrument reflect the content universe to which the instrument will be generalized” (Straub et al. 2004), was assessed using the Q-sorting procedure, which, according to Petter et al. (2007), is one of the best methods to ensure content validity for formative indicators. In this effort, we followed a two-round procedure. In the first round we gave a list of the previously created items and construct definitions to one regular student, one doctoral student, one associate professor, and one professor. The participants then had to match the items to the different constructs. The first round showed a low average hit ratio of 0.67 and a Cohen’s Kappa (Cohen 1968) of 0.63. After identifying and changing problematic items (e.g. wording, intersection between items), this procedure was repeated. In the second round the hit ratio rose to 0.85 and Cohen’s Kappa was clearly above the recommended threshold level of 0.65 (e.g. Todd and Benbasat 1992). After this round, two more items were modified. The pretest was conducted to have a first test of the overall instrument, especially concerning wording, length, and instructions (Moore and Benbasat 1991). The questionnaire was distributed to sales and consulting divisions of one of the largest cloud service providers worldwide, as well as to professors, associate professors, and doctoral students. The survey was distributed online. Under each question page a textbox was given, allowing the participants to freely comment on problems. 19 questionnaires were completed. A few changes were made, such as the shortening of introductory text or re-wording of “my cloud enterprise system” to “our cloud enterprise system” to highlight the organizational character of the study. The quantitative evaluation of the formative measurement model is described in the subsequent chapter. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 112 Table 2. Primary Constructs and Definitions Construct Definition Literature Sources System Quality (Formative) The desirable characteristics of a system, e.g. ease of use, reliability, response time, etc. Bailey and Pearson 1983 DeLone and McLean 1992 Delone and McLean 2003 Information Quality (Formative) The desirable characteristics of system output, e.g. completeness, format, relevance, etc. Bailey and Pearson 1983 DeLone and McLean 1992 Delone and McLean 2003 Net Benefits (Formative) The extent to which an information system is beneficial to individuals, groups and organizations. DeLone and McLean 1992 Delone and McLean 2003 System Investment (Reflective) “The financial and other resources committed to the acquisition, implementation, and use of an information system.” Gill 1995 Keil et al. 2000 Furneaux and Wade 2011 Technical Integration (Reflective) “The extent to which an information system relies on sophisticated linkages among component elements to deliver required capabilities.” Swanson and Dans 2000 Furneaux and Wade 2011 4.3. Data Analysis The data was analyzed using SmartPLS (Ringle et al. 2005) and SPSS. SPSS was used to calculate variance inflation factors and to run additional exploratory factors analysis. We chose a variance-based approach to analyze the structural model for four reasons. First, PLS is well suited to analyze small to medium sample sizes, providing parameter estimates at low sample sizes (Chin et al. 2003; Hulland 1999). Second, PLS is more appropriate for exploratory research (Gefen et al. 2011), especially to explore new structural paths within incremental studies which build on prior models (Chin 2010). Third, due to its variance-based approach, PLS is better suited for predictive application. As the goal of the study was to find drivers of organizational level continuance, and not to test a specific behavioral model, PLS is adequate in this context. Fourth, continuance forces were measured formatively, which is adequately supported by PLS. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 113 5. RESULTS The full-scale The PLS estimates were reported according to recommendations provided by Hair et al. (2011), and in a 2-step approach, as outlined by Chin (2010). The measurement model and the path model were both analyzed with parameter settings using 115 cases and 5000 samples (Hair et al. 2011). Missing values were replaced using the “mean replacement” algorithm supported by SmartPLS. 5.1. Measurement Model The reflective measurement model was assessed by estimating internal consistency, as well as discriminant and convergent validity (see Appendix, Table 4). The instrument showed satisfactory reliability, as reflective factor loadings were all above 0.64, which is clearly above the proposed threshold level of 0.5 (Hulland 1999). Composite reliability also was adequate, with all constructs being above 0.85 (Nunnally and Bernstein 1994). Convergent validity was established as average variance extracted (AVE) of all constructs was clearly above 0.5 (Fornell and Larcker 1981). All square roots of each AVE were higher than the corresponding latent variable correlations, showing a desirable level of discriminant validity (see Table 3). Table 3. Discriminant Validity Formative measures were assessed using the 3-step procedure proposed by Hair et al. (2013) (see Figure 3). The results can be found in the Appendix (Table 5). In a first step, convergent validity was assessed, which is the “extent to which a measure correlates positively with other measures of the same construct” (Hair et al. 2013). In other words, formative constructs should highly correlate with reflective measures of the same construct. This test is also known as redundancy analysis (Chin 1998). All constructs showed adequate convergent validity, with path strengths ranging from 0.82 to 0.87, which is above the threshold level of 0.8 (Chin 1998). The reflective Latent Construct 1 2 3 4 5 6 1. System Quality formative 2. Information Quality 0.68 formative 3. Net Benefits 0.63 0.54 formative 4. Technical Integration -0.15 -0.05 -0.16 0.89 5. System Investment -0.28 -0.07 -0.25 0.68 0.73 6. Continuance Intention 0.68 0.52 0.56 -0.28 -0.16 0.74 Note: The diagonal (bold) shows the construct's square root of AVE CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 120 REFERENCES Ahtiala, P. 2006. “The optimal pricing of computer software and other products with high switching costs,” International Review of Economics & Finance (15:2), pp. 202–211. Ajzen, I. 1991. “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes (50:2), pp. 179–211. Anthony, R. 1965. Planning and Control Systems: A Framework for Analysis, Boston: Harvard University. Arkes, H. R., and Blumer, C. 1985. “The Psychology of Sunk Cost,” Organizational Behavior and Human Decision Processes (35:1), pp. 124–140. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. 2010. “A View of Cloud Computing,” Communications of the ACM (53:4), pp. 50–58. Ayyagari, R., Grover, V., and Purvis, R. 2011. “Technostress: Technological Antecedents and Implications,” MIS Quarterly (35:4), pp. 831–858. Bailey, J. E., and Pearson, S. W. 1983. “Development of a Tool for Measuring and Analyzing Computer User Satisfaction,” Management Science (29:5), pp. 530–545. Benlian, A., and Hess, T. 2011. “Opportunities and Risks of Software-as-a-Service: Findings from a Survey of IT Executives,” Decision Support Systems (52:1), pp. 232–246. Benlian, A., Koufaris, M., and Hess, T. 2011. “Service Quality in Software-as-a-Service: Developing the SaaS-Qual Measure and Examining Its Role in Usage Continuance,” Journal of Management Information Systems (28:3), pp. 85–126. Benlian, A., Vetter, J., and Hess, T. 2012. “The Role of Sunk Cost in Consecutive IT Outsourcing Decisions,” Journal of Business Economics (82:2), pp. 181–213. Bhattacherjee, A. 2001. “Understanding Information Systems Continuance: An ExpectationConfirmation Model,” MIS Quarterly (25:3), pp. 351–370. Bhattacherjee, A., Perols, J., and Sanford, C. 2008. “Information Technology Continuance: A Theoretical Extension and Empirical Test,” Journal of Computer Information Systems (49:1), pp. 17–26. Bibi, S., Katsaros, D., and Bozanis, P. 2012. “Business Application Acquisition: On-Premise or SaaS-Based Solutions?,” IEEE Software (29), pp. 86–93. Cenfetelli, R., and Bassellier, G. 2009. “Interpretation of Formative Measurement in Information Systems Research,” MIS Quarterly (33:4), pp. 689 – 708. Chin, W. W. 1998. “The Partial Least Squares Approach to Structural Equation Modeling,” In Modern Methods for Business Research, G. A. Marcoulides (ed.), Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 294–336. Chin, W. W. 2010. “How to Write Up and Report PLS Analyses,” In Handbook of Partial Least Squares, V. Esposito Vinzi, W. W. Chin, J. Henseler, and H. Wang (eds.), Springer Berlin Heidelberg, pp. 655–690. Chin, W. W., Marcolin, B. L., and Newsted, P. R. 2003. “A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic - Mail Emotion/Adoption Study,” Information Systems Research (14:2), pp. 189–217. Cohen, J. 1968. “Weighted Kappa: Nominal Scale Agreement Provision for Scaled Disagreement or Partial Credit,” Psychological Bulletin (70:4), pp. 213–220. Davenport, T., and Short, J. 1990. “The New Industrial Engineering: Information Technology and Business Process Redesign,” Sloan Management Review (31:4), pp. 11–26. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 121 Davis, F. D. 1989. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly (13:3), pp. 319. DeLone, W., and McLean, E. 1992. “Information Systems Success: The Quest for the Dependent Variable,” Information Systems Research (3:1), pp. 60–95. Delone, W., and McLean, E. 2003. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems (19:4), pp. 9– 30. Diamantopoulos, A., and Winklhofer, H. M. 2001. “Index Construction with Formative Indicators: An Alternative to Scale Development,” Journal of Marketing Research (38:2), pp. 269–277. Esteves, J. M., and Bohoquez, V. 2007. “An Updated ERP Systems Annotated Bibliography: 2001-2005,” Communications of the AIS (19:18), pp. 386–446. Fishbein, M., and Ajzen, I. 1975. Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research, Reading, MA: Addison-Wesley. Fitzgerald, G., and Russo, N. L. 2005. “The Turnaround of the London Ambulance Service Computer-Aided Despatch system (LASCAD),” European Journal of Information Systems (14:3), pp. 244–257. Fornell, C., and Larcker, D. F. 1981. “Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,” Journal of Marketing Research (18:1), pp. 39–50. Furneaux, B., and Wade, M. 2011. “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly (35:3), pp. 573–598. Gable, G. G., Sedera, D., and Chan, T. 2008. “Re-Conceptualizing Information System Success : The IS-Impact Measurement Model,” Journal of the AIS (9:7), pp. 377–408. Gartner. 2011. “Gartner Forecast: Public Cloud Services, Worldwide and Regions, Industry Sectors, 2010-2015, 2011 Update,”Stamford. Gartner. 2012. “Forecast: Software as a Service, All Regions, 2010-2015, 1H12 Update,”Stamford. Gefen, D., Rigdon, E. E., and Straub, D. W. 2011. “An Update and Extension to SEM Guidelines for Administrative and Social Science Research,” MIS Quarterly (35:2), pp. iii–xiv. Geisser, S. 1975. “The Predictive Sample Reuse Method with Application,” Journal of the American Statistical Association (70:350), pp. 320–328. Gill, T. G. 1995. “Early Expert Systems: Where Are They Now?,” MIS Quarterly (19:1), pp. 51– 81. Hair, J. F., Ringle, C. M., and Sarstedt, M. 2011. “PLS-SEM: Indeed a Silver Bullet,” The Journal of Marketing Theory and Practice (19:2), pp. 139–152. Hair, J., Hult, T., Ringle, C., and Sarstedt, M. 2013. A Primer on Partial Least Squares Strucutral Equation Modeling PLS-SEM, Thousand Oaks: Sage. Hulland, J. 1999. “Use of Partial Least Squares (PLS) in Strategic Management Research: a Review of Four Recent Studies,” Strategic Management Journal (20:2), pp. 195–204. Jeyaraj, A., Rottman, J. W., and Lacity, M. C. 2006. “A Review of the Predictors, Linkages, and Biases in IT Innovation Adoption Research,” Journal of Information Technology (21:1), pp. 1–23. Keil, M., Mann, J., and Rai, A. 2000. “Why Software Projects Escalate: An Empirical Analysis and Test of Four Theoretical Models,” MIS Quarterly (24:4), pp. 631–664. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 122 Ko, D.-G., Kirsch, L. J., and King, W. R. 2005. “Antecedents of Knowledge Transfer from Consultants to Clients in Enterprise System Implementations,” MIS Quarterly (29:1), pp. 59–85. Leonard-Barton, D. 1988. “Implementation as Mutual Adaptation of Technology and Organization,” Research Policy (17:5), pp. 251–267. MacKenzie, S. B., Podsakoff, P. M., and Jarvis, C. B. 2005. “The Problem of Measurement Model Misspecification in Behavioral and Organizational Research and Some Recommended Solutions.,” Journal of Applied Psychology (90:4), pp. 710–730. MacKenzie, S., Podsakoff, P., and Podsakoff, N. 2011. “Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques,” MIS Quarterly (35:2), pp. 293–334. Mathieson, K., Peacock, E., and Chin, W. W. 2001. “Extending the Technology Acceptance Model: The Influence of Perceived User Resources,” ACM SIGMIS Database (32:3), pp. 86. Mell, P., and Grance, T. 2009. “The NIST Defintion of Cloud Computing,”. Moore, G. C., and Benbasat, I. 1991. “Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation,” Information Systems Research (2:3), pp. 192–222. Nunnally, J. C., and Bernstein, I. H. 1994. Psychometric Theory, New York: McGraw-Hill. Oliver, R. L. 1980. “A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions,” Journal of Marketing Research (17:4), pp. 460–469. Petter, S., DeLone, W., and McLean, E. 2008. “Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships,” European Journal of Information Systems (17), pp. 236–263. Petter, S., Straub, D., and Rai, A. 2007. “Specifying Formative Constructs in Information Systems Research,” MIS Quarterly (31:4), pp. 623–656. Ringle, C., Wende, S., and Will, A. 2005. “SmartPLS 2.0 M3,”. Rogers, E. 1983. Diffusion of Innovations, (5 th, )Free Press. Rousseau, D. 1985. “Issues of Level in Organizational Research: Multi-Level and Cross Level Perspectives,” Research in Organizational Behaviour (7), pp. 1–37. Salleh, S., Teoh, S., and Chan, C. 2012. “Cloud Enterprise Systems: A Review of Literature and Adoption,” In Proceedings of the 16th Pacific Asia Conference on Information Systems. Segars, A. H., and Grover, V. 1998. “Strategic Information Systems Planning Success: An Investigation of the Construct and Its Measurement,” MIS Quarterly (22:2), pp. 139–163. Straub, D., Boudreau, M.-C., and Gefen, D. 2004. “Validation Guidelines for IS Positivist Research,” Communications of the AIS (13:24), pp. 380–427. Susarla, A., Barua, A., and Whinston, A. B. 2009. “A Transaction Cost Perspective of the ‘Software as a Service’ Business Model,” Journal of Management Information Systems (26:2), pp. 205–240. Swanson, E. B., and Dans, E. 2000. “System Life Expectancy and the Maintenance Effort: Exploring Their Equilibration,” MIS Quarterly (24:2), pp. 277–297. Teo, H. H., Wei, K. K., and Benbasat, I. 2003. “Predicting Intention to Adopt Interorganizational Linkages: An Institutional Perspective,” MIS Quarterly (27:1), pp. 19–49. Todd, P., and Benbasat, I. 1992. “The Use of Information in Decision Making: An Experimental Investigation of the Impact of Computer-Based Decision Aids,” MIS Quarterly (16:3), pp. 373–393. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 123 Tornatzky, L., and Fleischer, M. 1990. The Processes of Technological Innovation, Lexington, MA: Lexington Books. Urbach, N., Smolnik, S., and Riempp, G. 2009. “The State of Research on Information Systems Success – A Review of Existing Multidimensional Approaches,” Business & Information Systems Engineering (1:4), pp. 315 – 325. Urbach, N., Smolnik, S., and Riempp, G. 2010. “An Empirical Investigation of Employee Portal Success,” The Journal of Strategic Information Systems (19:3), pp. 184–206. Walther, S., and Eymann, T. 2012. “The Role of Confirmation on IS Continuance Intention in the Context of On-Demand Enterprise Systems in the Post-Acceptance Phase,” In Proceedings of the 18th Americas’ Conference on Information Systems. Walther, S., Plank, A., Eymann, T., Singh, N., and Phadke, G. 2012. “Success Factors and Value Propositions of Software as a Service Providers - A Literature Review and Classification,” In Proceedings of the 18th Americas’ Conference on Information Systems. Wang, Y.-H. 2011. “The Role of SaaS Privacy and Security Compliance for Continued SaaS Use,” In NCM 2011 Proceedings, pp. 303–306. Wang, Y.-S. 2008. “Assessing e-Commerce Systems Success: a Respecification and Validation of the DeLone and McLean Model of IS success,” Information Systems Journal (18:5), pp. 529–557. Webster, J., and Watson, R. T. 2002. “Analyzing the Past to Prepare for the Future: Writing a Literature Review,” MIS Quarterly (26:2), pp. xiii–xxiii. Wixom, B. H., and Todd, P. A. 2005. “A Theoretical Integration of User Satisfaction and Technology Acceptance,” Information Systems Research (16:1), pp. 85–102. Xin, M., and Levina, N. 2008. “Software-as-a-Service Model: Elaborating Client-Side Adoption Factors,” Proceedings of the 29th International Conference on Information Systems. CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 124 APPENDIX Table 4. Quantitative Assessment of Measurement Model (Reflective) loadings t-value Composite Reliability AVE Composite Reliability 0.74 0.85 CI1 We intend to continue the subscription of our cloud enterprise system rather than discontinue its subscription. 0.866 12.300 CI2 We intend to continue the subscription of our cloud enterprise system than to subscribe to any alternative means. 0.853 18.727 loadings t-value Composite Reliability AVE Composite Reliability 0.89 0.96 TI1 The technical characteristics of the system make it complex. 0.931 19.343 TI2 The system depends on a sophisticated integration of technology components. 0.964 22.714 TI3 There is considerable technical complexity underlying this system. 0.938 18.156 loadings t-value Composite Reliability AVE Composite Reliability 0.73 0.89 SI1 Significant organizational resources have been invested in this system. 0.641 2.253 SI2 We have commited considerable time and money to the implementation and operation of the system. 0.947 3.148 SI3 The financial investments that have been made in this system are substantial. 0.946 3.120 * One item was dropped due to poor psychometric properties. Continuance Intention* (reflective) (Adapted from Bhattacharjee 2001) Technical Integration (reflective) (Adapted from Furneaux and Wade 2011) System Investment (reflective) (Adapted from Furneaux and Wade 2011) CONTINUANCE OF CLOUD-BASED ENTERPRISE SYSTEMS 125 Table 5. Quantitative Assessment of Measurement Model (Formative) Redundancy Analysis, Assessing Multicollinearity, Significance and Contribution VIF t-value weights loadings Our cloud enterprise system… NB1 … increases the productivity of end-users. 3.696 0.160 0.034 0.751 NB2* … increases the overall productivity of the company. 3.557 2.078 0.485 0.806 NB3* … enables individual users to make better decisions. 1.875 1.786 0.342 0.660 NB4 … helps to save IT-related costs. 2.912 1.072 0.287 0.515 NB5 … makes it easier to plan the IT costs of the company. 2.475 1.474 -0.308 0.331 NB6 … enhances our strategic flexibility. 3.923 0.595 -0.153 0.492 NB7 … enhances the ability of the company to innovate. 3.559 1.278 -0.331 0.313 NB8 … enhances the mobility of the company's employees. 2.855 0.342 0.082 0.657 NB9 … improves the quality of the company's business processes. 2.156 0.918 0.235 0.593 NB10 … shifts the risks of IT failures from my company to the provider. 1.888 1.495 0.328 0.562 NB11 … lower the IT staff requirements within the company to keep the system running. 1.708 0.539 0.141 0.365 NB12 … improves outcomes/outputs of my company. 1.955 0.504 0.122 0.514 Net Benefits (reflective) (Adapted from Wixom and Watson (2001)) f² Redundancy Analysis 0.815 NB13 … has changed my company significantly. 23.901 0.903 NB14 … has brought significant benefits to the company. 91.381 0.938 VIF t-value weights loadings Our cloud enterprise system… SQ1# … operates reliabliy and stable. 1.570 0.729 0.088 0.530 SQ2# … can be flexibly adjusted to new demands or conditions. 2.463 1.399 0.257 0.785 SQ3# … effectively integrates data from different areas of the company. 2.152 0.941 -0.148 0.619 SQ4# … makes information easy to access (system accessibility). 2.201 0.093 0.015 0.574 SQ5 … is easy to use. 2.245 0.450 0.071 0.586 SQ6# … provides information in a timely fashion (response time). 1.941 0.234 -0.035 0.515 SQ7* … provides key features and functionalities that meet the business requirements. 2.257 2.117 0.338 0.803 SQ8* … is secure. 1.334 2.090 0.250 0.638 SQ9 … is easy to learn. 2.308 0.342 -0.055 0.504 SQ10 … meets different user requirements within the company. 2.031 0.543 0.105 0.654 SQ11 … is easy to upgrade from an older to a newer version. 1.643 1.053 0.152 0.638 SQ12* … is easy to customize (after implementation, e.g. user interface). 2.006 1.857 0.318 0.762 System Quality (reflective) (Adapted from Wixom and Todd (2005)) f² Redundancy Analysis 0.808 SQ13# In terms of system quality, I would rate our cloud enterprise system highly. 141.426 0.969 SQ14# Overall, our cloud enterprise system is of high quality. 136.564 0.969 VIF t-value weights loadings Our cloud enterprise system… IQ1# … provides a complete set of information. 2.313 0.070 0.016 0.726 IQ2# … produces correct information. 2.280 0.194 -0.054 0.661 IQ3# … provides information which is well formatted. 2.711 0.010 -0.025 0.725 IQ4#* … provides me with the most recent information. 2.793 1.632 0.460 0.879 IQ5 … produces relevant information with limited unnecessary elements. 2.774 1.412 0.393 0.905 IQ6 … produces information which is easy to understand. 2.903 1.491 0.317 0.841 Information Quality (reflective) (Adapted from Wixom and Todd (2005)) f² Redundancy Analysis 0.868 IQ7# Overall, I would give the information from our cloud enterprise system high marks. 85.378 0.961 IQ8# In general, our cloud enterprise system provides me with high-quality information. 69.523 0.956 # Wixom and Todd (2005); * significant at least at the p=0.1 level Net Benefits (formative) System Quality (formative) Information Quality (formative) CONCLUSION 126 CHAPTER VII: CONCLUSION 1. RESEARCH SUMMARY The research question of the thesis was as follows: What factors influence the organizational level continuance intention of cloud-based enterprise systems (ES)? In an effort to answer this question, the thesis provided five interrelated papers regarding the continuance and success of operational cloud-based ES. In Chapter 2 a conceptual model to study the continuance of operational cloud-based ES was developed. Subsequently, within the third Chapter, an instrument was created to formatively measure cloud-based ES success-related variables. Within Chapter 4, the variables proposed in Chapter 2 were quantitatively explored using reflective measures of well-validated reflective measurement instruments. In Chapter 5, it was investigated, which influence distinct success variables have on the continuance intention of the strategic and management cohorts to dissolve problems regarding the broadness of the sample. Finally, in Chapter 6, the findings of the Chapters 2 to 4 were synthesized and the formative measurement instrument developed in Chapter 3 was used to quantitatively assess the final research model. The thesis provided evidence that continuance intention is both, influenced by information systems success variables, as well as continuance inertia. More specifically, system quality and net benefits had a significant 1 positive impact on continuance intention. Information quality showed to have no siginificant impact on the dependent variable. In contrast to the development of the hypothesis, technical integration had a significant negative impact. System investment had a significant positive impact on continuance intention (both, in Chapters 4 and 6). The key empirical findings related to the research question are summarized in Figure 1. 1 Refers to significance at least at the p=0.1 level. CONCLUSION 127 Figure 1. Key Empirical Findings Operational Cloud-Based ES Success (Paper 2, Chapter 3) •Strategic flexibility, cost savings, improvement of outputs/outcomes, and organizational productivity had significant (p=0.1) impact on net benefits before re-specification •Net benefits was re-specified according to the original IS success model (DeLone and McLean 1992) •Reliability, provision of key features, and ease of customization had significant (p=0.1) impact on system quality before re-specification •System quality was re-specified into architecture agility, system performance, business requirements, ease of utilization, and security •Information quality showed to be robust in the context of cloud-based ES Subscription Renewal of Cloud-Based ES (Paper 3, Chapter 4) •System quality, technical integration, system investment, attitude, and net benefits had significant (p=0.1) impact on continuance intention •Confirmation had a significant (p=0.1) impact on attitude and net benefits •Technical integration had a negative impact on continuance intention •Information quality had no significant impact on continuance intention Operational Cloud-Based ES –Stakeholder Perspectives (Paper 4, Chapter 5) •System qualtiy had the highest impact on continuance intention in the strategic cohort •Information quality had the highest impact on continuance intention in the management cohort •Information quality showed to significantly (p=0.1) differ between cohorts •Information qualtiy had no significant impact on continuance intention of the strategic cohort •Net benefits had no significant impact on continuance intention of the management cohort Continuance of Cloud-Based ES (Paper 5, Chapter 6) •System quality had the highest impact on continuance intention •System quality, net benefits, technical integration, and system investment had significant (p=0.1) impact on continuance intention •Information quality had no significant impact on continuance intention •Technical integration had negative impact on continuance intention CONCLUSION 128 2. IMPLICATIONS FOR INFORMATION SYSTEM THEORY The work makes two main theoretical contributions which are not artifact specific. 2.1. An Imperative for Information Systems Success Research The first theoretical contribution lies in studying the role of information systems success as post-acceptance variables in continued information systems use. Especially on an organizational level of analysis, there is a lack of empirical findings concerning the relationship between success variables and continuance intention (Petter et al. 2008; Urbach et al. 2009). Net benefits and system quality both explained a decent amount of the variance in continuance intention, showing the importance of post-acceptance success evaluations on the intention to continue the subscription of cloud-based ES. Interestingly, within Chapters 4 and 6 2 , there was no significant correlation between information quality and continuance intention, even though information needs of the distinct stakeholder groups are among the most important IT executive concerns. Triggered by this finding, we conducted an exploratory stakeholder analysis in Chapter 5, revealing that information quality contributes to the prediction of continuance intention in the management cohort, but not in the strategic cohort. The results propose that the prediction of continuance intention might be dependent on patterns and mechanisms which have yet to be identified and which cannot be reduced to an individual behavioral mechanism. The information systems success model (DeLone and McLean 1992) and its revision (Delone and McLean 2003) have been criticized in various publications (e.g. Seddon 1997) for its questionable hypotheses network, which was developed based on the theory of communication (Shannon and Weaver 1949) using a content-based literature review. The information systems success model has often led to confusion as it is generally designed to be applied both, on an organizational level of analysis and an individual level of analysis (Petter et al. 2008). However, the success variables cannot all be applied on an organizational context. For instance, user satisfaction is an individual level variable solely, whereas system quality can both, be interpreted as organizational level and individual level variable. In this thesis only the success variables were included, but not the hypotheses suggested by the model. The success variables showed to adequately represent success, and showed to be a good predictor of organizational level continuance. This clarifies two things, which have to be 2 Note that the sample used in Chapter 4 is a subset of the sample used in Chapter 6. Hence, it is not surprising that the data analysis yields similar results, even though the measurement of the success dimensions differed between the Chapters 4 and 6. CONCLUSION 129 addressed in future research on information systems success. First, the variables, which are applicable (e.g. information quality, system quality, and net benefits) are adequate and meaningful on an organizational level. Therefore, the variables themselves are a good representation of success by themselves and should be included in future research. This is especially supported by the fact that the information systems success variables captured all identified SaaS success dimensions exhaustively. Second, the mix of individual level and organizational level elements concerning the whole information systems success model leads to misinterpretation and false accumulation of empirical knowledge. Research in this area will have to clearly separate organizational level information systems success and individual level information systems success, leading to distinct theoretical perspectives, models, and hypotheses development, which makes it necessary to completely revise the existing body theoretical contributions on information systems success (especially on an organizational level). This also relates to the criticism of Prof. Dr. Peter Mertens (Buhl et al. 2010), who validly points out that professional information systems are installed to support an organization, hence, its success should be assessed related to its organizational impact and organizational effectiveness. 2.2. The Divergence of Adoption, Continuance, and Discontinuance Research The second theoretical contribution of the thesis, especially regarding Chapter 6, is the provision of an organizational level continuance framework. In this effort, it was possible to integrate elements of the discontinuance framework (Furneaux and Wade 2011), which showed to be important variables in explaining the variance of the dependent variable. In this case, it was shown that variables, which influence discontinuance intention in latter stages of the information systems lifecycle, are also meaningful even in an early stage after adopting the system. It is clear that the importance of variables varies between the different stages of the information systems lifecycle or adoption scenarios. For instance, it makes a difference if a system is adopted and the system to be replaced is highly embedded within an IT infrastructure or an information system is newly added on top of an existing infrastructure. In the first case, the organization might decide not to replace the system due to unpredictability of system failures (e.g. Furneaux and Wade 2011), whereas in the second case this disposition does not play a role. Hence, clearly differing between these stages is an imperative for information systems research (especially when considering cross-sectional research design), which is not considered even in the case of recent top publications (Jeyaraj et al. 2006), where CONCLUSION 136 REFERENCES Ajzen, I. 1991. “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes (50:2), pp. 179–211. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. 2010. “A View of Cloud Computing,” Communications of the ACM (53:4), pp. 50–58. Benlian, A., and Hess, T. 2011. “Opportunities and Risks of Software-as-a-Service: Findings from a Survey of IT Executives,” Decision Support Systems (52:1), pp. 232–246. Benlian, A., Koufaris, M., and Hess, T. 2011. “Service Quality in Software-as-a-Service: Developing the SaaS-Qual Measure and Examining Its Role in Usage Continuance,” Journal of Management Information Systems (28:3), pp. 85–126. Buhl, H., Mertens, P., Schumann, M., Urbach, N., Smolnik, S., and Riempp, G. 2010. “Letter to the Editor: Statements on the Contribution by Urbach et al. from Issues 4/2009,” Business & Information Systems Engineering (52:2), pp. 109–114. DeLone, W., and McLean, E. 1992. “Information Systems Success: The Quest for the Dependent Variable,” Information Systems Research (3:1), pp. 60–95. Delone, W., and McLean, E. 2003. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems (19:4), pp. 9–30. Furneaux, B., and Wade, M. 2011. “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly (35:3), pp. 573–598. Gable, G. G., Sedera, D., and Chan, T. 2008. “Re-Conceptualizing Information System Success : The IS-Impact Measurement Model,” Journal of the AIS (9:7), pp. 377–408. Jeyaraj, A., Rottman, J. W., and Lacity, M. C. 2006. “A Review of the Predictors, Linkages, and Biases in IT Innovation Adoption Research,” Journal of Information Technology (21:1), pp. 1–23. Moore, G. C., and Benbasat, I. 1991. “Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation,” Information Systems Research (2:3), pp. 192–222. Petter, S., DeLone, W., and McLean, E. 2008. “Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships,” European Journal of Information Systems (17), pp. 236–263. Seddon, P. B. 1997. “A Respecification and Extension of the DeLone and McLean Model of IS Success,” Information Systems Research (8), pp. 240–253. Shannon, C. E., and Weaver, W. 1949. The Mathematical Theory of Communication, Urbana: University of Illinois Press. Urbach, N., Smolnik, S., and Riempp, G. 2009. “The State of Research on Information Systems Success – A Review of Existing Multidimensional Approaches,” Business & Information Systems Engineering (1:4), pp. 315 – 325. VITA 137 VITA Sebastian Walther, born 1983 in Speyer, went to school at the Copernicus Gymnasium in Philippsburg, where he made his Abitur in 2003. After a year of civil service at the hospital in Bruchsal, he studied industrial engineering at the Karlsruhe Institute of Technology (KIT), former University of Karlsruhe, and at the Singapore Management University. In Decembre 2010 Mr. Walther started as doctoral candidate at the Chair for Information System Management at the University of Bayreuth, Germany, with visiting scholar positions at the Queensland University of Technology (9/2012-12/2012), Australia, and the University of St. Gallen (2/2013-7/2013), Switzerland. During his studies at KIT he worked as an intern for several global companies, such as PricewaterhouseCoopers, SAP and A.T. Kearney. LIST OF PUBLICATIONS 138 LIST OF PUBLICATIONS List of Publications Citation VHBJourqual WI Orientieru ngsliste Walther, S., Markovic, I., Schuller, A., and Weidlich, A. 2010. “Classification of Business Models in the E-Mobility Domain,” In Proceedings of the 2nd European Conference Smart Grids and EMobility, pp. 35–42. N/A N/A Walther, S., Eymann, T., and Horbel, C. 2011. “A Service-Dominant Logic Based Service-Productivity Improvement Framework,” In RESER Conference. N/A N/A Walther, S., and Eymann, T. 2012. “The Role of Confirmation on IS Continuance Intention in the Context of On-Demand Enterprise Systems in the PostAcceptance Phase,” In Proceedings of the 18th Americas’ Conference on Information Systems. D B Walther, S., Plank, A., Eymann, T., Singh, N., and Phadke, G. 2012. “Success Factors and Value Propositions of Software as a Service Providers - A Literature Review and Classification,” In Proceedings of the 18th Americas’ Conference on Information Systems. D B Walther, S., Eden, R., Phadke, G., Eichin, R., and Eymann, T. 2012. “The Role of Past Experience with On-Premise on the Confirmation of the Actual System Quality of On-Demand Enterprise Systems,” In Pre-ICIS Workshop on Enterprise Systems in MIS, Orlando. N/A N/A Walther, S., Phadke, G., and Eymann, T. 2012. “The Service-Productivity Learning Cockpit - A Business Intelligence Tool for Service Enterprises,” In Bayreuther Arbeitspapiere zur Wirtschaftsinformatik. N/A N/A Wieneke, A., Walther, S., Eichin, R., and Eymann, T. 2013. “Erfolgsfaktoren von On-Demand-Enterprise-Systemen aus der Sicht des Anbieters - Eine explorative Studie,” In Proceedings of the 11th International Conference on Wirtschaftsinformatik, Leipzig. C A Dörr, S., Walther, S., and Eymann, T. 2013. “Information Systems Success - A Quantitative Literature Review and Comparison,” In Proceedings of the 11th International Conference on Wirtschaftsinformatik, Leipzig. C A LIST OF PUBLICATIONS 139 List of Publications (Continued) Citation VHBJourqual WI Orientieru ngsliste Walther, S., Sarker, S., Sedera, D., and Eymann, T. 2013. “Exploring Subscription Renewal Intention of Operational Cloud Enterprise Systems - A Socio-Technical Approach,” In Proceedings of the 21st European Confence on Information Systems, Utrecht. B A Walther, S., Sedera, D., Sarker, S., and Eymann, T. 2013. “Evaluating Operational Cloud Enterprise Systems Success: An Organizational Perspective,” In Proceedings of the 21st European Confence on Information Systems, Utrecht. B A Walther, S., Sarker, S., Sedera, D., Otto, B., and Wunderlich, P. 2013. “Exploring Subscription Renewal Intention of Operational Cloud Enterprise Systems - A Stakeholder Perspective,” In Proceedings of the 19th Americas’ Conference on Information Systems, Chicago. D B Phadke, G., Buchholz, S., Volz, B., Walther, S., Niemann, C., and Eymann, T. 2013. “An Agent-Based Simulation Tool for the Evaluation of Surgical-Operation Schedules,” In Proceedings of the 9th Conference of the European Social Simulation Association (ESSA). N/A N/A