Designing a Simple Folder Structure for a Complex Domain

An Interdisciplinary Journal on Humans in ICT Environments ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 7(3), November 2011, 216–249 216 DESIGNING A SIMPLE FOLDER STRUCTURE FOR A COMPLEX DOMAIN Abstract: In this paper I explore a case of designing a simple folder structure for a new e-learning software program for a university study program. The aim is to contribute to the theoretical base for human work interaction design (HWID) by identifying the type of relations connecting design artifacts with work analysis and interaction design processes. The action research method was used, with the author in a double role as university researcher and project manager of a developer group within the university. Analysis was conducted through grounded theory, inspired by the HWID framework. The findings support the use of a holistic framework with asymmetrical relations between work analysis and design artifacts, and between design artifacts and interaction design. The paper concludes with suggestions for modifying the general framework, and recommendations for a HWID approach to design artifacts. Keywords: human work interaction design, action research, grounded theory, work analysis, interaction design. INTRODUCTION Design artifacts contribute to the outcome of design and development processes, as well as to a greater understanding of the work itself and how people and organizations go about their work. In this paper, I aim to demonstrate how a qualitative methodology can highlight the nuances that matter for designers who are the stakeholders in a team involved in designing and decision making. Taking the case of a simple design of a folder structure in an e-learning system, I attempted to de-layer the various intragroup communications to unveil relations between the artifact and the dialogues that take place between the participants during the development. The analysis involved using grounded theory (Strauss & Corbin, 1998), with the explicit goal of modifying and adding to the general HWID framework proposed by Orngreen, Pejtersen, and Clemmensen (2008). The HWID framework relates work analysis, design artifacts, and interaction design processes. Applying this framework to an empirical case may involve various analyses. First, © 2011 Torkil Clemmensen and the Agora Center, University of Jyväskylä URN: NBN:fi:jyu-2011112211713 Torkil Clemmensen Copenhagen Business School Denmark Designing a Simple Folder Structure 217 work analysis (Button & Sharrock, 2009; Wilson & Corlett, 1990) may, for example, include an analysis of the organizational usefulness of the future design. This may involve the analysis of meeting agendas and resumes, consultant reports, organizational content templates and policies, interviews with key individuals in the organization, and other methods (Preece, Rogers, & Sharp, 2007; Rasmussen, Pejtersen, & Schmidt, 1990; Wilson & Corlett, 1990). Furthermore, investigating the different kinds of work procedures that the new design will support may include task analysis. Second, interaction design (Preece et al., 2007) may include analysis of the individual usefulness of the future interaction design by creating conceptual models, that is, explicit ideas about how future users should interact with the new design. Such uses could then be illustrated through scenarios. Furthermore, analysis of who the future users are may involve the construction of personas, that is, ideation of fictive users who represent a target group of the new design. Finally, analysis of the users’ mental interaction with the new design may include usability tests. However, in contrast to the many techniques available for work analysis and interaction design, the relation between work analysis, interaction design techniques, and the design artifact itself has not received much systematic treatment. Therefore, I explicitly explore and interpret in this study how design artifacts connect with work analysis and interaction design processes. The connection between interaction design and work analysis occurs through a series of separate analyses. However, I wanted to explore an approach that focuses on the type of relations that bind work analysis and interaction design together via design artifacts. The focuson-the-relations approach cover questions such as: Are the words, concepts, and other elements that are used in the design sketch taken from the work analysis? Does the design sketch convey the moods and feelings that the work analysis suggested? Does the design sketch illustrate how a task is supported? Do usability tests show that users find the design artifacts effective and efficient, thus providing them a good user experience? It may also include analysis of how the design as sketched should be maintained and how it will be compared to competitors’ choices of similar designs. Many other analyses are possible as well. What may be critical, however, is not to reduce the understanding of how work analysis and interaction design are connected to a series of method steps, but instead to see it from a holistic perspective. Thus the research question is: What types of relations are needed to connect work analyses and interaction design in the design of a simple artifact for a complex work domain? In the rest of the paper, I first provide a theoretical background for the research, and then describe the research methodology as an action research-oriented qualitative case study using grounded theory. After that I present the analysis of how the developer group in the study approached work analysis and interaction design through organizational analysis, task analysis, scenario development, and usability testing. In addition, I note how these analyses were applied in the discussions and interpretation of sketches and prototypes that were designed and used during the development of the folder structure, and how the developer group’s use of design sketches reflected possibilities for supporting different user groups’ interaction within their various work, learning, and life contexts. I conclude with lessons learned from the case, and provide further advice on how to conceptualize the process of connecting work analysis and interaction design with a focus on design artifacts. Clemmensen 218 THEORETICAL BACKGROUND AND RELATED WORK Work Analysis, Interaction Design, and Sketching Work analysis encompasses techniques such as analysis of work activities and work systems, and assessment of the workplace and products used in the work (see, e.g., Button & Sharrock, 2009; Kirwan & Ainsworth, 1992; Wilson & Corlett, 1990). Some system developers have perceived work analysis techniques as independent and not directly related to design (Clemmensen & Nørbjerg, 2004). Combining work analysis with design artifacts is closely related to approaches used in ethnographic field methods in participatory design (Blomberg, Suchman, & Trigg, 1996; Harper, 2000; Siegel & Dray, 2005). There is also an overlap with studies of design cognition, where researchers ask whether the abilities of the designer (e.g., general intelligence, visual abilities regarding imagery and perception, and creativity) influence the usefulness and quality of sketching (Akin, 2002; Hamel, 1995), and study design practice to describe how designers imagine their users during design (Hasdogan, 1996). Interaction design is presented in textbooks as an approach consisting of conceptual models, scenarios, task analysis, persona, think-aloud evaluation, and other user-centered techniques (Cooper & Reimann, 2003; Preece et al., 2007). In addition to being user-oriented, textbook approaches to interaction design also focus on the use of prototypes, storyboards and sketches, which interaction designers see as products or sources of inspiration in the design process rather than the interaction design itself. For example, sketches, such as freehand drawings or low-fidelity prototypes, have been studied for their role in design and have been found to stimulate reflection, particularly in the early stages of design (Oh, YiLuen Do, & Gross, 2004). When moving from analysis to design, that is, from conceptual models to physical design, interaction design relies heavily on iterative testing of prototypes with users of the future product (Preece et al., 2007). A large number of techniques for user requirement elicitation and user tests are available for use in interaction design (Preece et al., 2007). In many of these techniques, communication between stakeholders about user requirements is supported by the use of prototypes, mock-ups, and sketches. In the brief discussion above I stated that work analysis and interaction design partly overlap, but have different key concepts, use of techniques, and relations to design artifacts. Both work analysis and interaction design have been studied a great deal. However, not much has been said about the use of design artifacts, such as freehand sketches or lowand hi-fi prototypes, to connect work analysis and interaction design in one holistic process. Human Work Interaction Design (HWID) Although this study could have been set within several social science approaches to information technology (IT), for instance, information systems development research or design cognition, I chose to set the study within a developing research specialty of human–computer interaction (HCI) that is called HWID (Campos & Campos, 2009; Clemmensen, Campos, Orngreen, Pejtersen, & Wong, 2006; Katre, Orngreen, Yammiyavar, & Clemmensen, 2010). HWID is a comprehensive approach in HCI, and to provide an easy understanding and to illustrate the coverage of this research topic, I developed the model in Figure 1. Designing a Simple Folder Structure 219 Theories, Concepts, Frameworks, Models and Perspectives Methods, Techniques and Tools Field Studies Human Interaction Design Work Demographic characteristics, Education, Profession, Values, Subjective preferences, Skills, Knowledge, Cognitive resources, Emotions Work activities and tasks Work Contents, Goals, Functions, Tools Decisions and processes Collaboration Environment and Context (national, geographic, cultural, social, organizational) Figure 1. The general HWID framework (Adapted from Orngreen et al., 2008). Figure 1 shows the characteristics of humans and work domains and the interaction during their tasks and decision activities, individually or in collaboration. Analysis of users’ work and life, as well as the design of computer-based information systems, has inspired researchers to develop numerous theories, concepts, techniques, and methods. Some have been widely adopted by practitioners; others are used mainly by researchers, and these are naturally part of HWID research. In either case, such supporting concepts obviously influence work and user analysis, as well as the technology design. This is indicated in the top box in Figure 1. Environmental contexts, such as national, cultural, social, and organizational factors, impact the way in which users interact with computers in their work and life to the same extent as the nature of the application domain, the tasks, and the users’ skills and knowledge. The analysis and design of HWID thus necessarily includes these contextual factors. This is indicated in the bottom box of Figure 1. Following analysis of previous HWID studies, Orngreen et al. (2008) identified six main themes that reflect the major concerns that researchers perceive in HWID. These concerns fall into two primary categories:  Within interaction design processes:  encouraging the dialogue between users and designers in the design process;  bridging the gap between HCI and software engineering by working with user requirements and collaboration in software development processes;  supporting communication and design exploration through sketching.  Within work and user analysis:  bridging the gap between work analysis and interaction design through detailed case and field studies and/or action research projects; Clemmensen 220  providing rich contextual user descriptions, including methods to study unpredictable and opportunistic tasks;  broadening the scope of research aims to include social, organizational, and cultural aspects. Although this list presents these themes and problems as distinct, interaction design and work and user analysis are intertwined in practice, as demonstrated in previously published HWID research (Orngreen et al., 2008). In the qualitative study present in this paper, I develop an interpretive, case-based model of how work analysis and interaction design are connected in the minds of the stakeholders. My aim with this research is to enrich and sharpen the model of HWID that is presented in Figure 1. RESEARCH METHOD To answer the research question, I have chosen to do a qualitative analysis of a single case. In the sections below, I present a qualitative case study of the development of the simple folder structure shown in Figure 2 by using action research and grounded theory. The Case: Designing a Simple Folder Structure for a University E-learning System In October 2004 the members of the study board (that is, a committee of students, faculty, and administrators appointed to oversee the academic criteria of a university degree program) at a large university in Scandinavia (Copenhagen Business School;1 CBS) received an e-mail saying, The deans and the university administration have decided that the platform Sitescape now is mandatory for all courses and all students at the university. Therefore you at your study program have to begin using this platform no later than autumn semester 2005.2 The e-mail marked the end of a year-long political discussion in the study board about the value of retaining the old in-house-developed course administration system called DIVE (Døk [datalogi og økonomi] -studiets Informations-, Vejledningsog Elektroniske kommunikationssystem). It also began the transition to the new e-learning system described in this study. Figure 2 shows the Figure 2. A simple design for a complex work domain: The folder structure of a Bachelor study program’s e-learning site. The figure is in Danish; each entry in the folder structure represents a student class. Designing a Simple Folder Structure 221 end of this process, a simple folder structure to be used by teachers, students, and administrators at a bachelor study program in computer science and business administration at CBS. The case period was from January 2005 to January 2006. The bachelor study program at that time had more than 300 students, 50 teachers, and five administrative workers. It was supported by a small group of IT and e-learning experts from the university’s central learning unit, which supported the university’s 12 bachelor study programs. Each bachelor study program was given the liberty and the responsibility to design and implement its own folder structure. To this end, the study board established a developer group. The task of this group was to design for the various user groups a new folder structure that would facilitate design teaching and materials, well as to find an appropriate solution for the organizational memory problem, that is, how to store each year’s activities on the e-learning site. A number of meetings were held in order to design the work. My focus in the analysis was on the relation between work analysis and design artifacts, and interaction design and design artifacts. Outline of Methods Regarding sampling in this study, the rationale for choosing a single case is that the change process from an old e-learning system to a new one in a large university presents a unique opportunity to study the mediating role of design artifacts, such as sketches, within a large, complex organization. Although other organizations, such as commercial enterprises, could have been relevant, I would have had difficulties in obtaining the same kind of access to people and reasoning processes that I could in a public university. Moreover, the case could meet the aim of developing a theoretical base for HWID to address the challenges of HCI in a world where it is more usual to reconfigure and redesign an existing system rather than to develop a totally new system. Migrating to a new e-learning system in a large university happens every day around the globe, and this kind of system change is critical for the development of higher education. Hence I will be able to claim that the developed theory has some face validity. Finally, this case presents a challenge for grounded theory analysis. Compared to a classic qualitative research interview study with a few subjects that basically have the same perspective of the issues studied, this study is methodologically different. This case contains diverse sources, such as e-mails and screen dumps, and involves a large number of people performing various roles. The context of the case was the decision made by the university’s management that the study board was to replace their program’s in-house developed course administration system with the university’s standard course administration system. From this followed the need to design the folder structure of the new system in a way that accommodated or, in some cases, changed the course administration process familiar to the users. Thus, the researcher–situation interface was optimal because the organization’s management (the system owner) supported it. Materials I collected archival data, such as background reports, and (concurrent) exchanges of e-mail. Furthermore, I took notes from meetings, recorded videos of usability test situations, and assembled design artifacts used by the development group. The material was in Danish. Although the analysis involved all of the materials, this paper presents sections of the material that were transcribed and translated into English. All data were stored in paper form as well as scanned and Clemmensen 222 prepared for digital qualitative analysis by making the items primary documents (a proprietary name for the kind of documents or data sources that my qualitative data analysis software could use). This data collection and data management approach was chosen over, for instance, qualitative interviews or diary studies because I was involved personally in creating the change that was studied. Thus this research can be understood—but was not declared as such from the start—as action research, a method in which the researcher plays a double role, that of change agent and researcher of the change process. In this double role, then, I attempted to facilitate and attain the large-scale change in CBS’s technology usage as well as theoretical generalization. Participants The participants in this study comprised the developer group, drawn from the system’s expected users. The users of the system were initially identified as IT and e-learning experts from the university’s central learning unit (learning lab), students, administrative staff, and teachers. Since the bachelor study program was a computer science program, all users possessed basic IT skills. The teachers as a group (50 teachers) included both staff faculty and external lecturers, with diversity in age (25–65) and near equal distribution of males and females. Although most teachers at that time had been teaching for several years, some (e.g., PhD students) had taught only one or two courses. The administrative staff (5, 30–55, 80% female) comprised skilled office workers, employed to fit into the university’s administration, and hence with a working overview of the administration. The students (300) were mostly male (80%) and aged in their early 20s. While some students possessed computer and design skills comparable to or better than the other user groups, they lacked or were seen to lack an understanding of the university’s organization and purpose. In contrast, the IT and e-learning experts from the learning unit (5; 28–55, 60% male) were administrative staff with an academic background and a special interest in e-learning. They were employed with the specific purpose of promoting the university’s use of e-learning, and hence were considered experts with special access to both the university’s policies and technologies and pedagogy for e-learning. The study board recruited participants for the developer group from these user groups. There criteria for the recruitment were that a participant should be interested in the new system and that all user groups should be represented. Consequently, the developer group consisted of 11 individuals: four students, three administrators, two teachers with coordinator responsibilities, one IT and e-learning expert, and a chairperson (myself). This group was reasonably representative for all users of the new system with regard to age and gender, except that the group (as intended) consisted of individuals with a larger than average interest in the system. The group dynamics in this developer group resulted perhaps in even more eagerly expressed viewpoints than were necessary or usual in comparable kinds of system design. Therefore, the data were abundant. Data Analysis For the analytic framework, I used the general HWID framework presented in Figure 1, but I focused only on the part that concerns the relation between work analysis and interaction design. Three elements of the relations from the general HWID framework generated the primary interest: work activities and tasks, decisions and processes, and collaborations. The rationale for the analysis strategy was that these three relations were very detailed in the framework. Designing a Simple Folder Structure 223 Therefore, I used the general HWID framework as an initial source of inspiration for conducting a grounded theory analysis. Three substeps in the grounded theory analysis were performed as open, axial, and selective coding (Strauss & Corbin, 1998). I collected a total of 133 distinct data sources (texts, scanned documents and notes, videos, audios), of which I used 105 in this analysis (the remaining 28 sources were either not of interest or redundant data). The first substep in the postcase analysis had, in fact, begun already during the case, and it consisted of identifying and naming the concepts of interest to the investigation (open coding). I found concepts of interest through an iterative process using an emerging list of codes, and by listening to and looking for related segments in the data that seemed to concern the concept(s). In this way, the relevant data sources were segmented into meaningful units, and the segments were coded into categories that again were refined during the analysis by revisiting the segments of the data sources. The segments were mostly on the section or paragraph level, and the total number of segments for the 105 data sources was 151, giving an average of a little more than one segment per data source. This reflected that many of the data sources were one meaningful unit, for example, an e-mail, and should not be divided into several segments in the way that interview documents often are. In the analysis I focused on the work analysis and interaction design processes, and the different design artifacts. These were coded into 13 main categories that on average were grounded in 12 data segments. The next substep was the categorization of related phenomena (axial coding). Here I looked for relations between categories and the consequences thereof. In this substep, I visually inspected networks consisting of the 13 coded (and all associated) segments, then did co-occurrence searches of those codes (categories) that shared at least one segment and, based on the segments, named the relation. The final substep in the analysis involved looking for a common theme for all of the categories, to find a core category (selective coding) and its relations to other categories, and perhaps refine and develop these. The main category in the analysis was Design Artifacts. Because this category was in a sense given beforehand, I will focus the presentation of results on the relation between the subcategories of design artifacts, for example, design sketches and prototypes, and how these relate to the subcategories of work analysis and interaction design. The results are presented below in the Findings section. My presentation of transcript excerpts and analysis is governed by three rules in line with Dahler-Larsen (2008): authenticity (display data in their original form to force the reader to diagnose on the basis of the original situation), inclusion (displays not just examples, but rather the data set itself), and transparency (displays are explained, axes and dimensions made clear to the reader, and data sorting should be intuitive and easy to understand). Moreover, the presentation of the analysis involves using a network model: introducing the key concept design artifact, then the major subconcepts of work analysis and interaction design with their corresponding codes, and then the types of relations that connect these together. Figure 3 illustrates this process: The figure shows a number of codes presented within text boxes and their various relations. A relation is indicated by co-occurring quotations (i.e., two quotations embedded in, enclosing, overlapping, following, or preceding) that thus connect two codes. In the following text, each type of relation is illustrated with one or more quotations. Although the aim was to show every related quotation, I removed similar (i.e., redundant) quotations for the sake of clarity. Two numbers refer the quotation to its data source: The first number specifies the data source, and the second number indicates the quotation number from within the data source (see the Appendix for the list of data sources). Clemmensen 224 Time Work analysis Design artifact Interaction design Figure 3. The model of HWID in this study of a case of designing a simple folder structure for a study program. The numbers after the text in the boxes indicate, first, the total number of related quotations arising from the 105 source documents (note that since each quotation can be related to more than one code, adding up the numbers in the boxes does not give the total number of 151 quotes), followed by the number of relations that element has to the other elements in the figure documents. With respect to the confirmability of the analysis, I used time away from the data as a means of confirming the quality of the coding process. After a significant period of not reviewing the data, I returned to the original data and repeated anew (i.e., without preconceived categories or approaches to the data) a thorough coding process. The first and second coding did have some significant differences. For example, the focus of the first coding process, as reported in Clemmensen et al. (2006), was strictly on the relation between the work analysis and the design sketches. However, the focus of the second analysis, reported here, was broadened and included the relation between the work analysis, the designs is associated with is cause of archive is cause of archive is associated with creates emotions about reflections on strategy specifies how is associated with creates emotions about is part of creates conditions for designs specifies how creates conditions for~ reflections on The content templates for new system {18-3} Establish goals (design and other) {35-3} Organizational analysis {22-3} Elicit expertise from center of organisation {17-4} Give workgroup members acces to participate... {17-3} Folder structures imple mented {13-4} Scenario writing and Usability testing {21-1} Involve all teachers and students and other stakeholders {9-2} The new system is operative {22-3} Evaluation of the first semester {8-1} How to use the old system's information {13-2} The sketches {46-4} The prototypes {23-5} Designing a Simple Folder Structure 231 programmer from this group became involved to help the developer group and the system owner to decide where and how to archive the data from the old system. Two quotations—one each from the programmer and the IT manager—shed light on the “archive” relation between the work analysis (how to use the old system information) and the artifact (the implemented folder structure): …There are some issues that we need to clarify. The division in [the old system] is also used in [the new system]. Every folder in [the old system] becomes a “workspace” in [the new system]. News etc. will not be transferred to the archive, only data and documents etc…. There will be no access control on workspaces. All workspaces will have same rights [to access] …registered users can read, administrators can read and write…. (31:1) From the programmer’s point of view, the old system’s information structure should be applied directly to the new folder structure. This work analysis was perhaps too simple, and resulted in the old system’s archival data being the topmost and hence the first thing seen in the new folder structure. During an e-mail conversation with the programmer, the IT manager gave her opinion on how things should be, and how this would be experienced by the users: …Let us talk about structure when we know who are going to do the other stuff…but I will suggest that we call it archive from [old system name]. Then it will be on top [due to the “a” in archive] and under it the new folders will come… where there are several folders, the newest year will be at the bottom of the list due to the number… I think that will give a sufficient overview. (83:1) The IT manager and the programmer in the two quotations above produced a new kind of work analysis with focus on the use of archival data from the old system. This work analysis suggested that how to archive data was somehow related to the implemented folder structure. However, both the IT manager and the programmer expressed a strong bias towards their own analysis of the work, and they did not consider the already implemented folder structure as something essential to adhere to in their design of the archive. Instead they invented their own, new version of the folder structure, based on the structure in the old system. This illustrates how, even late in the design, a new work analysis may enter the design process, as in this case when a stakeholder group emerges more strongly (in this case the technical group of the IT manager and the programmer). This new work analysis may relate to the design artifact in ways that exert considerable pressure for rethinking the design sketches and prototypes. Furthermore, when eliciting expertise from the central administration of the organization as part of the work analysis, it turned out that considerations about how to archive data (made by experts from the central IT support unit) were part of the organization wide content templates for new folders in the e-learning system (made by IT and e-leaning experts from the central learning unit). The history of using similar folder structures in the organization was indicated by the experts’ work analysis and expressed in the content templates. This analysis was very convincing to the administrative staff: …it became clear during the meeting that the ideas that we had at our teacher meeting will not work in any way in the new system. It will be a waste of time to suggest a folder structure that is not optimal, if you ask those who have worked with the system for many years…. I am 100% convinced that [the teachers’ suggestion] will not be the final design…. (74:1) Clemmensen 232 Early in the design process, the administrative staff accepted the perspectives of the IT and e-learning experts. The work analysis completed by the experts showed that the content template design for the folder would benefit the administration. …It has been shown to make the administration less vulnerable... sickness, leave, change of personnel often are the cause of a knowledge loss [that] it takes years to overcome… this damages the study’s administration and reputation among teachers and students…with a detailed and centralized documentation [as in the new system], the administration will stand stronger in the ability to deal with students and teachers…. (96:1) …The structure. In the upper-level discussions for general information, news, teacher forum, administrative forum, study board, etc. Courses listed as workspaces with numbers. At second level, relevant discussions are listed with indication of the year…. (97:1) The work analysis performed by the experts gave an optimistic view of the content template provided by the experts, as illustrated in the quotations above, and promised several benefits for the administration. Similarly, the content template was presented as a finished solution, with no analysis of how the messy reality of organizational practices had changed the content template in real-life situations. In summarizing the Archival relation, the case analysis indicated that this relation appeared only on the work analysis side, not on the interaction design side. How to archive data was an issue significant to the central IT unit’s and central learning unit’s experts, and one about which no other stakeholders had much to say, and thus the experts’ view significantly influenced the final artifact. This illustrates that some kind of symmetry in the relation between work analysis and the design artifact, on one side, and the design artifact and interaction design, on the other, should be attempted in HWID. Work Analysis Focus on “Strategy” in Relation to the Design Artifact Strategy, in the form of evaluation, was the focus of the work analysis following the first semester that the new system was in operation. When the system was put into operation, it became apparent that a strategy for using the system was needed, something that was not addressed earlier in the process. The next quotations, which occurred between the evaluation of the first semester and the new system becoming operational, resulted from questions about what the old strategy had been and what would be the new strategy for using the system. Work analysis showed that the two systems had different strategies for folder structures: …[the new system] has inherited a deep folder structure from [the old system]. Other studies at the organization apply a flat folder structure which is more user friendly and makes archiving data more easy. Should we change [the new system]’s folder structure from the deep to a flat structure? (18:1) In this citation, three designs are compared: the old, the new, and other designs of the new system already in operation within the organization. When the new system was put in operation, the previously unaddressed discrepancy between the (old) system, as well as other comparable systems in the organization, became explicit. The obvious action was to ask for development or presentation of a strategy, as illustrated in the following citation. Designing a Simple Folder Structure 233 ...perhaps it would be a good idea to present the study’s strategy for e-learning, if there is such a strategy…. (19:1) The developer group manager thus asked for a strategy, even while expressing doubt about the existence of such a strategy. Given the late stage of the design process, this doubt was surprising. As it turned out, doubt about the strategy reflected a problem not only for the developer group, but demonstrated as well a general lack of clarity in the work organization. This is illustrated in the following quotation, where the project manager explicitly calls for clarification regarding the different systems within the organization. …[there is a] need for clarification of the division of tasks in [the new system] between study management, study boards and department! (24:1) Apparently the environment for the new system was not analyzed prior to the existence of the new system. However, individual users did not have major doubts about the strategy. The following citation illustrates how a teacher felt satisfied with the use of the new system. …I cannot come to the meeting on Thursday, but I have been fully satisfied with [the new system] - which I basically only have used to publish documents. (49:1) Hence, when the new system was put into operation, it became clear that the work analysis was inadequate on an organizational level, and also in relation to the larger environment of the system. Furthermore, there was no strategy relation between the interaction design and the artifact. Thus it was never clear in the old system what kind of interaction users were expected to have with the system. As the above quotation illustrates, the new system revealed that some teachers may not have held similar expectations as other stakeholders, and thus were satisfied to use the new e-learning system as a simple publishing system. To sum up, the Strategy relation was asymmetrical in its focus on the work situation. In this case, no strategy had been prepared for how users were going to interact with the system. Ideally, however, work analysis and interaction design should both have a strategy relation with the new system. Work Analysis “Is Associated With” the Design Artifact The relation Is Associated With was, similar to the Strategy relation, asymmetrical in the sense that it was only present from the work analysis perspective and not from the interaction design perspective. The participants expressed their thoughts about both the prototypes and how to elicit expertise from the IT and e-learning experts, as reflected in the next nine quotations. The elicitation of expertise was associated with the prototypes by presentations, instructions, discussions, questions, and physical handling of the prototypes. Something to Present …otherwise I can present it…. (52:1) Something to Be Instructed in …I hereby confirm our agreement about the instruction in [the new system]…. (62:1) Clemmensen 234 Something to Discuss with and Learn from Experts Once again thanks for a constructive meeting [on prototypes of e-learning systems], which gave us several new perspectives on our project. (70:1) Something to Discuss with and Learn from Students, Teachers, and Administrators If you want to have a look at [the new system] already now, type www.e-cbs.dk and click on cand.merc.dat, but beware, it is rather empty right now.... We cannot, however, just jump from [the old system] to [the new system] without a bit of preparation from the perspective of the students, teachers, coordinators and administrators. Therefore, the study board will prepare templates for all courses and invite you to a meeting about the use of [the new system] as a pedagogical tool in your teaching. (5:1) Something to Ask Questions about Is there a smart way to create the same set of questions on several forums? (76:1) Something to Handle Physically: Make sure that a mouse is with the laptop, as there are many clicks in [the new system] (84:1) Something to Experiment with You have all administrator rights and may virtually play with everything ... so you can for yourself experiment with design, graphics and functionality. (73:1) The last quote was one of two quotations in which participants expressed associations between eliciting expertise and the content templates. Thus, eliciting expertise was the first set of ways in which work analysis was associated with the prototypes of the new system and with the content templates. The second set of associations was expressed in quotations about the relation between the content templates and work analysis, and concerned who owned the design artifacts. Something Related to Different Stakeholder Groups and Work Tasks in the Organization Following factors:-“student view,” -external teachers and coordinators’ perception of existing Folder Structures -need for teacher dialogue across class and years -special folder structures, such as integration task, didactic forum, etc. (56:1) Something Owned by Someone In/Outside the Organization It must be said that it was pretty obvious that it probably will be me who will stand for it here; I will just have to discuss it with my boss…. therefore suddenly my strong involvement :-) (74:1) In this quotation, a member of the administrative staff expressed sudden interest in the process because she felt that she was being forced by her boss to take ownership of the prototype. All in all, the relation Is Associated With between work analysis and design artifacts concerned eliciting expertise in a broad sense, as well as concerned the ownership of the artifacts. Designing a Simple Folder Structure 235 Work Analysis “Is Cause Of” the Design Artifact Work analysis was the “cause of” the sketches in two ways: the elicitation of expertise and the organizational analysis. The causal relation between the elicitation of expertise and the sketches was expressed in the following three quotations. The experts had ideas about how sketches should look, and how the organization would react to the design artifact. …some of the themes in the article would be appropriate to discuss with the students if we are talking about more than just structuring of the content. (78:1) I have tried to give a picture of a group of young university students' feelings for digital aesthetics. (134:1) The elicitation of expertise was a cause of sketches to the degree that the experts’ designs were seen as versions to choose between (and not as, e.g., opportunities for dialog). Finally, I just confirm that the study board has today decided to go with version 3. (84:1) The second relation, the organizational analysis as a cause to have sketches of the new folder structure, came in several versions. Organizations’ E-learning Strategy …Speaking of the talk about e-learning ... Do you know about this site with text and background papers for the CBS e-learning strategy? (14:1) Students’ Organizational Analysis This is a “rar” file with all documents regarding [the old system] that Sebastian and his group have written.... (45:1) Management Report [the old system]. The next step. Figure 6.1: explanation of the rich picture (shows [the old system] as embedded in the organization). (100:1) It is too hard to figure out how things hang together behind the curtain, and no efforts are made [by the study program’s management] to recruit the study program's many experts to help. Information systems exist only in the shared consciousness of the involved. The Front for [the old system]’s improvement (FFDF) is an organization formed to facilitate students’ at [the organization’s] lives by improving their intranets called [the old system] .... FFDFs ultimate goal is to take over the operation of [the old system]. To give power over the system to a group of students that engage in how their study’s communication and information platform looks and works…. (142:1) Organizational Leaders Organizational Analysis …I frequently find it useful to be able to go back and see what actually happened at this or that course in the past. (46:1) Administrators’ Organizational Analysis …the structure used in MSc.’s new zone may eventually also be used on our study... (57:1) Clemmensen 236 System Developers’ Organizational Analysis …each working group (students, administrators, teachers) will submit a proposal for a folder structure of [the new system]. (144:1) Summing up, the organizational and work analysis from the IT and e-learning experts’ point of view was not the only influence on the sketching. Indeed, a number of other stakeholders’ perspectives included in the organizational and work analysis directly influenced the sketching of the new folder structure. Interaction Design “Creates Emotions About” Design Artifacts A number of relations occurred only from an interaction design perspective and were thus asymmetrical. One of these relations was Creates Emotions About. The interaction design techniques evoked emotions in the participants about the design artifacts. Six quotations (two presented here) illustrated how the relation between the fact that the new system was in operation and the wish to involve all teachers and students and other stakeholders was about the creation of emotion. REMINDER: [THE OLD SYSTEM] IS DEAD! - Invitation to a meeting on [the new system]. (72:1) In this quotation the project manager declared the old system dead, as if it had been a living entity. The user involvement that was a central feature of the interaction design created emotions around the old and the new systems. Furthermore, the prototypes were objects of emotion, since scenario writing and usability testing were emotionally engaging for the involved stakeholders. This involved also one of the IT and e-learning experts, who clearly was happy to participate in on-site user testing with students, teachers and administrators: I'm on!. (54:1) Although this emotion-creating capability does not appear in Figure 3 to have been part of the work analysis, there were indeed indications that the emotions surrounding the software were important parts of the work. For example, the logo of the old system was a picture of a scuba diver because the system acronym was DIVE. People in the organization felt so strongly about the logo that it was kept for the new system. Another example was the strong dissatisfaction with the old system, and a wish to be involved with the design of the new system, which motivated the student members of the developer group, as obvious from the existence of a special student organization with the sole purpose of improving the e-learning system. Interaction Design “Specifies How” Design Artifacts Should Be The relation Specifies How was an asymmetrical relation present only in the interaction design perspective. When, as part of the interaction design, the IT manager wanted to give the developer group’s members access to participate, she did this by specifying how the folder structures were implemented. The Admin area now has a description of the procedure, from the moment when information about the student is being typed into [an administrative system], and until he Designing a Simple Folder Structure 237 or she [the student] has access to [the new system] (if these access rights are set correctly there). (20:1) In this quotation, the IT manager explained the procedure for giving members access to the implemented folder structures. Similarly, the establishment of design goals as a step forward in the interaction design was done by specifying how the sketching would proceed. This relation was expressed in 12 quotations, a few of which are provided here: Assignments Given …Deliberations of the committee must culminate in a report to the study board that contains one or more proposals for templates for structuring [the new system] for different user groups: secretarial, teacher, student, coordinators. (1:1) User Identities Constructed …Christian comes to the meeting and presents the students' case. (7:1) I would in fact like that she becomes the person at the department who knows [the new system and] who can give access rights to people and show them how it works (if it then becomes necessary). (30:1) Did not answer your mail yesterday. I [administrator] will attend the meeting on…” (35:1) It sounds really exciting. Thanks for the offer! I [expert in the use of such systems] am eager to help. (80:1) I am about to create a discussion forum at [the new system], where we can share [the old system] information in the project. In this connection, I need your official mail addresses so I can create you and give you access. (81:1) It is clear, then, that the Specifies How relation was asymmetrical from the interaction design side. It explained procedures, gave assignments, and constructed user identities. Interaction Design “Designs” Artifacts The Design relation connected interaction design with the artifacts to be designed. While interaction design and artifact design may be perceived by some as synonymous concepts, I argue that artifact design always involves more perspectives than interaction design does, so it makes sense, at least analytically, to talk about a relation between the two development streams. The following two quotations illustrates the “design” relation between the prototypes (a design artifact) and the wish to give the developer group members access to participate (an interaction design technique). Establish a Place to Share Sketches and Other Design Artifacts See figure 6. (121:1). This quotation is a visual quotation (see Figure 6) that shows members of the developer group and their access rights in the system. Giving members of the developer group access to a place in the new system formed their view of what the system could look like. Clemmensen 238 Figure 6. Screen dump of members of the developer group and their access rights in the system. Establish a Way to Read and Critique Design Artifacts within the Group Prototypes high fidelity; “hierarchical folder structure”; “flat folder structure” – “flathierarchical combined”; “Reading the prototypes -group discussion of plus and minus for each prototype.” (139:16) Similarly, there was one quotation in which a member of the administrative staff expressed the design relation between the sketches and a wish to give developer group members access to participate. Establish the Importance and Status of Sketches …then it must be left to the people who are responsible for the final version [of the folder structure], they must include all our requirements in the final versions. This is merely my opinion, but since I am only on the sideline, I would not take a strong position on anything, we may figure this out on [our meeting] Wednesday. (57:2) To sum up, the Design relation established a place and method to share and create sketches. It also addressed the motivation to sketch the new folder structure. Designing a Simple Folder Structure 239 Interaction Design “Creates Conditions For” Design Artifacts The relation Creates Conditions For was a symmetrical relation only from the interaction design perspective. It reflected how the establishment of design goals created conditions for the content templates for the new system. One quotation illustrated how a participant’s creation of a standard folder structure for file sharing and discussion in the system created specific conditions for the developer groups’ discussions and design of sketches and prototypes. …I have now created a team workspace for committee members [the developer group] and presented a little information material…. (92:1) In 11 quotations, the participants expressed how the establishment of design goals created conditions for making content templates relevant for the new system. Here are four representative quotations: Demanding ... in consultation with the deans, who decided on the platform, [the new system] is now the obligatory platform for all courses and all students at CBS.... There is now much knowledge about [the new system], a knowledge which you can draw on when the platform shall be designed and implemented. I suggest that you contact the IT department and Learning Lab for further discussion on how this can be done…. (3:1) Demoing I can show you some other study’s zones, etc.… (13:1) Preparing You are invited to join a committee of the DØK Study Board to prepare the transition from [the old system] to [the new system]. (17:1) Requiring solutions Goals for the committee’s work: [the old system] shuts down in mid-September. All reexams will be run in [the old system]. (21:1) All in all, the relation Creates Conditions For connected the establishment of design goals with the design artifacts by applying standard designs, demanding that a process of system development be started, demonstrating other systems as conditions for the new system, preparing for the change from one system to another, and requiring solutions for new systems within a given time frame. In an overall summary of the analysis of the case, I found a variety of relations between work analysis and design artifacts, and between interaction design and design artifacts, as shown in Table 1. Clemmensen 240 Table 1. Relations Between Design Artifacts and Work Analysis and Interaction Design. Relation Frequency Related to Work Analysis Related to Interaction Design Is Part Of 1   Strategy 1  Archive 2  Creates Conditions For 2  Creates Emotions About 2  Designs 2  Is Cause Of 2  Reflections On 2   Specifies How 2  Is Associated With 3  Total 19 DISCUSSION The research question was, What types of relations are needed to connect work analyses and interaction design in the design of a simple artifact for a complex work domain? Two main findings arise from this case analysis: 1. During the design of a simple artifact, such as a folder structure for a large organization, different relations between the work analysis, interaction design, and design artifacts are expressed (see Table 1). 2. The pattern of relations in the HWID case studied here is not as symmetrical as expected, but rather asymmetrical (see Figure 3). This suggests that the current understanding of HWID (see Figure 1) should be modified to accept that design artifacts connect, but have different relations to, work analysis and interaction design. Regarding the first finding, the identification of this complexity supports the idea of having an encompassing, holistic, general HWID framework (see Figure 1) that can help to combine work analysis and interaction design explicitly. Even though it is beyond the scope of this paper to discuss the relationship between HWID and either of the system development or participatory design approaches, the findings can be seen as a justification for a new complex HWID approach that focuses more directly on the relation between organizational work analysis, interaction design, and very simple IT artifacts than existing system development and participatory design approaches do. Regarding the second finding, only two types of relation are common across the framework: Reflections On and Is Part Of. The balance of the relationships are asymmetrical, as the two last columns in Table 1 show. Hence focusing on either work analysis or interaction design techniques gives only half the picture of the development of the design artifact, and leaves many important relations unexplored. Researchers and work teams need to focus on both Designing a Simple Folder Structure 247 Author’s Note I extend thanks to each member of the developer group for allowing his/her contributions to be part of this study. All correspondence should be directed to Torkil Clemmensen Department of IT Management, Copenhagen Business School, Howitzvej 60, 5.15 DK - 2000 Frederiksberg C [email protected] Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889 www.humantechnology.jyu.fi Clemmensen 248 APPENDIX The 105 (of the 144 Collected) Data Sources Used in the Analysis. P 1 E-mail from project manager to developer group members P 3 E-mail from the organization’s director of administration to (previous) system owner P 5 E-mail from administrative staff to developer group P 7 E-mail from student developer group member to project manager P11 E-mail from the system owner (the head of study) to the administrative staff (the study secretariat) P13 E-mail from IT manager to project manager P14 E-mail from (e-learning) expert to project manager P15 E-mail from study program’s administrative staff to developer project group P17 E-mail from project manager inviting developer group members to become a member P18 E-mail from project manager to study program’s administrative staff P19 E-mail from project manager to system owner P20 E-mail from IT manager to developer group P21 E-mail from project manager to developer group P24 E-mail from project manager to system owner board (study board) P30 E-mail from administrative leader to project manager P31 E-mail from programmer to IT manager P35 E-mail from system administrator or project manager P45 E-mail from students to project manager P46 E-mail from system owner to project manager P49 E-mail from user (teacher) to project manager P52 E-mail from study program’s administrative staff to developer group P54 E-mail from IT expert to project manager P56 E-mail from project manager to developer group P57 E-mail from administrative staff to project manager P58 E-mail from IT manager (central IT expert in the organization) to developers, IT staff, and system owner P60 E-mail from system owner to developers and IT staff P61 E-mail from IT manager to system owner P62 E-mail from study program’s administrative staff to developer group P64 E-mail from study program’s administrative staff to developer project group P66 E-mail from a colleague to the system owner, in their roles as researchers, about the possibilities for writing a research paper on the system development P67 E-mail from system owner to colleague P68 E-mail from a colleague to the system owner, in their roles as researchers, about the possibilities for writing a research paper on the system development P69 E-mail from system owner to project manager (responsible for the design of the new folder structure) Designing a Simple Folder Structure 249 P70 E-mail from (e-learning) expert to developer group P72 E-mail from project manager to organization P73 E-mail from (e-learning) expert to developer group P74 E-mail from study program’s administrative staff to project manager P75 E-mail from one study program’s administrative staff to another P76 E-mail from project manager to IT manager P77 E-mail from student member of developer project group to project manager P78 E-mail from (e-learning) expert to project manager P80 E-mail from (e-learning systems) expert to project manager P81 E-mail from (e-learning systems) expert to project manager P83 E-mail from IT manager to programmer P84 E-mail from IT manager to developer group P92 E-mail from (e-learning) expert to developer group P96 E-learning platform experience document from IT manager P97 IT manager’s description of the organization’s content template for the new system P100 Student report on the old system P117 PowerPoint slides on the use of the new system presented by an expert from the central administration of the organization P121 [Screendump]: Members of the developer group and their access rights in the system P128 E-mail from study program’s administrative staff to developer project group P134 Document on blackboard e-learning platform use from (e-learning) expert to project manager P139 A personal note from the project manager P141 Written annotations on teachers’ sketch P142 Excerpt from Website material on the front for improvement of [the old system]– a student initiative P144 Project manager’s minutes of project meeting