Assessment of the Usability of the "Alpha" Version of the Prototype "Octopus" -Application of the Heuristic Evaluation Model of User's Graphic Interfaces

ASSESSMENT OF THE USABILITY OF THE "ALPHA" VERSION OF THE PROTOTYPE "OCTOPUS"-APPLICATION OF THE HEURISTIC EVALUATION MODEL OF USER'S GRAPHIC INTERFACES VITO CARIOCA [email protected]PBEJA.PT ALDO PASSARINHO [email protected] This article, based on the global logic of dissemination of the "Octopus" project, assumes as its main aims: i) the creation of an instrument to evaluate the application of the usability tests of the users'graphic interfaces (in the alpha version of the prototype), according to the heuristic method of evaluation of graphic interfaces (Nielsen, 1993); ii) to contextualize the usability tests of the resource centre's graphic interfaces, in the frame of the global project evaluation.Considering the body of reasons that guided the development of an instrument to assess the usability of the resource centre graphic interfaces, it is important to bear in mind the aims of the project. These are based on a logical of development of interfaces (human-machine), facilitators of the interaction between the users of "Octopus" and the computer that supports the functions offered by the resource centre. 1. Project Octopus - Transnational online resource centre, in the domain of environmental education 1.1. General assumptions The emergency of new realities in what concerns the diversity of databases supported by technology which require high technological literacy (Norman, 1993) implies the assumption of an attitude of continuous reflection and the creation of mechanisms of access to information. This principle gave us the basis to the project, in the intention of developing a website which allows the access of users from different regions, with common interests, to sources in the domain of environmental education. This was considered an exploratory area, because, for its transversal characteristics, seemed adequate to test the model, but it doesn't exclude the multidisciplinary possibilities of the resource centre. This Resource Centre is called so, because it functions as the tentacles of an octopus, both internally, by providing access to several databases inter-connected (in the prototype only the one related to environmental education will be created), and externally, through the effective contribution of each partner, so as allow the users a broader search. In terms of concrete results to get, its possible identify the basis body of orienting goals: - To create a resource centre of didactics features online; - To create and to customize a database in transnational environmemtal education; - To identify and to characterize the potencial users of the centre, with the intention to produce an adequate interface to the profile of the users; - To search and to evaluate existing media resources in the thematic area. The development of the project essentially consists in the design and administration of a "Transnational On-line Resource Centre", through a Web Site that may help the users to do their tasks faster and with more efficacy, transforming their computer into a truly "cognitive artefact", in the words of Norman (1993). ~This global logic assumes, in the specific context of this project, the following framework: 1. 2. Main steps of the project With a varied partnership and great experience in the areas of intervention, the project, with a duration of two years, began in October 2001. The first year was mainly dedicated to the construction of the Resource Centre, and the second to its application in the Net and consequent evaluation. Picture number 1 specifies the different steps: 2. Assessment of the "alpha" version of the prototype According to the findings foreseen in the "Octopus" project (the creation of typified graphic interfaces according to the end user), we intend to develop an hypermedia application whose interfaces may be sufficiently intuitive for its users, this is, for all who may be interested in acceding and sharing the educational resources in the area of environmental education through the resource centre proposed. The development of intuitive graphic interfaces for the users of the centre is essential for the success of the project, and for the success of the interaction that they may establish with the hypermedia application. The creation and the development of graphic interfaces adequate to the interaction between the users of the hypermedia application and the artefact that supports it is important. Also important is the development of those interfaces in the perspective of a learning environment in or through which one may have access and may share learning resources and contents. In what concerns those resources and contents, one may say that they are quite abundant in the multidimensional WWW. However, they are not, by themselves, a source of learning. They need a "context", as stated Figueiredo (2002): In this perspective, we are interested in the " On-line Resource Centre in Environmental Education" as a metaphor of a context that facilitates the access to materials, resources and learning contents embodied in a hypermedia application whose graphic interface with the user may be usable for the amplitude of the audience it covers. Thus, we assume that the development of graphic interfaces usable for the audience to cover will be an objective to accomplish in order to: i) facilitate the interaction between the users and the system; ii) create a context in which resources and learning contents "become alive". It was based on the development of usable graphic interfaces that we established a methodology for the development of the project. This methodology involves the creation of a first version of a functional prototype for the resources centre (designed by alpha version of the prototype). This prototype will be tested in what concerns its usage by all the users of each one of the countries that participate in the project. According to Hackos and Redish (1998,pp. 386-7), the assessment of a prototype by its users will provide us  Does it convey a consistent conceptual model?  Does it match the user's mental model?  Does it match the user's way of working?  Does it use the users' words?  Does it cover the tasks that users expect to be able to do with the product?  Does it work for all the scenarios (situations) that users say occur?  Does it streamline tasks for the user?  Does it help users make the transition from what they've been doing before?  Does it divide the work well between the computer and the user?  Does it provide messages where and when the user needs them?  Does it maintain consistency in the look and feel across screens ?  Does it maintain consistency in where buttons, icons, and other navigation tools are across screens?  Does it maintain consistency in the vocabulary used across screens? According to the same authors, these dimensions can be assessed walking through the prototype, starting with a created scenery, which represents a story or a created situation. This way, one can see how users carry out a task interacting with the system. For example, how users identify problems that result from wrong or illogical design options that may prejudice the quality of interaction in terms of the facility and quickness of carrying out a task, and that are on the conceptual basis of the development of the application being tested. Nielsen (1993) also proclaims the usage of scenarios as an hypothesis for the assessment of the interfaces of systems: "...listing the various steps a user would take do perform a few realistic tasks. Such a scenario should be constructed on the basis of a task analysis of the actual users and their work in order to be as representative as possible of the eventual use of the system." ( p.159) Hackos and Redish consider that the navigation through the prototype based on scenarios will be the first evaluation technique, mainly if this is not done with the real users but with representative users of the ones the application is for. It is precisely the impossibility of testing the prototype with real users that lead us to the option of testing the prototype with representative users, structuring the usability test according to preestablished scenarios, and based on real tasks. Besides, we considered it necessary to adopt an evaluation method that could guarantee us that the evaluation resulting from the usability tests would give us useful guidance for the qualitative growth of the graphic interfaces of the alpha version of the prototype, in a way that the beta version might reflect qualitatively the outcome of that evaluation. Thus, the option fell upon the adoption of a heuristic model of evaluation of the users'graphic interfaces (Nielsen, 1995). It was, thus, with the purpose of assessing the usability of the graphic interfaces of the alpha version of the prototype (current phase of the project), that we developed an instrument (cf. attached document 1). This instrument aims at evaluating to what extent is the interface proposal sufficiently consistent, intuitive and facilitator of the interaction between the users and the computer in what concerns the accomplishment of tasks related to the access and share of didactical resources, or learning contents, in the area of environmental education. This instrument will be applied to a reduced but representative number of users. 3. Heuristic Evaluation Model The heuristic evaluation model adopted in this research is framed by the group of heuristics of usability of interfaces, proposed by Nielsen (1993). These heuristics can be useful either in the analysis of graphic interfaces or other intermediaries of the interaction between human and machine, such as users' interface designers. According to this author, the heuristics that lead to the success of the interaction between users and interfaces are defined according to the principles of usability. Namely: 1. Simple and natural dialogue; 2. Speak the users' language; 3. Minimize the user's memory load; 4. Consistency; 5. Feedback; 6. Clearly marked exits; 7. Shortcuts; 8. Good error messages; 9. Prevent errors 10. Help and documentation The same author also considers also that these characteristics can be the basis for a systematic evaluation of the interaction between an interface and its users. He calls this evaluation heuristic evaluation. Heuristic evaluation consists of an evaluation of the usability of the interfaces with the user, based on a series of rules. The heuristic evaluation consists, basically, in the analysis of how a reduced number of users of a system perform real tasks through the interaction with the interfaces of the system, confronting the kind of interaction with the heuristics mentioned above. One of the factors we accounted for when we adopted the application of this method, to the detriment of others with higher costs, which a project like "Octopus" couldn't support, was the relation between costs and benefits. According to Nielsen, it is possible to reduce the percentage of errors of the system with the application of a reduced number of tests. However, two conditions are required. One is that we apply the tests to a group of users that effectively represent the universe; the other is that we guarantee that the usability objectives are clear. Thus, the epistemological assertions of this method assume a practical dimension in the context of "Octopus", specifically in the assessment of the alpha version of the prototype. 4. Aims of the assessment of the usability of the prototype One of the main concerns of the work group was to define the aims of the assessment of the alpha version of the prototype. Our perspective of the resource centre must is that it must be a usable system, where errors that may difficult the interaction are reduced; its functionalities are easily memorised, and its usage gives pleasure to its users. Therefore,we decided to adopt, as orienting goals to our study, those defended by Nielsen (1993). According to this author, the assessment of the usability of a prototype are:  To perform tasks without the occurrence of mistakes;  To reduce the user's memory load:  To perform tasks in less than 90" (response time)  To perform tasks according to the users' language;  To perform important tasks in different ways (consistency of the interface);  To reduce the task errors;  To make the necessary help available to the performance of the tasks (help and documentation);  To make good error messages available during the task performance (help for identifying, diagnosing and recovering the errors);  To reduce the number of commands needed to perform a task;  To mark exits clearly;  To give suitable feedback available after the task performance These are the practical aims of usability that we intend to accomplish, and the ones that we accounted for in the creation of the assessment instrument (cf. Attached document 1). 5. Assessment instruments 5.1. Structuring elements The instrument developed to assess the alpha version of the Octopus prototype is divided in four parts, each one with its own operational purpose, namely: i) Instructions to the team of assessment of the usability of the alpha version of the prototype - Octopus (cf. Attached document Octopus-eseb-005a) - document justified due to the fact that the application of the usability tests is done by each one of the partners of the project, with representative users of each country, Because of this, clear and precise guidelines are needed so that the conduction of the tests may be the same in each one of the countries. ii) Inquiry using the questionnaire - Profile of the user (cf. Attached document Octopus- eseb-005a) -based on the need to collect data concerning the users that participate in the test in order to define their profile: iii) Guide of the tasks the users must perform during the usability tests (cf. Attached document Octopus-eseb-005a) - Guiding document of the real tasks the users must perform in interaction with the system. These tasks are organised according to each one of the proposed scenarios: iv) Individual registration grid of the usability tests (cf. Attached document Octopuseseb-005a) - Document where the "annotators" will register the observed results of the interaction of the users with the system in what concerns the number of clicks and required time to perform the tasks successfully. 5.2. Inquiry through questionnaire We created a questionnaire aiming at the characterisation of the profile of the users participating in the usability tests (cf. Attached document Octopus-eseb-005a). The structure of the questionnaire implies seven closed questions, through which we want to characterise the profile of the inquired at the following levels: socio-professional; technological literacy; knowledge about environmental education; kind of interaction they establish with a new application; usage of the Internet. All the users participating in the tests must fill this questionnaire up. Although this instrument is not directly related to the usability tests, we assume that it is essential to prevent a possible skewing of the results, which may be related to the kind if technological literacy of the users that participate in the tests. Once that the usability tests are applied in each one of the countries participating in the project, the creation of such an instrument also allows the team of the ESEB, responsible for the assessment of the prototype, to have the notion of the profile of the users that participated in the tests. 5.3. Guide for the conduction of the usability tests The guide for the conduction of the usability tests (cf. Attached document Octopuseseb-005a) is the instrument directly related to the usability tests and to the performing tasks. This guide is basically oriented to describe the tasks that the user must perform during the tests. Its conception is based on the aims of the usability assessment and in a group of tasks, associated to the access and share of resources in the area of environmental education. These tasks may be performed by the users of the resource centre according to the level of access to its functionalities. The guide of performing tasks is an itinerary of the tasks that the user must perform. The tasks are systematized according to the 3 scenarios proposed. For each task we need to know the objectives of usability that we have to assess, as well as the estimated time to perform the task (TePT) and the number of clicks estimated to perform the task (NcePT). The structure of the guide is based on the need to test the tasks inherent to the functionalities associated to different levels of access to the resource centre. Therefore: - The tasks that occur in the context of the 1st scenario are the ones that do not oblige to a registration of the user in the database of the resource centre. For that reason all users must accomplish these tasks. The registration of the user, getting a username and a password, in the data base will be the last task of this scenario; - The tasks correspondent to the 2nd scenario will be the ones that give access to a range of functionalities of the resource centre. In order to accomplish these tasks, the users need to register in the data base, and need a username and a password; - The 3rd scenario corresponds to a group of tasks that can only be accessed by the administrators of the resource centre. In this scenario, the kind of tasks proposed is eminently associated to the management of the resource centre. Therefore, they can only be applied to the users that participate in the development of the project. 5.4. Individual registration grid of the usability tests The guide of performing tasks comes together with an individual registration grid of the usability tests (cf. Attached document Octopus-eseb-005a) and should be used by the annotators (member of the Octopus team). The characteristics of the computer used in the test must be registered in this grid, as well as: whether the user accomplished the TePT or not; the necessary time to accomplish the task (TnPT); whether the user accomplished the NcePT or not; the number of clicks needed to accomplish the task (NnnPT). The annotators must also register the observations they consider appropriate according to the objectives of usability. 6. Sample selection Bearing in mind that if the participant users are representative of the universe, it will not be necessary to apply a high number of tests in order to lead an heuristic evaluation of the usability of the graphic interfaces, we considered that the sample selection should guarantee an equitable participation of the users of each one of the countries that participate in the project. Another factor that conditioned the selection of the sample was the number of scenarios that structure the real tasks that must be performed by the users. Considering that we will have a different user's profile for each one of the scenarios (see picture number 2), this implies three different types of users participating in the tests, according to their different profiles and levels of access to the system. According to this, we will have three users with different profiles in each one of the five countries that participate in the project, which makes a sample of, at least, fifteen users participating in the tests. According to the different scenarios and the different user's profiles, we will have: - Users without registration - they only perform tasks in the 1st scenario; - Authorized users - the ones that perform tasks in both the 1st and the 2nd scenario, according to their permission of uploading resources to the system; - The administrators - according to their profile, these users can perform tasks in the 3rd scenario, and, as such, they will accomplish the tasks of the three different scenarios. BIBLIOGRAFÍA.-