Augmented and Alternative Communication System Based on Dasher Application and an Accelerometer

Augmented and Alternative Communication System Based on Dasher Application and an Accelerometer Isabel Gómez, Pablo Anaya, Rafael Cabrera, Alberto J. Molina, Octavio Rivera, and Manuel Merino Electronic Technology Department, Universidad de Sevilla, Spain {igomez,almolina}@us.es, {pjanaya,rcabreraac}@gmail.com, {octavio,manmermon}@dte.us.es Abstract. This paper describes a system composed by predictive text input software called “Dasher” and a hardware used to connect an accelerometer to the computer. The main goal of this work is to allow people with motor disabilities to have a flexible and cheap way to communicate. The accelerometer can be placed on any body part depending on user preferences. For this reason calibration functionality has been added to dasher software. The calibration process is easy and requires only some minutes but it is necessary in order to allow system can be used in different ways. Tests have been carried out by placing the accelerometer on the head. A rate of 26 words per minute is reached. Keywords: dasher, accelerometer, open and flexible system, text input. 1 The R & D or Application Idea The objective of this research project is to study dasher and its possibilities when it is handled by an accelerometer. Dasher is a text input system which incorporate prediction1 [1]. The system infers the most probable next characters, calculating the use frequency through a training text. During the program execution, letters move from the right to the center of the screen. Those letters are shown in color boxes which size depends on the probability of its occurrence (Figure 1). Dasher has different modes of operation: 2-D continuous input; 1-D continuous input; discrete input. This allows that different input devices could be used. In 2-D mode, both values are used to move the cursor on the screen, and the speed depends on the relative position of the cursor respect to center. In 1-D mode, there is only one input, midrange values of the input control the zooming direction, values towards the extreme of the available range allow the user to zoom out and pause. In discrete input mode Dasher zoom in depends on which region the user select. The accelerometer used for this application has two axis so 2-D mode of Dasher is possible. The system may be used by users with the ability to make small movements with any body part. Therefore, users with motor impairments who cannot use the standard keyboard and mouse are the main beneficiaries of this system. 1 Dasher is free open source software. Fig. 1. Example of using Dasher. Here it can be observed the most probable words that can be formed with the selected letters. 2 The State of the Art in This Area Several uses of Dasher with different input devices can be found in [1]. It can be used with a device based in breath in 1-D mode [2]. In discrete mode, it can be used in different ways depending on the number of input buttons [3]. In the 2-D mode it can be used with eye tracking systems based on image processing [4]. Some proposal about the use of dasher with a Brain computer Interface system can be found in [5, 6] but results are not good, the conclusion is that at the moment another alternatives are preferable. Accelerometers can be applied in Assistive Technology in different ways. In [7, 8] they are used in rehabilitation systems. In [7] an automatic head position monitoring system is designed for control the recovery process after an ophthalmological operation. In [8] accelerometers and gyroscopes are settled in wrists and ankles to detect the appropriate movements in a designed system for tele-rehabilitation. In [9, 10] accelerometers are placed on the head, they are used for computer access proposal. The systems described are complex because the whole computer control is pursued. In [11] effectiveness and reliability as an interaction device is evaluate when accelerometer is placed in a handheld device. The advantage of using this kind of interaction is that one of the user's hands is free and the device's tiny screen is totally visible. It has been found that the use of dasher with accelerometers is limited to handheld devices [1]. Our proposal is different because we use a computer with the accelerometer located on any body part. 3 The Methodology Used 3.1 System Architecture In figure 2 is shown the system architecture. The first component is the accelerometer. An ADXL322 was used for this purpose. This device is a low cost, low power, complete dual-axis accelerometer with signal conditioned voltage outputs, which is all on a single monolithic IC. The product measures acceleration with a full-scale range of ±2g. The ADXL322 is an ultra small package (4 x 4 x 1.45 mm LFCSP). Amplification and filtering board is connected to the accelerometer; the aim of this board is to eliminate quick and undesirable movements and it allows the system to work with a reduced angle range what increments system usability. The microcontroller-based system reads the accelerometer x-y coordinates and communicates that acceleration to a PC. Another function of this system is to perform digital signal processing. In order to increment the system effectiveness, we have programmed a moving average filter. Arduino has been used for implementing this system. Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software [12]. It is based on the Atmel Atmega328 microcontroller. The last element is the personal computer where dasher is installed. Fig. 2. System Architecture 3.2 Software Analysis and Design Our aim in this work is the analysis of Dasher software to connect any kind of input device. The first step is to connect the accelerometer. In our very case, we are working in biosignal processing and we have as a main target the use of electromyography and electrooculography to control this text input software. The first attempt was connecting dasher using socket as it was the most flexible option. Furthermore it is assumed that the software is ready for a configuration of this kind of input. However, the option above described is not implemented. Instead, we edited the dasher code to make it read an emulated serial port. In this way, the accelerometer reading is done via an USB port. Once the data has been obtained, it is necessary to adapt them to the values that Dasher uses to place the cursor on the screen: timing, scaling and adapting to screen resolution. The accelerometer can be placed on anywhere depending on the user’s preferences. For this purpose a calibration function was added to the Dasher code. (Fig.3). The calibration process is easy and requires only a few minutes but it is necessary in order to allow the system to be used by different users. The user is asked to execute a circle movement in the widest way he is able to. 3.3 System Testing The tests were carried out by placing the accelerometer on the head. In Figure 4 we can see the actual set up. We will be using WPM (word per minute) as our unit to measure the efficiency tests results. Fig. 3. Calibration Menu Fig. 4. Test System Three non disable users (two experts and one novice) introduced a dictation text using the system; the system was trained with another text (the training test). Tests are performed in three cases depending on the characteristics of the training text: 1. without training text (untrained system) 2. text that doesn't match to dictation text, 3. text that is well matched to the dictation text. A training text of 400 words and a dictation text of 40 words were used. A user is considered a novice when has not used the system previously, on the other hand an user reaches the expert grade with few uses of the system. 4 Results The more the machine trains the more it “learns” and the users become more familiar with the usage of the application. Proficiency is achieved by increasing the usage time of the system with a text that the user is getting into. (See table 1). Table 1. Results measured in WPM Training cases Expert1 Expert2 Novice Untrained 2,6 3,4 2 Not matched 4,8 5,33 3,69 Matched 17,33 26 13 As we can see in [2], these are the results obtained when using Dasher with other types of input devices. For gaze use, these ranges go from 15 to 25 WPM, for driving with breathing 8 to 17 WPM and finally with the actual mouse which goes between 25 and 35 WPM. Therefore the results obtained with the accelerometer are within the range of those that can be achieved with other devices. 5 Conclusion and Planned Activities In this paper it has presented the design and testing of a very low cost system that enables the communication to a disabled user. The system is uses a user-friendly interface and the user acquires a high level of expertise in a short period of time. Tests have been conducted with able-bodied users but we are working to test the system with disabled users and we hope to present the result of these tests in the final version of this application. Two lines are opened as planned activities: 1. Connect Dasher with input devices based on biosignals. 2. Study the use of accelerometers in tele-rehabilitation systems design. Acknowledgements. This project has been carried out within the framework of a research program: (p08-TIC-3631) – Multimodal Wireless interface funded by the Regional Government of Andalusia. References 1. The Dasher Project, http://www.inference.phy.cam.ac.uk/dasher 2. Shorrock, T.H., Mackay, D.J.C., Ball, C.J.: Efficient communication by breathing. In: Deterministic and Statistical Methods in Machine Learning, Springer, Heidelberg (2005) 3. Mackay, D.J.C., Ball, C.J., Donegan, M.: Efficient communications with one or two buttons. In: Fisher, R., Preuss, R., von Toussaint, U. (eds.) Proceedings of Maximum Entropy and Bayesian Methods, ser. Amer. Instit. Phys. Conf. Proc., vol. 735, pp. 207–218. Amer. Inst. Phys., Melville (2004) 4. Ward, D.J., Mackay, D.J.C.: Fast hands-free writing by gaze direction. Nature 418(6900), 838 (2002) 5. Wills, S.A., Mackay, D.J.C.: DASHER- An Efficient writing system for Brain-Computer Interfaces? IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2) (June 2006) 6. National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov 7. Felton, E.A., Lewis, N.L., Wills, S.A., Radwin, R.G., Williams, J.C.: Neural Signal Based Control of the Dasher Writing System. In: Proceedings of the 3rd International IEEE EMBS Conference on Neural Engineering (2007) 8. Hamel, M., Fontaine, R., Boissy, P.: In-Home Telerehabilitation for Geriatric Patients. IEEE Engineering in Medicine and Biology Magazine 27(4) (July/August 2008) 9. Nakazawa, N., Yamada, K., Matsui, T., Itoh, I.: Development of Welfare Support- Equipment for Personal Computer Operation with Head Tilting and Breathing. In: IECON Thirty-First Annual Conference of the IEEE Industrial Electronics Society (2005) 10. Chen, Y.-L.: Application of Tilt Sensors in Human-Computer Mouse Interface for People with Disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering 9(3) (September 2001) 11. Sad, H.H., Poirier, F.: Evaluation and modeling of user performance for pointing and scrolling tasks on handheld devices using tilt sensor. In: Second International Conferences on Advances in Computer-Human Interactions, IEEE Computer Society, Los Alamitos (2009) 12. Arduino, http://www.arduino.cc/