Combining Smart Tags and Body Fixed Sensors for Disabled People Assistance

Combining Smart Tags and Body Fixed Sensors for Disabled People Assistance Juan Antonio Álvarez1, Carlos Pérez2, Cecilio Angulo2, and Juan Antonio Ortega1 1 ETSII. University of Seville Avenida Reina Mercedes, s/n. 41012 – Sevilla, Spain [email protected], [email protected] 2 CETpD – UPC. Technical University of Catalonia Rambla de l’Exposició s/n, 08800 – Vilanova i la Geltrú, Spain [email protected], [email protected] Abstract. The world population is rapidly aging in industrialized countries and at a growing rate. As well as the percentage of elderly people is constantly increasing, so related health costs are augmenting, demanding novel technological solutions that both, enhance elder daily life independence and reduce the social and economic overhead of their hospitalization. The general framework of the presented work is about technological platforms for helping elder and handicapped people in a non-intrusive manner. The research in human motion monitoring, having in mind the particular objective of fall prevention, and the use of RFID technology combined with text to speech techniques are merged on a general purpose assistive platform able to help all kind of people, including those who has not impairment but are seeking to improve their quality of live. Keywords: Multimodal sensor fusion, assistive systems, body monitoring. 1 Introduction Society demands adequate, accessible and affordable social services for disabled people and, in general, needing healthcare. Focusing in these needs as objective, this article presents on-going research that is creating novel knowledge on both, technology and accessible and adequate technological tools for people with dependencies needing medical monitoring or wanting to improve their quality of daily life. In particular, research is centred on human motion monitoring, having in mind the objective of fall prevention, and the use of RFID technology combined with text to speech techniques with the aim to make available an assistive platform able to help all kind of people. Several works on human motion capture through artificial vision systems are in the literature concerning different application scenarios. These systems are usually based on the use on single or multiple cameras [1], producing in this form undesirable reactions from assisted people. A less intrusive system should be used in assistive environments, for example using fusion of data sensed with a device built in the form Combining Smart Tags and Body Fixed Sensors for Disabled People Assistance of wearable computing [2]. For particular diseases like Parkinson, there exist some interesting solutions [3, 4], but there are specifically focused in this disease, not for a general purpose. A more general system with a wider scope will be proposed in this work, considering as primary target users elder people with visual impairments and/or with motion difficulties. There exist some projects with similar target users [4, 5], but they are mainly concerned on middleware or data mining procedures. In contrast, our work is offering complete electronics solutions based on body fixed sensors in the form of a wearable motion monitoring device, named KneeMeasurer, and approximate indoor positioning by using RFID tags technology, both integrated through a computing system running on a PDA unit. This technological platform signifies a non-intrusive communication link between the assisted users, their usual environments and their helpers. According to some inquiries, users prefer to be assisted by known people like their relatives, so a non-intrusive, secure and personal communication must be true. In the next section an exemplified scenario is presented in order to illustrate the capabilities of the assistive platform. Next, the proposed architecture is presented. Some evidences of the completed work on the wearable motion monitoring device, the tag-based text-to-speech processing, and the integrated communication through a PDA unit are presented showing the viability of the proposed platform. Some conclusions and future work are presented to conclude. 2 Scenario To show the possibilities of our system, a complete scenario is explained: “John must take in some medicines at 4:00 PM, but he forgets them. Fortunately, a tag-based identification subsystem access to the stored information and detect that he hadn’t took the object tagged with the medicine. The wearable PDA reads, using the Text to Speech technology, the reminder associate: “John, you must take the medicine sited on the cupboard of the kitchen, have you done? Touch your PDA-screen once if you do, twice if you don’t”. John touches it twice and goes to the kitchen; when he takes his medicine, he deactivates the reminder. Later, John has to do some exercise to keep fit according to doctor prescription, but he did not. At 7:00 PM, the reminder service explains John that he has not done too much exercise during the day, and it would be recommendable to do it before dinning. At 9:30 John is having a shower and unfortunately he fall on it and loss his consciousness. Although he is not carrying the wearable PDA and the motion monitoring device, the last place where the location service had information was the bathroom because he takes the tagged shampoo. The long time period with neither change in the location service nor tagged elements is detected as a risky situation so the communication service sends an alert text to the communication server that resends it as short messages (sms) to mobile phones of a relative or/and helpers. 3 Technology RFID technology combined with text to speech techniques will be combined with a body fixed sensors motion monitoring system to create an assistive platform able to J.A. Álvarez et al. help all kind of people in similar scenarios like that described above, including those who has not impairment but are seeking to improve their quality of live. 3.1 RFID Radio frequency identification has been selected for the design of the positioning system [6, 7] because it is a growing technology being integrated on every element in our daily life, in the same form that bar codes were introduced some time ago. Nowadays, the most extended RFID technology is “High Frequency” (HF), working in the band of 13.56 MHz, being used as a basis for the communication system called NFC (“Near Field Communication”). The main disadvantage of this technology is that read distances range from 3 to 5 centimetres. A second general RFID approach is “Ultra High Frequency” (UHF), working in the range of frequencies from 850 to 950 MHz. This technology is much less extended, but is the one that tends to more expand in the industrial identification fields due to its long reaches (until 8m in the latest versions of passive tags and 30 – 100m with active tags). In the next years this latter technology will prevail in the industrial area and a large number of products will come to our homes with the factory label, like now they are coming with bar codes. Unfortunately, nowadays the unique portable RFID reader Compact Flash for PDA devices is HF, so we are obligated to select this technology for our on-going applications. 3.2 KneeMeasurer A novel methodology designed for measuring human joint angles, in a first version for knee angles, has been developed for the assistive platform in indoor and outdoor environments using a wearable gesture interface [8]. The knee-mounted KneeMeasurer device consists of two modules endowed with a dual axis accelerometer, a digital signal processing element and a wireless communication component. Accelerometers are commonly used to sense leg movements for monitoring the user gait in his/her daily life. Joint angles can be directly calculated in the processing units situated on the nodes of the KneeMeasurer and this information be sent by Bluetooth communication to a handled device. As a first application of this motion monitoring device, completely wireless on-line monitoring of the knee angle is performed. The accelerometers used are Analog Devices ADXL203. They are precision dualaxis accelerometers with an scale of ±1.7 g. Adjusting the working bandwidth of them, it is possible to achieve a resolution of 13 mg, that is a typical measured error of 0.8 degrees of tilt. The accelerometers have two analogical outputs, one for each axis, that are directly connected to an analogical to digital converter. The sensor boards are both connected by cable to a process unit which captures all the sensor measures, calculates knee angles and sends it to a desktop, a laptop or a handled by Bluetooth communication. A deeper explanation of the system is showed on [8]. The power consumption is a critical factor in wireless devices, even more if the size of the system is also an important parameter. The sensors must been chosen with the power consumption as a premise. Each accelerometer consumes about 0.6 mA in Combining Smart Tags and Body Fixed Sensors for Disabled People Assistance normal operation, and gyroscopes consume 4 mA approximately each one. For this reason, it has been tried to minimize the consumption of every part of the design by eliminating the use of the gyroscope. A critical element to be also considered due to power cost is the microcontroller. The dsPIC30F processing unit consumes about 22 mA in normal operation, and together with the ZigBee module transmitting it consumes about 28 mA. The total power consumption of the system in its normal operation is of about 31 mA, but it can be reduced using communication interruptions and putting the micro controller in sleep mode during periods of inactivity. In the case of using continuous Bluetooth communication to a PDA, the total consumption increase until 65mA. Another factor to take in consideration is the size of the prototype. One of the objectives to accomplish is to achieve a system the less annoying possible for the patient that has to wear it. So trying to miniaturize the components is also a priority. In the actual prototype, the build sensor board is sized about 30x30mm and the microcontroller board has approximately the same size. Fig. 1. Three-tier architecture and prototype image 4 Architecture Three-tier architecture has been selected to deal with all the received and processed information. The first one is the Monitoring tier, in charge to store both, data collected from the body fixed sensors wearable motion monitoring device (the KneeMeasurer) and data coming from RFID readers. The second tier analyses the information stored on the first one and it interprets data for obtaining places associated to the tags identification and matching simple patterns like walking, sitting or lying down. Used technology has been described in the last Section and each service on the third tier will be now detailed. J.A. Álvarez et al. 4.1 Audio-Description Service Audio description will be a main service for people with sight problems to fully inform about any kind of object tagged with a RFID tag. It is an additional narration that describes all significant information such as medicines composition or dosage or material, colour and description of clothes. The information for each tag will be stored in distributed databases, the system maintaining diverse databases at different levels. Figure 2 shows a hierarchical description of them. The first one is a small customized database in the device memory, keeping the personal user database. These data allow personal tags recorded by the user with his voice with no need to be linked to a remote server. Whether the associated information to a tagged-product is not located on this PDA database, a second one is disposed on a local server, usually a PC-like machine at home. This machine will control the data “expiring” in the local database and will look for the data in Internet. Server will also incorporate data verification and renovation systems, maintaining reasonable size limits without loosing of speed. Fast processing is also a critical point for the system due to the time consumption on queries send through Internet. Although a PDA can be connected by itself to Internet, it is being considered that this machine “bridge” becomes necessary because the power cost to connect a portable device, like a PDA, to Internet and to maintain a large size data base updated is excessive for the present batteries characteristics. On the other hand, it must be had in mind the present PDAs memory limitations, data base size and speed limits. In the future it cannot be discarded that the portable device can exert both functions. Finally a Global server is considered, so that third, properly authorized, people can access to introduce global information on the product identifiers. This server will maintain the global data base to which the local servers will access, under automatic consultation, to update their databases. This structure should be able to be integrated within standard ISO 18006, and more concretely, a desirable objective is to approach the system as much as possible to the EPCglobal recommendations, a standards precursory organization. Fig. 2. Scheme of hierarchical databases Combining Smart Tags and Body Fixed Sensors for Disabled People Assistance 4.2 Location Service Using the RFID infrastructure for the audio description service, a simple indoor location system was built. Although more accuracy would be desirable, the requirements for a reliable and continuous indoor position determination are not very wearable [9]. The constraints imposed over the service were hard, due to the kind of RFID readers (HF) and their range (5 cm). When the portable reader detect a position tag is possible to locate the person when he goes through the doors and when the reader detects an object is possible to associate the object with a place or an action. Tags at shoulder-height were placed on both sides of doorframes on each door. When the user enters the room facing forward, the portable reader will reads the tag on the right of the doorframe so knowing the tag identification and the place located, deduction of position is done. Figure 3 shows how are placed the tags. A 5-seconds time window is waited to allow sequences of two tags read. A useful situation is when the user walks the corridor up or down. For energy consumption optimality, tags reader is not always reading, otherwise batteries will be down in a short time. Reader tags will be active only when the KneeMeasurer is detecting activity movement of the user, i.e. whether the user is walking or he/she is actively changing the posture. Fig. 3. Tags and static readers on strategic places Typical approaches distribute different RFID readers on different places, such as the security readers at the doors of most major retail stores. A large infrastructure is needed compared to our approach where the RFID reader is located in a handled device carried by the user and tags are located in the frame of doors. This technique dramatically reduces costs when few people are tracked. 4.3 Reminder Service This service is oriented to patients who must take some medicaments or/and do some physical exercise under medical supervision. To achieve this goal, a Web accessed database was implemented where all the data from different treatments and J.A. Álvarez et al. medicaments were stored. Each user owns a prescription about exercises and medicines so reminders are programmed to be read during each day. For the exercises, the PDA unit is reading via Bluetooth the KneeMeasurer motion monitoring system and periodically sends via WiFi the activity in work. Due to the variety of rehabilitation exercises, it can be distinguished a number of exercises in order to monitor them: •Walk or run exercises: a fast frequency of changing angles is measured. When duration of this pattern matches with the proposed duration of the exercise the reminder will be deactivated. •Exercises in a place. The location service is needed to know the change of room. •Legs movement. For rehabilitation exercises, the movement of the knee wearing the sensors is monitored; the KneeMeasurer is useful so that the reminder would be deactivated. •Legs static. In this case the reminder is always active. The timetable for the rehabilitation exercises is softer than taking medicines and could be done during all the day. To avoid remind very late, the last hour to do it is 7:00 PM. If at this hour the recorded activity does not match with the duration and frequency of the exercise, the text that may be played is sent to the PDA and it reproduces using the text to speech technology. In order to make easy and simple the interfaces, speech recognition was avoided and the implemented one was tactile yes/no interface. 4.4 Communication Service Each minute, when the information about read objects and, angles is sent to the server using Web Services, the server evaluates both, the information stored and retrieved. On the evaluation it takes account of the day and hour, the activity of the user, whether he/she is at home, the room where the objects are read by the RFID readers. If a risk pattern is detected, for example a large period of inactivity with a ‘lied down’ pattern and the room where the user is positioned has not any sofa or bed for sleeping, the server will send short messages to helper’s mobiles. 5 Conclusions and Future Work A technological platform for helping both, elder and handicapped people in a nonintrusive manner have been presented. The research in human motion monitoring and the use of RFID technology combined with text to speech techniques are merged on an assistive platform. Main problems in the system are the position accuracy and the power consumption. The first one is being analysed in [8]. To overcome it, we are moving to use different biometric information. The second one is studied from the KneeMeasurer and the PDA energy consumption. The KneeMeasurer is currently handled using ZigBee communication and, furthermore, tests with the Wii-mote, having more sensors and actuators than our KneeMeasurer, show that its autonomy is about 50 hours. Combining Smart Tags and Body Fixed Sensors for Disabled People Assistance From the PDA perspective, implementation under Windows Mobile OS, that does not stop until a hard reset is done, allows programming the reminder throw the system’s clock such as the tasks reminder included on it. Hence, only the PDA clock is working and only programmed reminders activates all the processor. A future work is to integrate sensors using compact flash or secure disk ports on the PDA discarding wireless payload. Acknowledgments This work has been partly granted by the projects FAMENET-InCare (TSI2006-13390- C02-02), EXODUS-ADA (DPI2006-15630-C02-01), and SAPIENS (IMSERSO 106/5 2005) from the Spanish Ministry of Education and Culture. References 1. Chella, A., et al.: A System for Simultaneous People Tracking and Posture Recognition in the context of Human-Computer Interaction. In: International Conference Computer as a Tool, EUROCON 2005. vol. 2, pp. 991–994 (November 21-24, 2005) 2. Farella, E., et al.: A Wireless Body Area Sensor Network for Posture Detection. In: Proceedings. 11th IEEE Symposium on Computers and Communications, ISCC ’06, June 26-29, 2006, pp. 454–459. IEEE Computer Society Press, Los Alamitos (2006) 3. Moore, S.T., et al.: Long-term monitoring of gait in Parkinson’s disease. In: Science Direct, Gait & Posture, Elsevier, Amsterdam (2006) 4. Han, J., et al.: Gait detection from three dimensional acceleration signals of ankles for the patients with Parkinson’s disease. In: The International Special Topic Conference on Information Technology in Biomedicine (ITAB 2006) 5. Yiping, T., et al.: Elder abnormal activity detection by data mining. In: SICE 2004 Annual Conference. vol. 1, August 4-6, 2004, vol. 1, pp. 837–840 (2004) 6. RFID Whitepaper: alliedworld.com, Allied Business Intelligence Inc. (2002) 7. Jackson, R.J.: Radio Frequency Identification (RFID), A White Paper available on: http://home.agh.edu.pl/~luc/my_doc/what_is_rfid.pdf 8. Raya, C., et al.: KneeMeasurer. A Wearable Interface for Joint Angle Measurements. In: drt4all Conference (April 2007) 9. Retscher, G.: Location Determination in Indoor Environments for Pedestrian Navigation. In: Position, Location, and Navigation Symposium, 2006 IEEE/ION, pp. 547–555 (April 25-27, 2006) View publication statsView publication stats