Evaluating Human Activity Recognition Systems for AAL Environments

See discussions, s a s, and au ho p o iles o his publica ion a : h ps://www. esea chga e.ne /publica ion/235960648 E alua ing Human Ac i i y Recogni ion Sys ems o AAL En i onmen s Con e ence Pape · Janua y 2013 DOI: 10.1007/978-3-642-37419-7_10 CITATIONS 3 READS 92 1 au ho : Some o he au ho s o his publica ion a e also wo king on hese ela ed p ojec s: E alua ing AAL Sys ems h ough Compe i i e Benchma king View p ojec Heal hy and E icien Rou es in Massi e Open-Da a Based Sma Ci ies-Ci izen (TIN2013-46801-C4-1-R View p ojec Juan An onio Al a ez-Ga cia Uni e sidad de Se illa 75 PUBLICATIONS630 CITATIONS SEE PROFILE All con en ollowing his page was uploaded by Juan An onio Al a ez-Ga cia on 02 June 2014. The use has eques ed enhancemen o he downloaded ile. E alua ing Human Ac i i y Recogni ion Sys ems o AAL En i onmen s Juan An onio ´ Al a ez-Ga c´ıa ? Compu e Languages and Sys ems Depa men , Uni e si y o Se ille, Spain [email p o ec ed] Abs ac . E AAL Ac i i y Recogni ion ack’s main goal is o e alu- a e one o he pilla s o Ambien Assis ed Li ing (AAL): human ac i i y ecogni ion (AR). In his edi ion 4 eams om Uni ed S a es, I eland, Spain and Japan pa icipa ed in he compe i ion. Resul s show ha ac- cele ome e based solu ions a e p omising due o hei small size and hei in eg a ion in complex de ices such as mobile phones o elas ics wea able s aps. Keywo ds: AAL, ac i i y ecogni ion, acking 1 In oduc ion Ac i i y Recogni ion (AR) is a esea ch a ea whe e he objec i e is o ecognize human ac i i ies. The au oma ic and unob usi e iden i ica ion o use s ac i i ies is one o he challenging goals o con ex -awa e compu ing [1] and is expec ed o be a p ac ical solu ion o moni o aged people: Acco ding o UNFPA[2] al hough cu en ly only Japan has an olde popula ion o mo e han 30 pe cen , by 2050, 64 coun ies a e expec ed o join Japan. Ou objec i e is o measu e AR ha dwa e and so wa e pe o mance h ough a compe i ion. Since compe i ions in compu e science ha e a long-es ablished adi ion, he design o E AAL has been inspi ed by o he success ul compe- i ions. Some pas and cu en compe i ions ha e been analyzed in o de o iden i y success ul p ac ices, speci ically: HARL [3],OPPORTUNITY [4], HASC [5] and BSN Con es [6]. The main di e ence wi h he p eceding ones is ha his ack add esses bo h so wa e and ha dwa e so ha whole s a e o he a o ac i i y ecogni ion can be examined. The main objec i e o his compe i ion is o implemen an ac i i y ecogni ion sys em (ARS) ha ecognizes he ollowing ac i i ies: lie, si , s and, walk, bend, all and cycle (using a s a iona y bike). The compe i ion ook place om 9 o 13 July, 2012 a he CIAmI Li ing Lab [7] in Valencia (Spain). In his ack he e is no limi a ion o he numbe o de ices ha can be used and compe ing solu ions can be based on a a ie y ?This wo k was suppo ed in pa by he Eu opean Commission in he amewo k o he FP7 p ojec uni e sAAL unde Con ac 247950. 2 Lec u e No es in Compu e Science o senso s and echnologies, including: accele ome e s, gy oscopes, magne ome- e s, p essu e senso s, mic ophones, senso ne wo ks, mobile phones, came as, e c.. O he echnologies o combina ions o hem a e also conside ed accep able p o ided hey a e compa ible wi h he cons ain s o he hos ing Li ing Lab. Figu e 1 shows he CIAmI map. Fig. 1. Map om CIAmI Li ing Lab. 2 Benchma ks Compe i o s a e in i ed o ins all and un hei ARS du ing a ime slo , in his case o wo hou s and a hal , di ided in h ee subslo s: Ins alla ion, Benchma k and Remo al phases. Du ing he second phase, he ARS a e e alua ed. An ac o (an e alua ion commi ee membe ) pe o ms a p ede ined physical ac i i y ip ac oss he sma home. Audio signals synch onize he ac o mo emen s in each pe o mance ( wice pe eam) in o de o ge he same g ound u h o all he pa icipan s. The pa h E alua ing Human Ac i i y Recogni ion Sys ems o AAL En i onmen s 3 ollowed by he ac o and he ac i i ies a e he same o each pe o mance, and hey we e no disclosed o compe i o s be o e he applica ion o he benchma ks. Simila ly, he posi ion o he s a iona y bike and he place o he all a e no e ealed ei he . Once he wo pe o mances a e execu ed, he one wi h be e o e all ma k is used o compa e each eam. A c i ical issue ( ha was communica ed in ad ance o he compe i o s) is he age o he ac o , ha is equi ed in o de o ain and p epa e hei algo i hms p ope ly. The ac o was also ained o epea he ac i i ies in he benchma ks always in he same way ( ollowing he mp3 ile explained in Sec ion 3). The all is also c i ical, because many di e en kind o alls a e possible [8]. Fo his eason we published in ad ance a ideo o he all ha had been execu ed by he ac o du ing he expe imen . A eco ded pe o mance can be seen in his ideo1. 3 Re e ence Localiza ion Sys em The e e ence ARS is used o ob ain he g ound u h da a. In o de o ge app oxima ely he same g ound u h o all he con es an s, audio signals we e used o synch onize he ac o mo emen s. An mp3 ile indica es he nex ac i - i y ha he ac o mus pe o m and a coun down (“ h ee, wo, one, now”) o pe o m i . When he ac o hea s he wo d “now”, he begins he ansi ion o he nex ac i i y. In some cases, such as BSN Con es , esea che s iden i y an- si ions o ecognize he nex ac i i y. In ou case wi h 7 ac i i ies, he numbe o possible ansi ions (some o hem no e y p obable) is 42. Since he numbe o ansi ions is high and i is no i ial o e alua e hem, we decided no o e alua e he ansi ions bu only he ac i i ies. To e ie e compe i o and g ound u h da a, a local se e accep s socke s wi h he ac i i y code and he ime when i is iden i ied. A compe i o sys em only need o send his in o ma ion o he se e . A local NTP se e is also a ailable o synch onize he ime. To ob ain g ound u h da a, an e alua o uses an And oid applica ion o ma k he ac i i ies (ac i i y code) o ansi ions (-1 code) h ough his mobile phone and o send his in o ma ion o he socke se e . To his pu pose, he e alua o ollows he ac o o see exac ly when he s a s he ac i i ies o ansi ions. Fo ins ance, i he ac o is s anding s ill and hea s “Cycling 3, 2, 1, now!” he begins o hold he handleba s and mo e his leg o go up he bike. When he wo d “now” is spelled, he e alua o pushes he bu on in he applica ion o iden i y he end o p e ious ac i i y and he beginning o a ansi ion. The e alua o la e on pushes again he bu on o ma k he end o he ac i i y. 1h p:// imeo.com/52843550 4 Lec u e No es in Compu e Science 4 E alua ion C i e ia The ollowing c i e ia and weigh s we e used o e alua e he sys em o e e y eam: Accu acy (0.25) I e alua es he ecognized ac i i y ins ances (500 ms) using F-measu e 2∗p ecision∗ ecall p ecision+ ecall o compu e i . Recogni ion delay (0.2) I e e s o he elapsed ime be ween he ins an in which he use begins an ac i i y and he ime in which he sys em ecognizes i . Ins alla ion complexi y (0,15) I is a measu e o he e o equi ed o ins all he ARS in a la , measu ed by he e alua ion commi ee as a unc ion o he pe son-minu es o wo k needed o comple e he ins alla ion. Use Accep ance (0.25) I cap u es how much in asi e he ARS is in he use s daily li e and he eby he impac pe cei ed by he use ; his pa ame e is e alua ed by he E alua ion Commi ee using a ques ionnai e. In e ope abili y wi h AAL sys ems (0,15) I is e alua ed using a ques- ionnai e ha measu e he use o open sou ce solu ions, use o s anda ds, a ailabili y o lib a ies o de elopmen and in eg a ion wi h s anda d p o- ocols. 5 Con es an s and esul s A e pee e iew, i e eams we e accep ed bu one o hem wi hd ew due o inancial cu backs o i s ins i u ion. Hence only ou compe i o s pa icipa ed in he challenge, CUJ ( om Chiba Uni e si y, Japan), CMU ( om Ca negie Mellon and U ah Uni e si ies, USA), DCU ( om Dublin Ci y Uni e si y, I eland) and USS ( om Uni e si y o Se ille, Spain). The echnology used by e e y eam is desc ibed he e: 5.1 DCU Team Ti le o pape Visual Expe ience o Recognising Human Ac i i ies Lis o Au ho s and a ilia ions Na Li, Ma in C ane, Hea he Ruskin om Cen e o Scien i ic Compu ing and Complex Sys ems Modelling, School o Compu ing, Dublin Ci y Uni e si y. B ie desc ip ion DCU Team uses a SenseCam2hanging om he ac o ’s neck o e alua e o -line he ac i i ies. The DCU sys em wo ked o line and he e- o e no accu acy and delay sco e was assigned. 5.2 CUJ Team-B onze Ti le o pape Human Beha io Recogni ion by a Mobile Robo Following Hu- man Subjec s. 2h p:// esea ch.mic oso .com/en-us/um/camb idge/p ojec s/sensecam/ E alua ing Human Ac i i y Recogni ion Sys ems o AAL En i onmen s 5 Lis o Au ho s and a ilia ions Myagma baya Ne gui1, Ne ez Imamoglu1, Yuki Yoshida1, Jose Gonzalez2, Masashi Sekine2, Kazuya Kawamu a1and Wenwei Yu1. 1. Medical Sys em Enginee ing Depa men , G adua e School o Enginee - ing, Chiba Uni e si y. 2. Resea ch Cen e o F on ie Medical Enginee ing, Chiba Uni e si y. B ie desc ip ion CUJ Team uses a obo (ini ially a Pionee 3-AT3bu due o di icul ies in anspo om Japan, a Roomba4was used ins ead) wi h wo kinec s5. The i s Kinec is used o a oid obs acles when ollowing he ac o and he o he o ecognize hei ac i i ies. A e he E aal expe ience CUJ decided o use only one Kinec . 5.3 CMU Team-Sil e Ti le o pape An Ac i i y Recogni ion Sys em o Ambien Assis ed Li ing En i onmen s Lis o Au ho s and a ilia ions Jin-Hyuk Hong, Julian Ramos, Choonsung Shin, Anind Dey om Human-Compu e In e ac ion Ins i u e Ca negie Mel- lon Uni e si y. B ie desc ip ion CMU Team p oposes a solu ion composed by h ee subsys- ems: A ches wea able elas ic s ap6capable o measu ing se e al physiolog- ical signals, an And oid mobile phone and a sys em o indoo localiza ion based on Radio omog aphic imaging7. 5.4 USS Team-Gold Ti le o pape Ac i i y ecogni ion sys em using AMEVA me hod Lis o Au ho s and a ilia ions Luis Miguel So ia1, Luis Gonzalez-Ab il2, Miguel ´ Angel ´ Al a ez de la Concepci´on1,, Juan An onio O ega Ram´ı ez1 1. Compu e Languages and Sys ems Depa men , Uni e si y o Se ille. 2. Applied Economics I Depa men , Uni e si y o Se ille. B ie desc ip ion USS Team uses an and oid mobile phone placed on he igh hip. The use ac i i ies a e ecognized by means o he accele ome e em- bedded in he mobile phone. 5.5 Resul s Table 1 shows he esul s o each eam and c i e ia o he bes o bo h pe - o mances. The winne (USS eam) ob ained accep able esul s in pe o mance, bu i s simplici y (al hough i uses mul iple ma hema ical me hods i only ely on accele ome e s) and in e ope abili y gi e good ma ks in all he e alua ed c i e ia. Gold and Sil e eams used accele ome e -based solu ions. 3h p://www.mobile obo s.com/ esea ch obo s/p3a .aspx 4h p://s o e.i obo .com/ amily/index.jsp 5h p://en.wikipedia.o g/wiki/Kinec 6h p://www.zephy - echnology.com/bioha ness-b 7h p://span.ece.u ah.edu/ adio- omog aphic-imaging 6 Lec u e No es in Compu e Science Table 1. Bes pe o mance esul Team Accu acy Delay Ins alla ion Use Accep ance In e ope abili y Final Sco e USS 4,33 9 10 7,47 7,63 7,3945 CMU 7,17 9 0 7,93 6,15 6,4975 CUJ 2,59 2 0 5,6 5,09 3,5235 DCU 0 0 10 5,2 1,25 2,9875 6 Conclusions Nex edi ion some imp o emen s may ega d he kind o ac i i ies (some mo e complex ac i i ies can be aken in o accoun ) and he possibili y o simula e an aged ac o using speci ics ki s o a oid apid mo emen s. Resul s sugges ha he e is s ill space o o he edi ions o his compe i ion in he u u e, and encou age us o con inue in his ack o he nex yea . Re e ences 1. B ush, A.B., K umm, J., Sco , J., Saponas, S.: Recognizing ac i i ies om mobile senso da a: Challenges and oppo uni ies. In: Ubicomp’ 11. (2011) 2. Uni ed Na ions Popula ion Fund and HelpAge In e na ional, ed.: Ageing in he Twen y-Fi s Cen u y: A Celeb a ion and A Challenge. Uni ed Na ions Popula ion Fund (Decembe 2012) 3. Wol , C., Mille, J., Lomba di, L., Celik u an, O., Jiu, M., Baccouche, M., Delland ea, E., Bicho , C.E., Ga cia, C., Sanku , B.: The li is human ac i i ies da ase and he icp 2012 human ac i i ies ecogni ion and localiza ion compe i ion. Technical Repo RR-LIRIS-2012-004, LIRIS Labo a o y (Ma ch 2012) 4. Sagha, H., Diguma i, S.T., del R. Mill´an, J., Cha a iaga, R., Cala oni, A., Roggen, D., T ¨os e , G.: Benchma king classi ica ion echniques using he oppo - uni y human ac i i y da ase . In: Sys ems, Man, and Cybe ne ics (SMC), 2011 IEEE In e na ional Con e ence on. (Oc obe 2011) 36 –40 5. Kawaguchi, N., Ogawa, N., Iwasaki, Y., Kaji, K., Te ada, T., Mu ao, K., Inoue, S., Kawaha a, Y., Sumi, Y., Nishio, N.: Hasc challenge: Ga he ing la ge scale human ac i i y co pus o he eal-wo ld ac i i y unde s andings. In: ACM AH 2011. (2011) 6. Vi ali, L., Roozbeh, J.: Powe awa e wi eless da a collec ion o bsn da a eposi o- ies. In: In e na ional Con e ence on Body Senso Ne wo ks. (2011) 7. Ma ´ınez, A., Llo en e, M.A., L´aza o, J.P.: Ciami li ing lab: an economically sus- ainable echnological ool o open inno a ion. In: I ENoLL Li ing Lab Summe School. (2010) 8. Nou y, N., Fleu y, A., Rumeau, P., Bou ke, A.K., ´ OLaighin, G., Rialle, V., Lundy, J.: Fall de ec ion - p inciples and me hods. In: P oceedings o he 29 h Annual In e na ional Con e ence o he IEEE Enginee ing in Medicine and Biology Socie y (EMBS 2007). (2007) 1663–1666 View publica ion s a sView publica ion s a s