A new approach to study gait impairments in Parkinson’s disease based on mixed reality

Oc obe 2022 Bea iz Mi anda A new app oach o s udy gai impai men s in Pa kinson’s disease based on mixed eali y Bea iz Ma ia Redondo Mi anda A new app oach o s udy gai impai men s in Pa kinson’s disease based on mixed eali y Oc obe 2022 i Bea iz Ma ia Redondo Mi anda A new app oach o s udy gai impai men s in Pa kinson’s disease based on mixed eali y Mas e disse a ion Mas e Deg ee in Biomedical Enginee ing Medical Elec onics Disse a ion supe ised by P o esso Doc o C is ina Manuela Peixo o dos San os Oc obe 2022 DIREITOS DE AUTOR E CONDIÇÕES DE UTILIZAÇÃO DO TRABALHO POR TERCEIROS Es e é um abalho académico que pode se u ilizado po e cei os desde que espei adas as eg as e boas p á icas in e nacionalmen e acei es, no que conce ne aos di ei os de au o e di ei os conexos. Assim, o p esen e abalho pode se u ilizado nos e mos p e is os na licença abaixo indicada. Caso o u ilizado necessi e de pe missão pa a pode aze um uso do abalho em condições não p e is as no licenciamen o indicado, de e á con ac a o au o , a a és do Reposi ó iUM da Uni e sidade do Minho. A ibuição-NãoCome cial-SemDe i ações CC BY-NC-ND h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/ ii AGRADECIMENTOS Es a disse ação, desen ol ida du an e o úl imo ano, oi o esul ado de mui o abalho e es o ço que nunca pode ia e acon ecido sem a con ibuição di e a e indi e a de mui as pessoas que me são mui o que idas, es ando-me ag adece -lhes. Em p imei o luga , que o ag adece à minha o ien ado a, P o esso a C is ina, pela opo unidade que me deu em abalha nes e p oje o, po oda a mo i ação e o ien ação que oi dando, e ainda po odas as euniões de in a minu os que se p olonga am po duas ho as, po que nem udo é abalho. Admi o- a como p o issional, mulhe e mãe. Es ou ex emamen e ag adecida po oda a sua dedicação e supo e. De seguida, a Helena. Pessoa que mais me acompanhou du an e es e ano e cé eb o do p oje o +sense. Que o ag adece -lhe po odas as ho as em chamada mesmo após e mos passado o dia no labo a ó io, po me e ajudado na ecolha de dados no hospi al, po me ensina li e almen e udo, a é ecei as de bolo da caneca. A paciência, ajuda, dedicação e mo i ação mos adas não são mensu á eis. À minha pa cei a de mes ado Ma a, o 4º ano oi uma ba alha encida ao lado dela. Ag adeço pelas incon á eis noi es a abalha e a can a e as e ças- ei as no labo a ó io. A odos os meus amigos da uni e sidade, ag adeço não só a companhia nas aulas mais dolo osas como odas as b incadei as, saídas à noi e e po me e em dado a conhece mais deles e das e as deles. Ag adeço ambém às minhas colegas de casa po ou i em odos os meus d amas e à melho pessoa que a uni e sidade me deu, a minha a ilhada, Ma ia Pimen a, po se ão disponí el e ão amiga. A odos os meus “amigos de pon e”, B una, Guida, Jucas, Ca oli, Inês, Nelson, Gina, Luces, Jaime, ob igada po me acompanha em desde o secundá io, po odos os e ões, po odos os ca és de sábado à noi e e po se em semp e um po o segu o. Apesa de nos conhece mos há mui os anos, ensinam-me semp e uma coisa no a odas as semanas, nem que seja os no os sabo es da Água das Ped as. Não menos impo an e, enho de ag adece ao Spo i y, às suas playlis s e a is as, po e em sido os meus melho es amigos calman es du an e es e pe íodo. A música semp e se á um pedaço de mim, mesmo que enha seguido o mundo da engenha ia. Po im ag adeço à minha amília, aos meus pais e ambém à Lua, po se em o meu maio supo e inancei o e emocional, po me e em dado as melho es condições que podia pedi , pelo in e esse que mos am pela minha á ea mesmo que não pe cebam semp e. E ambém ao meu i mão po me emp es a equipamen os e po me ajuda quando a ecnologia não que se minha aliada. Mui o ob igada a odos. Bia iii STATEMENT OF INTEGRITY I he eby decla e ha ing conduc ed his academic wo k wi h in eg i y. I con i m ha I ha e no used plagia ism o any o m o undue use o in o ma ion o alsi ica ion o esul s along he p ocess leading o i s elabo a ion. I u he decla e ha I ha e ully acknowledged he Code o E hical Conduc o he Uni e si y o Minho. i ABSTRACT Pa kinson’s disease (PD) is he second mos common neu odegene a i e diso de a e Alzheime 's disease. PD onse is a 55 yea s-old on a e age, and i s incidence inc eases wi h age. This disease esul s om dopamine-p oducing neu ons degene a ion in he basal ganglia and is cha ac e ized by a ious mo o symp oms such as eezing o gai , b adykinesia, hypokinesia, akinesia, and igidi y, which nega i ely impac pa ien s’ quali y o li e. To moni o and imp o e hese PD- ela ed gai disabili ies, se e al echnology-based me hods ha e eme ged in he las decades. Howe e , hese solu ions s ill equi e mo e cus omiza ion o pa ien s’ daily li ing asks in o de o p o ide mo e objec i e, eliable, and long- e m da a abou pa ien s’ mo o condi ions in home- ela ed con ex s. P o iding his quan i a i e da a o physicians will ensu e mo e pe sonalised and be e ea men s. Also, mo o ehabili a ion sessions os e ed by assis ance de ices equi e he inclusion o quo idian asks o ain pa ien s o hei daily mo o challenges. One o he mos p omising echnology-based me hods is i ual, augmen ed, and mixed eali y (VR/AR/MR), which imme se pa ien s in i ual en i onmen s and p o ide senso y s imuli (cues) o assis wi h hese disabili ies. Howe e , u he esea ch is needed o imp o e and concep ualize e icien and pa ien -cen ed VR/AR/MR app oaches and inc ease hei clinical e idence. Bea ing his in mind, he main goal o his disse a ion was o design, de elop, es , and alida e i ual en i onmen s o assess and ain PD- ela ed gai impai men s using mixed eali y sma glasses, in eg a ed wi h ano he high- echnological mo ion acking de ice. Using speci ic i ual en i onmen s ha igge PD- ela ed gai impai men s ( u ning, doo ways, and na ow spaces), i is hypo hesized ha pa ien s can be assessed and ained in hei daily challenges ela ed o walking. Also, his ool in eg a es on-demand isual cues o p o ide isual bio eedback and os e mo o aining. This solu ion was alida ed wi h end-use s o es he iden i ied hypo hesis. The esul s showed ha , in ac , mixed eali y has he po en ial o ec ea e eal-li e en i onmen s ha o en p o oke PD- ela ed gai disabili ies, by placing i ual objec s on op o he eal wo ld. On he con a y, bio eedback s a egies did no signi ican ly imp o e he pa ien s’ mo o pe o mance. The use expe ience e alua ion showed ha pa icipan s enjoyed pa icipa ing in he ac i i y and el ha his ool can help hei mo o pe o mance. Keywo ds: Pa kinson’s disease; Vi ual eali y; Augmen ed eali y; Mixed Reali y; Rehabili a ion; Gai disabili ies; Senso y cueing; Bio eedback. RESUMO A doença de Pa kinson (DP) é a segunda doença neu odegene a i a mais comum depois da doença de Alzheime . O início da DP oco e, em média, aos 55 anos de idade, e a sua incidência aumen a com a idade. Es a doença esul a da degene ação dos neu ónios p odu o es de dopamina nos gânglios basais e é ca ac e izada po á ios sin omas mo o es como o congelamen o da ma cha, b adicinesia, hipocinesia, acinesia, e igidez, que a e am nega i amen e a qualidade de ida dos pacien es. Nas úl imas décadas su gi am mé odos ecnológicos pa a moni o iza e eina es as desabilidades da ma cha. No en an o, es as soluções ainda eque em uma maio pe sonalização ela i amen e às a e as diá ias dos pacien es, a im de o nece dados mais obje i os, iá eis e de longo p azo sob e o seu desempenho mo o em con ex os do dia-a-dia. A a és do o necimen o des es dados quan i a i os aos médicos, se ão assegu ados a amen os mais pe sonalizados. Além disso, as sessões de eabili ação mo o a, p omo idas po disposi i os de assis ência, eque em a inclusão de a e as quo idianas pa a eina os pacien es pa a os seus desa ios diá ios. Um dos mé odos ecnológicos mais p omisso es é a ealidade i ual, aumen ada e mis a (RV/RA/RM), que ime gem os pacien es em ambien es i uais e o necem es ímulos senso iais pa a ajuda nes as desabilidades. Con udo, é necessá ia mais in es igação pa a melho a e concep ualiza abo dagens RV/RA/RM e icien es e cen adas no pacien e e ainda aumen a as suas e idências clínicas. Tendo is o em men e, o p incipal obje i o des a disse ação oi concebe , desen ol e , es a e alida ambien es i uais pa a a alia e eina as incapacidades de ma cha elacionadas com a DP usando óculos in eligen es de ealidade mis a, in eg ados com ou o disposi i o de as eio de mo imen o. U ilizando ambien es i uais especí icos que desencadeiam desabilidades da ma cha ( oda , po as e espaços es ei os), é possí el es a hipó eses de que os pacien es possam se a aliados e einados nos seus desa ios diá ios. Além disso, es a e amen a in eg a pis as isuais pa a o nece bio eedback isual e omen a a eabili ação mo o a. Es a solução oi alidada com u ilizado es inais de o ma a es a as hipó eses iden i icadas. Os esul ados mos a am que, de ac o, a ealidade mis a em o po encial de ec ia ambien es da ida eal que mui as ezes p o ocam de iciências de ma cha elacionadas à DP. Pelo con á io, as es a égias de bio eedback não p o oca am melho ias signi ica i as no desempenho mo o dos pacien es. A a aliação ei a pelos pacien es mos ou que es es gos a am de pa icipa nos es es e sen i am que es a e amen a pode auxilia no seu desempenho mo o . Pala as-Cha e: Doença de Pa kinson; Realidade i ual; Realidade aumen ada; Realidade mis a; Reabili ação; Desabilidades mo o as; Pis as senso iais; Bio eedback. i Table o Con en s 1 In oduc ion .............................................................................................................. 1 1.1 Mo i a ion ................................................................................................................... 2 1.2 P oblem s a emen ...................................................................................................... 4 1.3 Goals ........................................................................................................................... 5 1.4 Resea ch Ques ions ...................................................................................................... 6 1.5 Con ibu ions o Knowledge ......................................................................................... 6 1.6 Disse a ion S uc u e .................................................................................................. 7 2 Li e a u e Re iew ...................................................................................................... 8 2.1 In oduc o y Insigh ...................................................................................................... 9 2.2 Me hods .................................................................................................................... 10 2.2.1 Da a sou ces, sea ch s a egy and s udies selec ion ....................................................... 10 2.3 Resul s ....................................................................................................................... 10 2.3.1 Gene al Resul s ................................................................................................................ 10 2.3.2 VR/AR/MR in PD .............................................................................................................. 11 2.3.3 Technology suppo ing VR/AR/MR-based app oaches in PD .......................................... 13 2.3.4 Valida ion me hodology highligh s: pa icipan s, c i e ia s udy, se ing, p o ocols, Schedule, me ics.......................................................................................................................... 16 2.4 Discussion .................................................................................................................. 27 2.4.1 How ha e he VR/AR/MR-based app oaches been applied in PD o help pa ien s mi iga e gai disabili ies? .............................................................................................................. 27 2.4.2 Which echnologies ha e been used o suppo VR/AR/MR-based app oaches in PD? . 27 2.4.3 How ha e he VR/AR/MR-based app oaches been clinically alida ed in PD? ............... 28 2.5 Conclusions and Fu u e di ec ions .............................................................................. 29 3 Solu ion O e iew ................................................................................................... 32 3.1 P oblem desc ip ion ................................................................................................... 33 3.2 +sense ....................................................................................................................... 33 3.3 +sImme si e ............................................................................................................... 34 3.3.1 Mixed eali y sma glasses: Mic oso HoloLens 2 ......................................................... 35 3.3.2 Mo ion acking sys em: Xsens MVN Awinda ................................................................. 37 3.4 Conclusions ................................................................................................................ 39 4 Solu ion desc ip ion ................................................................................................ 40 xiii M MMSE Mini-Men al S a e Examina ion MoCA Mon eal Cogni i e Assessmen Mini- BESTes Mini-Balance E alua ion Sys ems Tes MR Mixed Reali y M1 Moni o ing es 1 - dice M2 Moni o ing es 2 - doo M3 Moni o ing es 3 – na ow spaces P PD Pa kinson’s Disease PIGD Pos u al Ins abili y and Gai Diso de PTF Pe cen age o ime ozen PC Pe sonal compu e PD- Pa kinson’s Disease pa ien s wi h eezing o gai ( eeze s) PDQL Pa kinson’s Disease Quali y o Li e ques ionnai e Q QoL Quali y o Li e S SI Semi imme si e SSQ Simula o Sickness Ques ionnai e SUS Sys em Usabili y Scale SAC S ess A ousal Checklis T T T aining TUG Timed Up and Go es TC1 Con ol es 1 TC2 Con ol es 2 T1 T aining es 1 – dice wi h a ows T2 T aining es 2 – doo wi h oo p in s T3 T aining es 3 – na ow spaces wi h oo p in s U xi UPDRS Uni ied Pa kinson’s Disease Ra ing Scale UPSRS-III Uni ied Pa kinson’s Disease Ra ing Scale pa III V VR Vi ual Reali y VGoT Videogame-o ien ed aining 1 INTRODUCTION 2 This disse a ion p esen s he wo k ca ied ou o e he pas yea , in eg a ed in he scope o he Mas e Deg ee in Biomedical Enginee ing a he Biomedical Robo ic De ices Lab (BiRDLAB) included in he Cen e o Mic oElec oMechanical Sys ems (CMEMS), a esea ch cen e o he Depa men o Indus ial Elec onics (DEI) o Uni e si y o Minho. The p ojec main goal was o de elop and alida e a mixed eali y (MR) ool o he assessmen and aining o gai disabili ies in Pa kinson’s disease. This solu ion was de eloped o b ing a new pa adigm shi . Thus, by igge ing PD- ela ed gai impai men s, pa ien s can be assessed and ained in e e yday si ua ions. The po en ial o mixed eali y, in eg a ed wi h a mo ion acking sys em, o mimic e e yday en i onmen s, was es ed, aiming a mo e objec i e medical assessmen , and enhanced and mo i a ional ehabili a ion exe cises. All he s eps pe o med o achie e his solu ion a e de ailed in his documen . 1.1 MOTIVATION Pa kinson’s disease (PD) is he second mos common neu odegene a i e diso de a e Alzheime 's disease [1]. I is belie ed ha PD physiopa hology lies in he loss o dopamine-gene a ing neu ons in he basal ganglia, which is ela ed o human mo emen con ol. The i s dyskinesias appea when he e is a de iciency in dopamine elease by hese cells [1], [2]. I s a e age onse is a 55 yea s-old and i s incidence inc eases wi h age [1]. This disease is cha ac e ized by se e al symp oms wi h gai impai men s being he mos common and disabling ones. Mo o symp oms include eezing o gai (FoG), b adykinesia (mo emen slowness), hypokinesia ( educed mo emen ampli ude), akinesia (p oblems ini ia ing mo emen ), es ina ion ( endency o speed up when pe o ming epe i i e mo emen s), igidi y, pos u al ins abili y, and educed mo emen au oma ici y, all o hem diminishing pa ien s’ quali y o li e [1]–[7]. Usually, pa ien s s a by educing hei walking eloci y, aking sho e s eps, and p esen ing some gai asymme ies and e en ually su e ing mo o eezing e en s. FoG is de ined as he “ sudden inabili y o con inue walking despi e he in en o main ain locomo ion” and “i is episodic and a iable by na u e ”, being one o he mos debili a ing and di icul impai men s o assess [3], [4], [7]. FoG e en s a e ypically igge ed by speci ic si ua ions, such as ini ia ion o walking, u ning du ing s eady-s a e walking, acing objec s, s ess, dis ac ion, and nea ing doo ways. E en i hese en i onmen s did no igge a comple e momen o gai blockage, hey con ibu ed o a dec ease in he s ep leng h and eloci y, occu ing es ina ion and akinesias. Mo eo e , hese gai disabili ies can be agg a a ed by dual- ask a en ional equi emen s [2], [3], [6]–[8]. 3 Mo ion acking sys ems make i possible o moni o pa ien s’ mo o unc ion using wea able senso s [9], while elec omyog aphy sys ems acqui e elec ical muscle ac i i y, e lec ing mo o luc ua ions [10]. Bo h sys ems a e a he o e on o mo o unc ion’s moni o ing, enabling o ga he da a wi h low-cos , po able, and minia u ized senso s. I has been possible o moni o kinema ic and elec omyog aphy-d i en in o ma ion abou pa ien s’ mo o condi ions, such as hei gai spa io empo al pa ame e s [2], [11], [12], FoG e en s du a ion and occu ence [2], [13]–[15], muscles ac i i y [10], o pos u al changes [11], [12]. Indeed, his in o ma ion ep esen s i ial da a o physicians o moni o , o e ime, he mo o s a e o hei pa ien s, con olling he p og ession o he disease, especially i hey can access hese da a collec ed on pa ien s’ home en i onmen s, du ing hei daily asks. This would esul in g ea e supe ision o he de elopmen o he disease and would allow ea men s o be pe sonalised o he pa ien . Despi e he con inuous p og ess o hese echnological solu ions in he con inuous moni o ing o PD-associa ed gai disabili ies, i is s ill di icul o easibly assess and ain pa ien s o hei common daily asks. Se e al esea che s ha e dedica ed hemsel es o he s udy o he neu ological o igin o hese impai men s in o de o cus omize ea men me hodologies o PD- ela ed gai disabili ies [16]. Despi e he scien i ic ad ances, di e en hypo heses a e s ill poin ed ou . I is only known ha he e may be a ailu e in he ac i i y o ne e messages be ween he cen al ne ous sys em and he e e en muscles, esponsible o mo emen , ha can cause mo o symp oms in PD [17]– [19]. To o e come hese mo o symp oms, esea che s poin ed ou cueing-based in e en ions [5]. These s a egies in ol e he use o ex e nal empo al o spa ial s imuli o acili a e mo emen , in he o m o isual, audi o y and ib o ac ile cues. These cues con ibu ion consis s in bypassing aul s in ne ous messages ha may be a he o igin o gai impai men s [17]–[19]. Indeed, hese senso y cueing s a egies a e in eg a ed in o bio eedback de ices, which ha e al eady been explo ed in he PD ield. These sys ems make use o wea able echnology o p o ide senso y acquisi ion and igge a cue in o ma ion (bio eedback). They can de ec a dec ease in cadence o a change o he lowe leg muscle ac i i y, and h ough he de ec ion o such mo o beha io s deli e senso y cues [8], [17]. Thus, hese in e en ions could lead o a change in pos u al con ol, s ep pa e n, and un eeze gai eezing e en s, p e en alls, and consequen ly could p omo e less a iabili y in gai and a mo e goal-o ien ed gai . Fu he , wea able sys ems allow hei in eg a ion in o pa ien s’ e e yday asks, ensu ing g ea e eedom o mo emen and com o [9]. Howe e , o he bes o knowledge, he e ec i eness o hese echnologies o ehabili a ion has seldom been in es iga ed and alida ed in eal-li e si ua ions. Thus, he use o i ual en i onmen s o imme se pa ien s in hose si ua ions could po en ia e 4 he bio eedback in e en ions. Fu he , hese s udies did no ollow a pa ien -cen ed app oach, some did no use ully wea able sys ems and did no include modula sys ems, meaning hey a e no easy o in eg a e wi h o he echnologies [2], [17]. Las ly, pa ien s and physicians we e a ely included in he de elopmen phase and he e we e e y ew accu a e and objec i e e alua ion me ics, plus i was unusual o use unc ional mo emen aining. Vi ual, augmen ed, and mixed eali y eme ged in he las decades as a p omising s a egy o allow pa ien s’ imme sion in cus omized i ual en i onmen s. In ac , hese pe sonalised i ual and in e ac i e en i onmen s allow pa ien s o be placed in si ua ions whe e hey can pe o m daily asks, ob aining mo e easible and na u al mo ion da a. When a modula de elopmen a chi ec u e is used, his VR/AR/MR equipmen may be in eg a ed wi h o he moni o ing and ac ua ion de ices, os e ing pa ien s’ mo o assessmen and aining. In his sense, in o de o o e come he limi a ions encoun e ed, his disse a ion aims o explo e he use o mixed eali y (MR) in he assessmen and aining o PD pa ien s du ing hei mo o asks. By de eloping a obus sys em o MR in eg a ed wi h a high echnological mo ion acking de ice, capable o assessing PD- ela ed gai disabili ies, i is expec ed o show how imme si e, in e ac i e MR echnology can o e a new me hodological amewo k o moni o ing and aining gai - ela ed beha iou s in PD. 1.2 PROBLEM STATEMENT Mo e eliable assessmen and consis en aining gea ed owa ds daily asks a e needed, and his can be achie ed by imme sing PD pa ien s in i ual en i onmen s o imp o e hei mo o unc ion assessmen and aining. I is expec ed o (i) explo e he use o MR o de elop and design i ual en i onmen s close o pa ien s’ daily eali y; (ii) de elop a modula a chi ec u e capable o in eg a ing he MR app oach wi h a mo o assessmen de ice; (iii) in es iga e he po en ial o senso y cues in imp o ing gai impai men s h ough augmen a i e cues. To add ess hese p oblem s a emen s, i is c ucial o ollow a use -o ien ed app oach capable o de eloping a mo o assessmen and aining s a egy close o pa ien s’ daily needs. This disse a ion will adop a sys ema ic app oach o answe hese key cons ain s. 5 1.3 GOALS The ul ima e goal o his disse a ion was o design, de elop, es , and alida e h ee di e en i ual en i onmen s, which ec ea e e e yday si ua ions, o assess and ain PD- ela ed gai disabili ies using MR sma glasses and a mo ion acking sys em. One o he mos dis inguishable ea u es o his disse a ion is i s mul i-disciplina y na u e spanning om neu osciences o algo i hms o MR, modula a chi ec u es, wea able senso s, and bio eedback s a egies. Thus, his wo k equi ed dealing wi h exis ing and on -end ha dwa e, designing i ual en i onmen s, gai analysis and segmen a ion, alida ing p o ocols wi h end use s, and da a analysis. To each his main goal, he ollowing s ep-goals and Key Pe o mance Indica o s (KPI) needed o be de ined and achie ed: Goal 1: Ga he knowledge abou VR/AR/MR s a egies used in PD o mo o aining and assessmen , h ough li e a u e e iews, o answe he ollowing ques ions: (i) “How ha e he VR/AR/MR-based app oaches been applied in PD o help pa ien s mi iga e gai disabili ies?”; (ii) “Which echnologies ha e been used o suppo VR/AR/MR-based app oaches in PD?”; and (iii) “How ha e he VR/AR/MR-based app oaches been clinically alida ed in PD?”. This goal ela es o KPI 1: summa ising he li e a u e h ough a leas wel e a icles; wha a e he mos common gai disabili ies in PD; wha a e he eal-wo ld si ua ions ha mos cause PD- ela ed gai disabili ies. Chap e 2 p esen s hese su eys. Goal 2: Implemen a ion o a modula , use -cus omised, echnological solu ion based on mixed eali y o imme se pa ien s in scena ios ha can igge PD- ela ed gai disabili ies. Based on he li e a u e e iew, i ual en i onmen s ha e oke PD- ela ed gai disabili ies and ha a e cus omisable acco ding o he use s' heigh will be de eloped. This will add ess he limi a ions iden i ied in VR/AR/MR-based app oaches in PD. This goal ela es o KPI 2: de elopmen o h ee i ual en i onmen s ha ep esen si ua ions ha ypically cause PD- ela ed gai disabili ies. Chap e 3 and Chap e 4 desc ibe he ma e ials and p ocedu es o his solu ion. Goal 3: Implemen a ion o a modula , use -cus omised, echnological solu ion based on mixed eali y in eg a ed wi h ano he high- echnological mo ion acking de ice o help pa ien s o e come PD- ela ed gai disabili ies. Ou comes co e he in eg a ion o a mo ion acking sys em wi h MR echnologies based on combined isual senso y cues, mo ion analysis and augmen ed eali y. This goal ela es o KPI 3: de elopmen o on-demand isual bio eedback s a egies in eg a ed in HoloLens 2; de elopmen o a eal- ime ini ial and inal con ac de ec ion algo i hm wi h a pe o mance highe han 96% o accu acy. Chap e 3 and Chap e 4 de ail he ma e ials, me hods and algo i hms used and de eloped o achie e his solu ion. 6 Goal 4: Valida ion o he p oposed MR s a egy wi h end-use s. I is in ended o collec and analyse da a o assess he usabili y, e iciency, and accep abili y o implemen ed s a egies (use -cen ed app oach). This goal ela es o KPI 4: alida ion o he solu ions wi h a leas en end-use s; s a is ically signi ican di e ences in spa io empo al pa ame e s be ween con ol and moni o ing es s, and la e be ween moni o ing and aining es s; SSQ sco e lowe han six een poin s; IMI sco e g ea e han i e poin s. Chap e 5 will e eal he ob ained esul s. 1.4 RESEARCH QUESTIONS Conside ing he ul ima e goal o his disse a ion and he s ep-goals p esen ed, ele an esea ch ques ions (RQs) we e iden i ied, as ollows: RQ 1: How ha e he VR/AR/MR-based app oaches and echnologies been applied o suppo PD pa ien s and how ha e hey been clinically alida ed? This ques ion ela es o Goal 1 and is answe ed in Chap e 2. RQ2: How o implemen a modula , use -cus omised, mixed eali y-based echnology solu ion ha imme ses pa ien s in en i onmen s ha (1) cause PD- ela ed gai impai men s; and ha (2) help o e come hese impai men s wi h he aid o a mo ion acking sys em and bio eedback s a egies? This issue conside s Goal 2 and Goal 3. The answe is de eloped h oughou Chap e 3 and Chap e 4. RQ3: How does he implemen ed modula echnological solu ion, based on mixed eali y in eg a ed wi h a mo ion acking sys em and wi h bio eedback s a egies, a ec he mo o pe o mance o PD pa ien s du ing assessmen and aining? This ques ion is linked o Goal 4 and is answe ed in Chap e 5. The p esen ed RQs a e summa ized and answe ed in Chap e 6. 1.5 CONTRIBUTIONS TO KNOWLEDGE The main con ibu ions o his disse a ion o knowledge a e: • Re iew on VR/AR/MR-based app oaches cu en ly deployed in PD o ain and assess gai disabili ies. • De elopmen o h ee i ual en i onmen s and i ual asks ha cause gai impai men s, cus omisable o each pa icipan . 7 • Implemen a ion and alida ion o an algo i hm o de ec ing ini ial and inal con ac s, in eal ime. Also, in his scope, an algo i hm o es ima e spa io empo al me ics was implemen ed, based on gai segmen a ion. • De elopmen o wo isual bio eedback s a egies, cus omisable o each pa icipan , o help o e come PD- ela ed gai impai men s. I is expec ed ha he de eloped wo k will lead o he elabo a ion o a jou nal a icle. Du ing his pe iod, I had he p i ilege o guiding wo s uden s o he In eg a ed Mas e s in Elec onic Enginee ing, in a p ojec o he cu icula uni "P oje o In eg ado ". In addi ion, I applied o a g an om he "Ve ão com Ciência" p og amme o he Fundação pa a a Ciência e Tecnologia (FCT). 1.6 DISSERTATION STRUCTURE This manusc ip is o ganized in o six chap e s, as ollows. Chap e 1 p esen s he mo i a ion, p oblem s a emen and he ul ima e goals o his wo k. Chap e 2 ou lines a comp ehensi e e iew o cu en li e a u e abou VR/AR/MR-based app oaches o s udy PD- ela ed gai disabili ies. The VR/AR/MR-based app oaches a e p esen ed and discussed, ega ding he VR/AR/MR echnology, embedded senso s, i ual asks, and clinical ou comes. The chap e inishes wi h a summa y o he indings. Chap e 3 add esses he o e iew o he solu ion. I s a s by desc ibing he p oblem. Then, p ojec +sense and i s modules a e p esen ed, as well as a desc ip ion o he ha dwa e included in he s a egy, men ioning i s need and echnical cha ac e is ics. Chap e 4 ou lines he solu ion desc ip ion. Fi s ly, an in oduc o y insigh is p esen ed, desc ibing he se up o be used. Secondly, he use -cen ed design o he solu ion is p esen ed, iden i ying he i ual asks and en i onmen s designed. Thi dly, he in eg a ion o he senso y sys em is desc ibed, s a ing by explaining he eal- ime ini ial and inal con ac s de ec ion algo i hm, up o he es ima ion o spa io empo al me ics, pe o med o line. Finally, he in eg a ion o he bio eedback s a egies is explained. Chap e 5 p esen s he alida ion p o ocol, esul s, and a c i ical discussion o he solu ion. Fu he mo e, i p esen s hei limi a ions and possible explana ions, along wi h esea ch sugges ions and imp o emen s. Chap e 6 concludes he disse a ion, while p o iding a b ie analysis o he p ojec and i s esul s, along wi h u u e esea ch insigh s. 2 LITERATURE REVIEW 15 I was obse ed ha he VR/AR/MR equipmen used by he selec ed s udies we e he Oculus Ri DK2 [2], [17], [28], HTC Vi e [11], [12], [20], [25], Google Glass [13], [23], HoloLens [15], [18], and HTC Vi e P o [22], [26]. Fu he mo e, in [21] a mic o display was a ached o he eyeglasses ame and in [14] a p o o ype o cus om-made sma glasses was designed. Finally, in [24] and [27] sma glasses we e no wo n, on he con a y, a compu e assis ed i ual eali y en i onmen (CAREN) and a C-Mill VR+ eadmill was used, espec i ely. The i ual en i onmen imme sion anged om ully imme si e [2], [11]–[15], [17], [18], [20]–[26], [28] o semi-imme si e [27], being equen ly used head-moun ed displays. Vi ual asks included mo o ac i i ies, like climbing s ai s [14], u ning [14], [15], and walking s aigh [11], [13], [14], [18], [21], [24], [27], [28], ( i.e., on a hallway as in [12], [17], [20]), o speci ic con ex s ha could igge PD- ela ed gai disabili ies, such as c ossing i ual [2], [12], [20] o eal doo s [13]. Addi ionally, when VR/AR/MR was applied o mo o aining s a egies, a cue-o ien ed game was used in [22], while in [23]–[26] pa ien s ollowed he asks indica ed on he i ual game, namely, dance [23], na iga e a i ual boa [24], d i e a ball o he inish line [24], smash lying objec s [24], comple e i ual wo ds [25] o play a box game [26]. An o e iew o some i ual asks and en i onmen s de eloped a e shown in Figu e 2-3. Rega ding he acquisi ion module, h ee sys ems we e ound: In e Sense IS900 [2], [17], Qualysis [20], IMU [13]–[15], accele ome e (G-Senso ) [24] and Vi e acke s [22], [25], wi h none o hem being buil in. IMUs we e placed in bo h ull body [14], [15] and lowe body [11], [13] con igu a ions. Wi h espec o he ac ua ion module, i was iden i ied he ype and which de ice we e used, and whe he i was buil in. All sys ems which had an ac ua ion module used buil -in ac ua o s, such as augmen a i e isual cues o ea phones [13]–[15], [17], [18], [21], [22], [24]. Howe e , [15] also had a non-buil -in ac ua o , namely, a speake o audi o y cueing. Fu he mo e, [24] used ou ypes o ac ua o s ( isual, audio, es ibula and ac ile), [13] used h ee ypes o ac ua o s (audio, lashing ligh and op ic low), [15] used wo ypes o ac ua o s ( isual and audio) whe eas [14], [17], [18], [21], [22] only used one ype o ac ua o , isual. Besides, isual cues we e used mo e han audi o y cues. 16 2.3.4 VALIDATION METHODOLOGY HIGHLIGHTS: PARTICIPANTS, CRITERIA STUDY, SETTING, PROTOCOLS, SCHEDULE, METRICS Table 2-3 summa izes he alida ion me hodology o he selec ed s udies. I highligh s he pa icipa ion and e alua ion o PD pa ien s, inclusion and exclusion c i e ia o pa icipan s selec ion, se ing, expe imen al p o ocols, schedule, and he esea ch e alua ion me ics. Uni ied Pa kinson’s Disease Ra ing Scale pa III (UPDRS-III) [2], [12]–[15], [17], [18], [20], [23]– [25], [27], [28] and Hoehn and Yah scale (H&Y) [11], [12], [14], [15], [18], [20], [21], [23]–[25], [27], [28] we e he mos commonly used a ing scales o symp oms o PD. Along wi h UPDRS-III, Pos u al Ins abili y and Gai Diso de sub-sco e (PIGD) [14], Ac i i ies-speci ic Balance Con idence (ABC) scale [11], [12], [22], [23], [28], Mini-Balance E alua ion Sys ems Tes (Mini-BESTes ) [11], [12], [20], [25], [28] and UPDRS-II [24] we e used o e lec he e olu ion o mo o unc ion. To indica e he pa ien s’ cogni i e and men al s a e, he Mini-Men al S a e Examina ion (MMSE) scale was used in [11], [14], [15], [27], Mon eal Cogni i e Assessmen (MoCA) in [12], [18], [20], [23], [28] and F on al Assessmen Ba e y (FAB) scale in [13]–[15]. FOG-ques ionnai e (FOG-Q) was used in [2], [12]–[15], [17], [18], [20]. To assess simula o sickness symp oms, he Simula o Sickness Ques ionnai e (SSQ) was used [25]. Figu e 2-3 - O e iew o some i ual asks and en i onmen s aken om [2], [3], [12], [13], [19], [20], [22]–[26]. 17 Table 2-3 – Clinical Highligh s o he de eloped VR/AR/MR echnologies o PD pa ien s, o e he las en yea s Goal Pape Pa icipan s C i e ia S udy Se ing P o ocol Schedule Me ics N Scales Inclusion Exclusion A [2] 10 - UPDRS-III; - FOG-Q; - - Labo a o y Walk unde 3 di e en i ual condi ions: (1) no doo ; (2) na ow doo way; (3) s anda d doo way. Single isi : (1) amilia isa ion phase; (2) 18 ials (6x each condi ion). To al: 20min - s ep cadence (mean and CV); - s ep eloci y (mean and CV); - s ep leng h (mean and CV); - du a ion o FoG episodes (mean and SD); - % ials wi h a FoG episode [11] 10 - UPDRS; - Mini- BESTes ; - ABC scale; - H&Y; - MMSE; - Diagnosis o PD; - - H&Y≤3; - MMSE > 24/30; - No o he pa hology in e ac ing wi h gai o causing dizziness; - No unco ec ed isual de iciency; - Abili y o walk 512 consecu i e s ides (±10– 15 min); - Labo a o y Walk in a andomized o de in 3 condi ions: (1) O e g ound Walking; (2) T eadmill Walking (3) imme si e Vi ual Reali y on T eadmill Walking Single isi - speed; - s ep leng h; - cadence; - SSQ [12] 10 - MDS- UPDRS-III; - NFoGQ; - MoCA; - Mini-BEST; - ABC Scale; - H&Y; - Diagnosis o PD; - Sel - epo ed FoG; - Sel - epo ed abili y o walk 400m wi hou assis ance om a de ice o ano he pe son; - No diagnosis o demen ia; - No unco ec ed ision o hea ing p oblems; - Labo a o y Walk in 5 en i onmen s: (1) Physical labo a o y wi hou VR; (2) i ual labo a o y wi hou obs acles; (3) i ual doo way; (4) i ual hallway; (5) i ual s ee scene wi h c owds Single isi - gai speed; - s ep leng h (mean and CV); - s ep wid h; - s ep ime; - s ep ime asymme y; - es ina ion; - SSQ [20] 12 - MoCA; - NFOGQ; - Mini-BEST; - Diagnosis o PD wi hou demen ia; - Labo a o y Walk unde 4 condi ions: (1) physical labo a o y; (2) i ual labo a o y; (3) i ual doo way; Single isi - kinema ic a iables; - gai speed; - s ep leng h (mean and CV); - s ep ime; 18 - MDS- UPDRS-III; - H&Y; - Sel - epo ed o clinician-obse ed FoG; - Abili y o walk 400 m wi hou assis ance om a de ice o ano he pe son; - No unco ec ed ision o hea ing de ici s; (4) i ual hallway. - s ep ime asymme y; - s ep wid h; - DLS; - es ina ion; - SSQ CoA [13] 12 “end-o - dose” - UPDRS-III; - NFOGQ; - FAB; - P esence o FoG mo e han wice pe day; - Able o walk 20m o e a la su ace wi hou walking aids; - Signi ican cogni i e impai men s; - Como bidi ies ha impai ed gai , o isual impai men s; Labo a o y Walk on 4 di e en walking cou ses in combina ion wi h 4 cueing condi ions: (1) me onome; (2) lashing ligh ; (3) op ic low; (4) no cue. Single isi : (1) amilia iza ion phase; (2) 16 di e en cue-cou se combina ions (2 ials each). To al: 2.5h - no. o FoG episodes; - du a ion o FoG episodes; - s ide leng h (mean and SD); - speed; - cadence (mean and SD); - in e iew (use expe ience) [18] 24 “on s a e” - UPDRS-III; - H&Y; - MoCA; - NFOGQ; - Olde han 18 yea s; - Diagnosis o PD; - Expe ience FOG in he dopamine gic “ON” s a e; - Addi ional neu ological diseases and/o o hopedic p oblems; - Inabili y o walk independen ly; Home/Labo a o y (1) HOME: walk a eezing p o oking ou e mul iple imes wi h and wi hou wea ing he HoloLens (wi hou Holocue); (2) LABORATORY: amilia ize pa icipan s o walking wi h he (on-demand) holog aphic cues; (3) HOME: equal o session 1 bu wea ing he HoloLens wi h and wi hou he Holocue 3 sessions o 1.5h, one week apa - no. o FoG episodes; - a e age du a ion o FoG episodes; - o al du a ion o FoG episodes; - % ime ozen (PTF) CoT [21] 20 - H&Y - - conside able isual de ici no compensa ed by co ec ion; - ocula mo emen dys unc ion; - gai dis u bances due o neu omuscula diseases; Labo a o y Walk a s aigh ack o 10 m: (1) baseline; (2) online display o ; (3) online display on; (4) esidual e ec s; (5) examina ion; Single isi - speed; - s ide leng h [14] 25 “end-o - dose” - UPDRS-III; - UPDRS- PIGD; - Diagnosis o PD; - Olde han 18 yea s old; - His o y o s oke; - Psychia ic disease; Labo a o y Walk on 3 di e en walking cou ses in combina ion wi h 5 cue condi ions: Single isi : - no. o FOG episodes; - % eezing ime; - s ide leng h (mean and SD); 19 - H&Y; - NFOGQ; - MMSE; - FAB; - P esence o FoG mo e han once pe day; - Se e e unco ec ed isual o hea ing impai men s; - Como bidi y limi ing ambula ion; - Inabili y o walk unaided; - Deep b ain s imula o o apomo phine pump; - Jejunal le odopa gel in usion; - MMSE sco e < 24; (1) augmen ed isual cue ba s; (2) augmen ed isual cue s ai cases; (3) con en ional 3D ans e se ba s on he loo ; (4) me onome; (5) no cueing. 2 sessions sepa a ed by 30min b eak To al: 2.5h-3h - cycle ime (mean and SD); - cadence; - speed; - in e iew (use expe ience) [17] 12 - UPDRS-III; - FOG-Q; - - Labo a o y Walk using isual cues: (1) 2 spa ial condi ions: 115% and 130% o an indi idual’s baseline s ep leng h and; (2) 3 di e en empo al condi ions: spa ial only condi ion, 100 and 125% baseline s ep cadence. Single isi : (1) amilia isa ion phase; (2) 6 di e en cueing condi ions (8x each condi ion). To al: 40min - S ep leng h (mean and CV); - S ep cadence (mean and CV); - S ep eloci y (mean and CV); (a baseline and pos in e en ion) [15] 16 “end-o - dose” - UPDRS-III; - H&Y; - MMSE; - NFOGQ; - FAB; - Diagnosis o PD; - P esence o FoG mo e han wice pe day; - MMSE sco e < 24; - FAB sco e < 13; - Como bidi y causing se e e gai impai men s; - Se e e bila e al isual o audi o y impai men s; - Inabili y o pe o m a 180◦ u n unaided; Labo a o y Pe o m a se ies o 180º u ns unde : (1) an expe imen al condi ion wi h AR isual cues and; (2) wo con ol condi ions: audi o y cues and no cues. Single isi : (1) 1 aining session: 3 blocks (15 ials each); (2) 2 expe imen al sessions: 3 blocks (15 ials each). - PTF; - no. o FoG episodes; - du a ion o FOG episodes; - cadence; - peak eloci y; - s ide ime (mean and CV); - s ep heigh (mean and CV); - max head-pel is sepa a ion; - ime o max head-pel is sepa a ion; - max medial CoM de ia ion; - u n ime; - in e iew (use expe ience) [22] 5 - - Diagnosis o PD; - H&Y I-III; - Able o walk independen ly; - Condi ions ha could ha e a ec ed exe cise unc ion; Labo a o y Play he game “T easu e Island Ad en u e” wi h and wi hou obs acles in combina ion wi h 3 le els: 35, 40, and 45cm be ween he isual cues 1 session pe week o 30min, o e 3 weeks - BBS; - ABC; - S ep dis ance; - Leg aising; (a baseline and pos in e en ion) 20 VgoT [23] 7 “ON” - UPDRS-III; - H&Y; - MoCA; - Diagnosis o PD; - H&Y > III; - MDS-UPDRS-III > 57; - Unable o wea o ope a e Google Glass; - Demen ia; Home Comple e a leas 3 modules o MTG pe day E e y day o 3 weeks - Mini-BESTes ; - one-leg s ance; - TUG; - dual- ask; - ABC scale; - BDI; - PDQL; - in e iew [24] 22 - UPDRS-II; - UPDRS-III; - H&Y; - Mini- BESTes ; - MMSE; - Diagnosis o PD; - H&Y≤3; - MMSE ≥ 24 - age > 85 yea s; - p esence o se e e medical and psychia ic illness po en ially in e e ing wi h he VR aining Labo a o y Comple e ou scena ios: (1) Na iga e a i ual boa h ough a slalom cou se; (2) walk ac oss he boa d; (3) d i e a ed ball, mo ing he oad up o he inish line; (4) swa a lying objec s ha eme ge along he pa h 20 con en ional physio he apy sessions + 3- mon h es + 20 sessios o CAREN aining - BBS; - TUG; - UPDRS-II; - UPDRS-III; - FES-I; - H&Y; - 10MWT; - s ide leng h; - cycle ime; - s ance phase/ ime; - swing phase/ ime; - pe cen age o single- and double-limb suppo ; - speed; - cadence; - s ep leng h; - s ep wid h [25] 9 “ON” - UPDRS-III; - H&Y; - Mini- BESTes ; - SSQ; - Diagnosis o PD; - No mo o luc ua ions; - H&Y I-III; - Olde han 18 yea s old; - Walking independen ly; - S able medica ion; - Uncon olled, in olun a y mo emen s (dyskinesia); - Musculoskele al inju ies; - Pain ha limi ed mo emen ; Labo a o y Comple e a puzzle ha consis ed o a wo d wi h missing le e s loca ed a eye le el in he i ual en i onmen 3 sessions o 30min each, o e 1 week To al: 1h30 - SSQ; - ITC-SOPI; - IMI; - SUS; [26] 4 - - H&Y II; - Inabili y o co ec ly espond o he assessmen p o ocol; - P esence o ca dio ascula , pulmona y, o Labo a o y Play he game BOX VR 2 sessions, 2 weeks apa ; 1s session: (1) amilia iza ion phase wi h S eam VR Home (9min); - SUS; - SSQ; - GEQ-pos game; - in e iew (use expe ience) 21 musculoskele al condi ion; - P esence o se e e isual loss; - Ve igo, epilepsy, and psychosis; (2) aining: game gym (3min); 2nd session: (1) amilia iza ion phase wi h TheBlue; (2) aining: game gym. [27] 29 PIGD + 23 non- PIGD 2h a e medica ion - UPDRS-III; - H&Y; - MMSE; - Diagnosis o p ima y PD; - <75 yea s old - H&Y s age I–III (“on” pe iod); - MMSE>24 (>20 o hose wi h only p ima y school educa ion); - Se ious complica ions o como bidi ies; - Special ea men equi ed o o he como bidi ies; - Deep b ain s imula ion o in i o implan s; - A ypical o seconda y PD; - Como bidi ies a ec walking; - se e e cogni i e, isual, and hea ing impai men ; - Using a psycho opic subs ance; Labo a o y Comple e 5 modules o C-Mill aining in each aining session 1 session o 30min pe day o 7 days: (1) amilia iza ion phase; (2) modules o C- Mill aining. - 10-me e walking es ; - TUG es ; - BBS; - Pos u e sway; - Gai adap abili y; - Bo g 6-20 Ques ionnai e; - pe cei ed isk o alling; - PDQL T [28] 11 “ON” - UPDRS-III; - H&Y; - MoCA; - ABC; - Mini- BESTes ; - Able o walk o 30 min on a eadmill; - 19<MoCA<30; - No o he neu ological diso de s; - Labo a o y Walk o 20 min on a eadmill while iewing a i ual ci y scene Single isi To al: 20min - CoP excu sion; - SSQ; - SAC; (a baseline and pos in e en ion) [Re .]: s udy e e ence; A: assessmen ; CoA: cue-o ien ed assis ance; CoT: cue-o ien ed aining; VGoT: ideogame-o ien ed aining; T: aining; UPDRS-III: Uni ied Pa kinson’s Disease Ra ing Scale pa III; FOG-Q: F eezin o Gai ques ionnai e; FAB: F on al Assessmen Ba e y; H&Y: Hoehn and Yah scale; MMSE: Mini-Men al S a e Examina ion; UPDRS-PIGD: Uni ied Pa kinson's Disease Ra ing Scale - Pos u al Ins abili y and Gai Diso de ; Mini-BesTes : Mini-Balance E alua ion Sys ems Tes ; SSQ: Simula o Sickness Ques ionnai e; MoCA: Mon eal Cogni i e Assessmen ; ABC: Ac i i ies-speci ic Balance Con idence scale; CV: coe icien o a ia ion; SD: s anda d de ia ion; PTF: pe cen age ime eezing; DLS: double limb suppo ; CoM: Cen e o Mass; BBS: Be g Balance Scale; ITC-SOPI: Independen Tele ision Commission Sense o P esence In en o y; IMI: In insic Mo i a ion In en o y; SUS: Sys em Usabili y Scale; GEQ-pos game: Game expe ience ques ionnai e-pos game; TUG: Timed Up and go Tes ; BDI: Beck Dep ession In en o y; PDQL: Pa kinson's Disease Quali y o Li e ques ionnai e; CoP: Cen e o P essu e; SAC: S ess A ousal Checklis . 22 Inclusion c i e ia included he diagnosis o PD [11], [12], [14], [15], [18], [20], [22]–[25], [27], abili y o walk independen ly [11]–[14], [18], [20], [22], [25], mo o luc ua ions absence [13]–[15], [25], musculoskele al inju ies absence [21], [25], lack o o he neu ological diso de s [14], [18], [24], [28], lack o se e e bila e al isual o audi o y impai men s [11]–[15], [20], [21], [26], [27], abili y o pe o m a 180º u ns unaided [15], lack o cogni i e impai men s [13], [23], lack o deep b ain s imula ion [27] o apomo phine pump and jejunal le odopa gel in usion [14], p esence o s able medica ion [25]. Fo he s udies ha e alua ed FoG, he p esence o his symp om was also an inclusion c i e ia [12]–[15], [18], [20]. Addi ionally, speci ic sco es o PD scales we e used o include pa icipan s: 19<MoCA>30 in [28]; MMSE>24 in [11], [14], [15], [24], [27]; FAB sco e>13 in [15]; H&Y s age I-III in [22], [25] , H&Y s age I-III while on medica ion in [27]; H&Y s age II in [26]; H&Y s age<III in [11], [23], [24]. Mo eo e , some s udies used clinical cha ac e is ics as inclusion c i e ia, namely, age: olde han 18 yea s old in [14], [18], [25]; younge han 75 yea s old [27] and younge han 85 yea s old [24]. Rega ding he alida ion scena ios, all [2], [11]–[15], [17], [20]–[22], [24]–[28] a icles conduc ed an in e en ion in a labo a o y se ing apa om [18], [23] which ollowed a home-based app oach. Those a icles ha e alua ed FoG used he ollowing me ics: du a ion o FoG episodes [2], [13], [15], [18], pe cen age o ials wi h a FoG episode [2], numbe o FoG episodes [13]–[15], [18] and pe cen age o eezing ime [14], [15], [18]. Mo eo e , gai - ela ed me ics we e used, such as s ep cadence (mean [2], [11], [13]–[15], [17], [24], coe icien o a ia ion (CV) [2], [17] and s anda d de ia ion (SD) [11], [13]), s ep eloci y (mean [2], [11]–[14], [17], [20], [21], [24], CV [2], [17] and SD [11], [24]), s ep leng h (mean and CV [2], [11], [12], [17], [20], [24]), s ide leng h (mean and SD [13], [14], [21], [24]), peak eloci y [15], s ep ime (mean and asymme y [12], [20]), s ide ime (mean and CV [15]), s ep wid h (mean [12], [20], [24] and SD [24]), es ina ion [12], [20], kinema ic a iables [20], double limb suppo (DLS) [20], s ep heigh (mean and CV [15]), maximum head pel is sepa a ion [15], ime o maximum head-pel is sepa a ion [15], maximum medial cen e o mass (CoM) de ia ion [15], u n ime [15], cycle ime (mean and SD [14], [24]), s ep dis ance [22], leg aising [22], s ance and swing phase [24], pe cen age o single limb suppo [24], pos u e sway [27], gai adap abili y [27], 10-me e walking es (10MWT) [24], [27] and cen e o p essu e (CoP) excu sion [28], in o de o analyse gai pe o mance. In e ms o balance analysis, Be g Balance Scale (BBS) [22], [24], [27], ABC scale [22], [23], Timed Up and Go Tes (TUG) [24], [27] and dual- ask [23], one-leg s ance [23], Mini-BESTes [23], Falls E icacy Scale In e na ional (FES-I) [24] and pe cei ed isk o alling [27] we e assessed. Fu he mo e, Simula o Sickness Ques ionnai e (SSQ) [11], [12], [20], [24]–[26], [28] was conduc ed o e alua e simula o 23 sickness symp oms, Independen Tele ision Commission Sense o P esence In en o y (ITC-SOPI) [25] o check pe cei ed sense o p esence, In insic Mo i a ion In en o y (IMI) [25] o sco e le els o mo i a ion, Sys em Usabili y Scale (SUS) [25], [26] o e alua e sys em o e all usabili y, Pa kinson's Disease Quali y o Li e Ques ionnai e (PDQL) [23], [27] o e alua e quali y o li e (QoL), Beck Dep ession In en o y (BDI) [23] o assess dep essi e diso de s a us, Bo g 6-20 Ques ionnai e [27] o check pa icipan s’ pe cei ed exe ion and a igue and S ess A ousal Checklis (SAC) [28] o assess s ess. Finally, in [13]–[15], [23], [26] an in e iew was conduc ed on use expe ience, and in [26] a game expe ience ques ionnai e-pos game (GEQ-pos game) was also unde aken. Gómez-Jo dana e al. [2] p oposed a s udy o assess i he p esence o i ual doo ways in a i ual en i onmen could induce FoG he same way eal doo ways do. Fo expe imen al p o ocols, he e we e h ee g oups, a g oup o heal hy pa icipan s as a con ol g oup, a g oup o PD pa ien s wi hou FoG and a g oup o PD pa ien s wi h FoG, named as eeze s (PD- ). All g oups walked along a hallway unde h ee di e en i ual condi ions (no doo , na ow doo way (100% o shoulde wid h) and s anda d doo way (125% o shoulde wid h)). The p esence o i ual doo s esul ed in a educ ion on s ep leng h and eloci y and an inc ease on gai a iabili y, wi h he wo s alues occu ing o PD- . The na ow doo was he one ha p o oked he mos FoG. Lheu eux e al. [11] aimed o assess he e ec s o adding an op ic low displayed h ough an imme si e i ual eali y headse du ing eadmill walking on gai . PD pa ien s we e ins uc ed o walk in a andomized o de in 3 condi ions: (i) o e g ound walking; (ii) eadmill walking; and (iii) imme si e i ual eali y on eadmill walking. As a esul , a g ea e s ep leng h and lowe cadence we e ob ained. SSQ was simila be ween he (ii) and (iii) condi ions. Yamagami e al. [12] in ended o in es iga e whe he i ual en i onmen s ha eplica e FoG- p o oking si ua ions would exace ba e gai impai men s associa ed wi h FoG compa ed o unobs uc ed VR and physical labo a o y en i onmen s. Pa icipan s pe o med a se ies o walking asks on i e di e en en i onmen s (physical labo a o y wi hou VR; i ual labo a o y wi hou obs acles; i ual doo way; i ual hallway; i ual s ee scene wi h c owds). The esul s showed ha FoG-p o oking VR en i onmen s could exace ba e gai impai men s ha a e ela ed o FoG. Besha a e al. [20] aimed o examine he e ec s o i ual doo ways and hallways on gai kinema ics among people wi h PD and FoG. Pa icipan s pe o med a se ies o walking asks on ou di e en condi ions (physical labo a o y; i ual labo a o y; i ual doo way; i ual hallway). As a esul , kinema ic changes commonly associa ed wi h FoG episodes we e ob ained. 24 Zhao e al. [13] in ended o e alua e hy hmic isual and audi o y cueing in a labo a o y se ing. Pa icipan s pe o med a se ies o walking asks on ou di e en walking cou ses (wide u n, na ow u n, ull u n, and doo way) in combina ion wi h h ee cues (me onome, lashing ligh and op ic low). A mo e s able gai pa e n wi h he aid o hese cues was ob ained bu FoG did no diminish signi ican ly. The me onome was mo e e ec i e han hy hmic isual cues and p e e ed by mo e pa icipan s. Gee se e al. [18] explo ed un amilia i y and habi ua ion e ec s associa ed wi h wea ing he HoloLens on FoG and e alua ed he po en ial immedia e e ec o Holocue on alle ia ing FoG in he home en i onmen . Pa ien s pe o med h ee sessions o 1.5h, scheduled one week apa . In he i s session, pa icipan s walked a eezing p o oking ou e mul iple imes wi h and wi hou wea ing he HoloLens (wi hou Holocue), in hei homes. Session 2 ook place in a labo a o y and consis ed o indi idually cus omise he cues o he Holocue applica ion in e ms o in e cue dis ance and p e e ed ype o cues and amilia ize pa icipan s o walking wi h he holog aphic cues. Finally, he las session ook place again a he pa ien s’ home. Pa icipan s walked he same ou e wi h he same condi ions as in session 1, while wea ing he HoloLens wi h and wi hou he Holocue applica ion. Wea ing he HoloLens (wi hou Holocue) did signi ican ly inc ease he numbe and du a ion o FOG episodes, bu his un amilia i y e ec disappea ed wi h habi ua ion o e sessions. Holocue had o e all no immedia e e ec on FOG, al hough objec i e and subjec i e bene i s we e obse ed o some indi iduals, mos no ably hose wi h long and/o many FOG episodes. Bada ny e al. [21] s udied he e ec s o isual eedback cues on gai . The i ual en i onmen consis ed o a i ual iled loo in a checke boa d a angemen . The expe imen al p o ocol was di ided in o 5 phases: (i) walking wi hou he de ice; (ii) walking wi h he de ice placed on bu wi h he display u ned o ; (iii) walking wi h he display u ned on; (i ) walking wi hou he de ice a e a 15-minu e b eak; and ( ) e-e alua ion o baseline pe o mance wi hou he de ice one week a e he i s examina ion. The esul s sugges ed ha wea ing he de ice u ned o esul ed in a negligible e ec o abou 2%. Wi h he display u ned on, 56% o he pa ien s imp o ed hei gai speed o s ide leng h o bo h. A e emo ing he de ice, 68% o he pa ien s showed o e 20% imp o emen in ei he gai speed o s ide leng h o bo h. One week la e , 36% o he pa ien s showed o e 20% imp o emen in baseline pe o mance wi h espec o he p e ious es . Janssen e al. [14] in es iga ed he usabili y o 3D augmen ed eali y cues compa ed o con en ional 3D ans e se ba s on he loo and audi o y cueing, in educing FoG and imp o ing gai pa ame e s. Pa ien s we e p esen ed o h ee walking cou ses (walking s aigh , s op and s a and u ning) wi h i e cue condi ions ( wo expe imen al condi ions: AR isual cues ba s, AR isual cues s ai case; and h ee 31 The e o e, a sys ema ic app oach was ollowed o iden i y he equi emen s o he sys em, om he poin o iew o he use and he echnologies, conside ing he limi a ions iden i ied in he li e a u e e iew, allowing o mo e on o he nex asks o he disse a ion. Table 2-4 - Iden i ied limi a ions o cu en VR/AR/MR-based app oaches and guidelines o hei mi iga ion Limi a ions Guidelines o be ollowed Technological Sma glasses cha ac e is ics: hea y, uncom o able, monocula and wi h a na ow ield o iew Sma glasses should be mo e ligh weigh , com o able, wi h a use - iendly design, binocula and wi h an adequa e eye calib a ion and ield o iew Explo e he use o mixed eali y Unknown moni o ing sys ems’ con ibu ion Explo e he use and po en ial o o he in eg a ed moni o ing sys ems speci ied o di e en gai impai men s Unclea co ela ion be ween i ual en i onmen s and asks and be e assessmen and aining S udy which a e he bes i ual en i onmen s and i ual asks o mo o assessmen and aining Valida ion Failu e o ca y ou usabili y, sa e y, and easibili y ques ionnai es Assessmen o he VR/AR/MR-app oach should include usabili y, sa e y and easibili y ques ionnai es and mo e objec i e es s Implemen a ion o a sui able amilia iza ion phase wi h VR/AR/MR equipmen Sho - e m in e en ions and ollow- up absence VR/AR/MR should inco po a e ea men p o ocols o se e al sessions pe week, o se e al weeks wi h longe ollow-up in e als Con ol g oup absence VR/AR/MR should in eg a e a con ol g oup o s udy he e ec s o i Con ol condi ion absence VR/AR/MR should in eg a e a con ol condi ion o dis inguish dis ac ion by he sma glasses 3 SOLUTION OVERVIEW 33 The ollowing chap e speci ies he ma e ials and me hods used o de elop he p oposed s a egy and o acqui e and p ocess all he da a equi ed. This includes i) an o e iew o he solu ion ound, s a ing by summa ising he p oblem aised; ii) a p esen a ion o he p ojec in which his disse a ion is in eg a ed; and iii) he espec i e p ojec module o which his disse a ion has con ibu ed; and i ) an in oduc ion o he de ices and sys ems used. 3.1 PROBLEM DESCRIPTION F om he li e a u e e iew i was concluded ha VR/AR/MR s a egies ha e he po en ial o no only imme se pa ien s in en i onmen s ha ec ea e daily si ua ions which may igge PD- ela ed gai disabili ies bu also o in eg a e bio eedback s a egies o help pa ien s eme ge om hese disabili ies. Howe e , some limi a ions we e iden i ied, such as he ac ha he sma glasses we e hea y, no usabili y, sa e y o easibili y ques ionnai es we e used, and he lack o con ol condi ions in he p o ocols, p e en ing a clea discussion o he esul s ob ained. Thus, a new s a egy mus include pa ien s wi h Pa kinson’s disease as a ge audience and will be implemen ed based on MR echnology, i.e., he combina ion o eal-wo ld imme sion and i ual objec s in e ac ion, in eg a ed wi h a mo ion acking sys em. Pa ien s’ mo o pe o mance will be eco ded and assessed by he mo ion acking sys em, which should p esen a eal- ime synch oniza ion wi h he MR echnology. Fu he , he analysis o use s’ mo ion will make i possible o p o ide on-demand isual cues, ollowing a isual bio eedback s a egy. In his sense, his disse a ion expec s o (i) de elop i ual en i onmen s ha lead o gai impai men s and (ii) in eg a e a bio eedback s a egy ha enables pa ien s o o e come hese episodes. The e o e, h ee di e en i ual en i onmen s we e de eloped, in which pa ien s we e imme sed and encou aged o pe o m mo o asks, ha co esponded o h ee si ua ions ha ypically cause PD gai impai men s ( u ning, c ossing doo s, na ow spaces). In addi ion, a bio eedback s a egy based on isual cues was p oposed o imp o e pa ien s’ mo o pe o mance in he same i ual en i onmen s. This solu ion is in eg a ed in he +sImme si e module o he +sense p ojec , which is p esen ed in he nex sec ion. 3.2 +SENSE This disse a ion is in eg a ed and in ended o con ibu e o he +sense p ojec . The p ojec aims o imp o e pa ien s’ quali y o li e, p omo ing less dependence on hi d pa ies by imp o ing hei mobili y 34 and mo o au onomy. In his sense, +sense o e s on -end high- ech solu ions based on wea able bio eedback de ices which ely on acquisi ion, in e p e a ion, and eedback o pa ien s’ senso imo o in o ma ion. Cu en ly, +sense is di ided in o ou modules, as shown in Figu e 3-1: (1) +sBio eedback; (2) +sMo ion; (3) +sC-suppo and (4) +sImme si e. The de elopmen o his disse a ion con ibu ed o he ou h module. 3.3 +SIMMERSIVE This module b ings a new pa adigm shi by using mixed eali y app oaches, in eg a ed wi h a mo ion acking de ice and bio eedback s a egies, as a complemen a y ool o mo o moni o ing and aining o PD- ela ed gai disabili ies. The h ee i ual en i onmen s allow he imme sion o he use in e e yday si ua ions, ha ing o pe o m daily mo o asks, in o de o achie e a mo e eliable mo o assessmen and ehabili a ion. Thus, his disse a ion b ings a s ep o wa d in he knowledge o how mixed eali y-based mo o assessmen and aining can be applied in Pa kinson's disease. The +sImme si e conside s he mul i ac o ial na u e o PD and inno a es by con ibu ing wi h a pa ien -cen ed app oach. Bea ing his in mind, wo de ices we e used: (1) mixed eali y sma glasses; and (2) mo ion acking sys em, explained in de ail in he ollowing sec ions. Figu e 3-1 - +sense modules. 35 3.3.1 MIXED REALITY SMART GLASSES: MICROSOFT HOLOLENS 2 In o de o implemen he mixed eali y s a egy, he Mic oso HoloLens 2 was used, a ully imme si e, po able, and wea able comme cial se up de ice o augmen ed/mixed eali y, Figu e 3-2. I consis s o an AR/MR headse and a USB Type-C cable, which allows o cha ge he sma glasses and connec hem o o he de ices such as compu e s. Some o he HoloLens 2 sys em speci ica ions a e men ioned in Table 3-1 [30]. HoloLens 2 p esen s see- h ough holog aphic lenses, enabling o see he eal wo ld, ne e losing he sense o eali y. Mo eo e , hey ha e se e al senso s ha allow head and eye acking, making i possible o he glasses o always know whe e he use is in space. These sma glasses ha e buil -in speake s and mic ophone. In his sense, beyond he isual eedback, hey can also p o ide audi o y eedback o use s. One o he bigges s eng hs is ha hey can unde s and he human and he en i onmen h ough hand acking, eye acking, oice, 6DoF acking and spa ial mapping, making hese glasses use - iendly. In addi ion, hey only need a USB Type-C cable o connec o a compu e , hey a e ligh weigh (556g), one can wea glasses unde hem, and hei ba e y las s up o 3 hou s o ac i e use. This as ange o ea u es o HoloLens 2 mo i a ed i s selec ion, as i was in ended o use mixed eali y sma glasses ha allow holog ams o be placed in eal space, while s ill seeing he eal wo ld. Figu e 3-2 - Mic oso HoloLens 2. 36 Table 3-1 - Speci ica ions o HoloLens 2 sma glasses [30] HoloLens 2 Technical Speci ica ions Display Op ics See- h ough holog aphic lenses (wa eguides) Resolu ion 2k 3:2 ligh engines Holog aphic densi y 2.5k adian s (ligh poin s pe adian ) Eye-based ende ing Display op imiza ion o 3D eye posi ion Senso s Head acking 4 isible ligh came as Eye acking 2 IR came as Dep h 1-MP ime-o - ligh (ToF) dep h senso IMU Accele ome e , gy oscope, magne ome e Came a 8-MP s ills, 10800p30 ideo Audio and speech Mic ophone a ay 5 channels Speake s Buil -in spa ial sound Human unde s anding Hand acking Two-handed ully a icula ed model, di ec manipula ion Eye acking Real- ime acking Voice Command and con ol on-de ice; na u al language wi h in e ne connec i i y Windows Hello En e p ise-g ade secu i y wi h i is ecogni ion En i onmen unde s anding 6DoF acking Wo ld-scale posi ional acking Spa ial Mapping Real- ime en i onmen mesh Mixed Reali y Cap u e Mixed holog am and physical en i onmen pho os and ideo Compu e and connec i i y SoC Qualcomm Snapd agon 850 Compu e Pla o m HPU Second-gene a ion cus om-buil holog aphic p ocessing uni Memo y 4-GB LPDDR4x sys em DRAM S o age 64-GB UFS 2.1 Wi-Fi Wi-Fi: Wi-Fi 5 (802.11ac 2x2) Blue oo h 5 USB USB Type-C Fi Single size Yes Fi s o e glasses Yes Weigh 566g So wa e Windows Holog aphic Ope a ing Sys em Mic oso Edge Dynamics 365 Remo e Assis Dynamics 365 Guides 3D Viewe Powe Ba e y li e 2–3 hou s o ac i e use Cha ging USB-PD o as cha ging Cooling Passi e (no ans) 37 HoloLens 2 has some ecommended sys em equi emen s (Table 3-2)[31] ha he hos compu e mus mee o p ope ly enjoy he expe ience. A compu e TUF Gaming wi h a NVIDIA GeFo ce GTX 1060 GPU was used o un and connec he so wa e needed o build he mixed eali y ool. Acco ding o Table 3-2, he compu e TUF Gaming comp ises all he minimum and ecommended equi emen s o use HoloLens 2 sys em. Table 3-2 - Compa ison o ecommended sys em equi emen s o using HoloLens 2 [31] and he speci ica ions o he used compu e (TUF Gaming FX505GM_FX505GM) Componen Recommended sys em equi emen s TUF Gaming FX505GM_FX505GM CPU 64-bi wi h 4 co es o equi alen In el® Co eTM i7-8750H CPU 2.20GHz GPU Di ec X 11.0 o la e WDDM 1.2 d i e o la e NVIDIA GeFo ce GTX 1060 RAM 8 GB o mo e 32 GB Ope a ing sys em 64-bi Windows 10 P o, En e p ise, o Educa ion (Hype -V suppo ) Windows 11 Home 22H2 3.3.2 MOTION TRACKING SYSTEM: XSENS MVN AWINDA The IMU-based mo ion cap u e sys em elies on MVN Awinda (Xsens, Enschede, The Ne he lands) [32], [33] gi en i s eliabili y o body mo ion analysis in ee-li ing condi ions. The lowe body con igu a ion (Figu e 3-3a) comp ises a o al o 7 wea able Wi eless Mo ion T acke s (MTw) senso s (Figu e 3-3b) which a e placed on he body h ough adjus able s aps (Figu e 3-3c). This sys em collec s he lowe -body kinema ic da a ha will be used o s udy he pa icipan s’ mo o pe o mance and ac acco dingly. Fu he mo e, his sys em was used o communica e wi h Uni y so wa e, p o iding in o ma ion abou he occu ence o gai ini ial o inal con ac (IC/FC) so ha he bio eedback could ac in HoloLens 2. Figu e 3-3 – Xsens MVN Awinda componen s. (a) lowe body con igu a ion; (b) MVN Mo ion T acke (MTw); (c) MVN Awinda s aps; (d) MVN Awinda s a ion. 38 The MTw senso s ha e embedded accele ome e s, gy oscopes and magne ome e s ha p o ide 3D accele a ion, 3D angula eloci y and 3D magne ic ield, espec i ely [32], [33]. These measu emen s become pa icula ly in e es ing o posi ion and o ien a ion es ima ion o human body segmen s. Thus, i was possible o de elop an algo i hm (Sec ion 4.3) o de ec ing ini ial and inal con ac s, acco ding o he da a coming om Xsens, namely he angula eloci y in y and he linea eloci y in z o he oo senso s. Da a om he MTw senso s a e wi elessly ansmi ed and synch onised by he Awinda S a ion (Figu e 3-3d). Du ing he da a acquisi ion sessions, i was used he MVN Analyze P o 2021.2, an easy- o-use so wa e o eal- ime iewing and eco ding, which allows he expo o mo ion cap u e da a o hi d pa y applica ions [32]. Fu he mo e, his so wa e has a s eaming ea u e which enables compu e s o s eam he cap u ed da a o e a ne wo k o o he clien compu e , in eal- ime, Figu e 3-4. This eal- ime ne wo k s eaming p o ocol is based on Use Da ag am P o ocol (UDP). The UDP P o ocol is unidi ec ional, is s a eless and does no equi e he ecei e o answe incoming packe s, which allows g ea e speed. Upon his, Xsens has de eloped plug-ins, a ailable o Uni y3D, o ee a asse s o e, o usage wi h hi d pa y ools as a clien applica ion, allowing o ecei e mo ion cap u e da a in eal- ime. The da a con en in he da ag am is de ined by he speci ic p o ocol se . Each da ag am s a s wi h a 24- by e heade ollowed by a a iable numbe o by es o each body segmen , depending on he selec ed da a p o ocol. All da a is sen in ‘ne wo k by e o de ’, which co esponds o big-endian no a ion. The heade con ains he ype o he da a and some iden i ica ion in o ma ion, so he ecei ing end can apply i o he igh a ge [34]. Thus, a new session was c ea ed o each “equipped” olun ee and an h opome ic da a was measu ed and egis e ed o build he pe son’s biomechanical model. A e , a calib a ion me hod is pe o med o align he MTw senso s wi h he use ’s body segmen s by he “Npose + Walk” ask. When a success ul calib a ion is achie ed and he “s eam” ea u e is on, as well as he "Linea Segmen Kinema ics", “Angula Segmen Kinema ics” and “Time code” da ag ams selec ed, i is inally possible o s a eco ding a eal- ime session acco ding o he de ined p o ocol. 39 3.4 CONCLUSIONS A e an ex ensi e li e a u e e iew abou he cu en ly VR/AR/MR-based app oaches used in PD, i was no iced ha mixed eali y may be he bes echnology o be used wi h indi iduals wi h Pa kinson's disease, as i allows i ual and in e ac i e objec s o be added o he eal wo ld, wi hou e e losing he sense o eali y. Thus, his disse a ion aims o explo e his echnology no only o he assessmen bu also o he aining o PD- ela ed disabili ies. To his end, his disse a ion is inse ed in he +sense p ojec , con ibu ing o he +sImme si e p ojec module and makes use o wo high- ech equipmen , namely he HoloLens 2 mixed eali y sma glasses and he Xsens mo ion acking sys em. Figu e 3-4 – Real- ime s eaming ea u e in MVN Analyze P o. 4 SOLUTION DESCRIPTION 47 The da a acquisi ion p o ocol consis ed o pe o ming wo di e en ials, h ee imes each, which consis ed o walking in a s aigh line along 10 me e s: (1) wi h he sma glasses OFF; and (2) wi h he sma glasses ON, showing a i ual scena io (scena io 3, na ow spaces). In o al each pa ien pe o med 6 ials. A e he expe imen al p o ocol was comple ed, he acqui ed da a was analysed in o line. To do his, he ials we e expo ed in MVN Analyze P o and hen he expo ed ials (m nx iles) we e loaded in o MATLAB. A e wa ds, he eal- ime IC/FC de ec ions we e compa ed wi h he Xsens oo con ac signals (g ound u h), as depic ed in Figu e 4-2. Thus, (1) The angula eloci y signal in he y-di ec ion om bo h ee was compa ed wi h he espec i e oo con ac signal o assess he pe o mance o eal- ime iden i ica ion o IC; (2) The eloci y signal in he z-di ec ion om bo h ee was compa ed wi h he espec i e oo con ac signal o assess he pe o mance o eal- ime iden i ica ion o FC. De ec ed gai e en s we e e alua ed conside ing hei accu acy (Equa ion 4-1), p ecision (Equa ion 4-2), sensi i i y (Equa ion 4-3), and speci ici y (Equa ion 4-4). These me ics po ay he pe o mance o he de eloped algo i hm. T ue posi i es (TP) co esponded o he gai e en s co ec ly iden i ied, ue nega i es (TN) ep esen ed gai e en s ha he algo i hm co ec ly de ec ed as a non-e en , alse posi i es (FP) co esponded o gai e en s no co ec ly iden i ied and alse nega i es (FN) he e en s ha should had been de ec ed. Fu he mo e, ad ance and delayed de ec ions we e also assessed based on hei pe cen age o occu ence and du a ion. Ad ance and delayed de ec ions we e conside ed om he TP de ec ions. Xsens HoloLens 2 Figu e 4-4 –Rep esen a ion o he de ices used in he e i ica ion es s. 48 Equa ion 4-1 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 (%)=𝑇𝑃+𝑇𝑁 𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁 Equa ion 4-2 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 (%)= 𝑇𝑃 𝑇𝑃+𝐹𝑃 Equa ion 4-3 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 (%)= 𝑇𝑃 𝑇𝑃+𝐹𝑁 Equa ion 4-4 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 (%)= 𝑇𝑁 𝑇𝑁+𝐹𝑃 4.3.2.2 RESULTS AND DISCUSSION Table 4-4 p esen s he pe o mance o he eal- ime IC and FC de ec ion algo i hm. I shows he accu acy, p ecision, sensi i i y, speci ici y and, ad ance and delayed de ec ions (by means o hei pe cen age o occu ence and du a ion). Table 4-4 – Resul s o he e i ica ion es s Me ic Mean (±SD) Accu acy (%) 98.93 (± 1.38) P ecision (%) 100.00 (± 0.00) Sensi i i y (%) 97.87 (± 2.75) Speci ici y (%) 100.00 (± 0.00) Delays ( eq %)) 0.27 (± 0.52) Delays ( ime (s)) 0.01(± 0.02) Ad ances ( eq %)) 0.19 (± 0.37) Ad ances ( ime (s)) 0.02 (± 0.03) The p oposed algo i hm showed o be signi ican ly accu a e (mean o 98.93%), sensi i e (mean o 97.87%), p ecise (100%), and speci ic (100%) o he es s pe o med, meaning ha he de eloped algo i hm is able o de ec , wi hou much e o , he ini ial and inal con ac s, p esen ing su icien capaci y o in eg a e he bio eedback s a egy, ha ing eached he KPI3, which de ined 96% as he pe cen age o accu acy o be me , ha is, in a space o 10 me e s whe e 20 s eps a e aken, he algo i hm de ec s 19 IC/FCs. Howe e , an adjus men o he h esholds was subsequen ly made using exis ing da a om he 49 p ojec da abase o 9 PD pa ien s. Ne e heless, u he modi ica ions may ha e o be made in he u u e conside ing he he e ogenei y o PD and he in a- and in e -subjec a iabili y. 4.3.2.3 CONCLUSIONS The p oposed eal- ime IC and FC de ec ion algo i hm has shown o be accu a e, sensi i e, p ecise, and speci ic. The adap abili y in oduced in he IC and FC de ec ion ensu es g ea e obus ness o he sys em in he e en ual occu ence o pe u ba ions. These aspec s make his algo i hm sui able o be in eg a ed wi h an ac ua ion sys em, i.e., wi h a bio eedback s a egy. Howe e , he e a e some u u e challenges such as he need o alida e his algo i hm wi h (1) da a collec ed om PD pa ien s; (2) da a collec ed om PD pa ien s a a ious s ages o he disease; and (3) da a collec ed o e ime. 4.3.3 SPATIOTEMPORAL METRICS ESTIMATION In o de o assess whe he he imme si e i ual en i onmen s we e able o igge PD- ela ed gai disabili ies and whe he he isual bio eedback was able o help pa ien s o e come hese impai men s, spa io empo al me ics we e es ima ed using he mo ion da a cap u ed by Xsens. Thus, a code was de eloped in MATLAB o his es ima ion. Table 4-5 p esen s he calcula ed spa io empo al pa ame e s, as well as he de ini ion and o mula o each and he uni s o measu emen . Fu he mo e, he a iabili y (SD) and asymme y (AS) o hese me ics we e also calcula ed. 50 Table 4-5 - Spa io empo al pa ame e s: desc ip ion, o mula and uni s [39] Spa io empo al pa ame e De ini ion Fo mula Measu ed uni s S ep du a ion Time be ween he con ac o wo consecu i e limbs in g ound 𝐼𝐶𝑖+1 − 𝐼𝐶𝑖 Seconds S ide du a ion Du a ion o one gai cycle, i.e., he in e al be ween wo sequen ial ini ial con ac s on he g ound by he same limb 𝐼𝐶𝑖+2 − 𝐼𝐶𝑖 Seconds S ance phase du a ion Du a ion o s ance phase o a io o s ance phase du a ion wi h s ide du a ion (𝐹𝐶𝑖+1 − 𝐼𝐶𝑖)×100 𝑆𝑡𝑟𝑖𝑑𝑒 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 Seconds o pe cen age Swing phase du a ion Du a ion o swing phase o a io o swing phase ime wi h s ide du a ion (𝑆𝑡𝑟𝑖𝑑𝑒𝑇𝑖𝑚𝑒𝑖+1 −𝑆𝑡𝑎𝑛𝑐𝑒𝑇𝑖𝑚𝑒𝑖)×100 𝑆𝑡𝑟𝑖𝑑𝑒 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 Seconds o pe cen age Double suppo phase du a ion In e al o ime o he double suppo phase o a io o double suppo phase du a ion wi h s ide du a ion (𝐼𝐶𝑖+1 −𝐹𝐶𝑖)×100 𝑆𝑡𝑟𝑖𝑑𝑒 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 Seconds o pe cen age S ep leng h Dis ance ha one pa o he oo mo es in on o he same pa o he o he oo du ing each s ep 2√2𝐿ℎ−ℎ2 ,ℎ= ∬ 𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑣𝑒𝑟𝑡𝑖𝑐𝑎𝑙 𝐼𝐶𝑖+1 𝐼𝐶𝑖 Me e s S ide leng h Dis ance be ween wo consecu i e ini ial con ac s on he g ound by he same limb 𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖+ 𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖+1 Me e s Veloci y Dis ance co e ed by he whole body in a gi en ime 𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖 𝑠𝑡𝑒𝑝 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛𝑖 Me e s pe second Cadence Numbe o s eps aken in a speci ic ime 𝑣𝑒𝑙𝑜𝑐𝑖𝑡𝑦𝑖×60 𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖 S eps pe minu e ROM Range o he signals 𝑚𝑎𝑥 (𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧) −min(𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧) Me e s pe second squa ed RMS Rela es o he ib a ion le els o a signal 𝑟𝑚𝑠 (𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧) Me e s pe second squa ed JERK Fi s ime de i a i e o accele a ion 𝑑𝑖𝑓𝑓 (𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧) Me e s pe second cubed ROM: ange o mo ion; RMS: Roo mean squa e. 51 Figu e 4-5 is a ep esen a ion o he gai cycle o a heal hy subjec , o easie in e p e a ion o he concep s. When he pa icipan is exposed o i ual en i onmen s in ended o assess mo o pe o mance, empo al me ics, such as s ep du a ion, a e expec ed o inc ease and spa ial me ics, including s ep leng h and eloci y, o dec ease, as hese pa ien s end o p esen a mo e cau ious beha iou in pe o ming hese asks, leading o slowe and smalle s eps [2], [12], [20] Con e sely, when bio eedback is used, empo al me ics a e expec ed o dec ease, and spa ial me ics a e expec ed o inc ease, i.e., mo e s able gai pa e n, as e , and bigge s eps [1], [13]–[15], [17]. 4.4 BIOFEEDBACK INTEGRATION Rega ding bio eedback s a egies, wo modali ies we e de eloped, one in open loop and he o he in closed loop, which a e explained below. 4.4.1 OPEN LOOP STRATEGY The bio eedback s a egy o i ual scena io 1, co ido wi h dices, consis ed o using isual cues ha indica ed he pa h o be aken by he hand wi h he dice, om he ini ial loca ion o he dice o he co esponding colou ed box. To his end, a ows we e d awn in he ai , in he colou o he co esponding Figu e 4-5 - Gai cycle o a heal hy subjec . 52 dice, o help he use o u n. A ows we e chosen because hey ep esen di ec ion and mo emen , ying o acili a e he u ning o he body. Each se o a ows displayed he colou ela i e o he playing dice so ha he use would no be con used abou which dice is ca ying. This s a egy is in ended o make u ning a mo e luid and easie ask, by p o iding isuospa ial cueing in o ma ion. Figu e 4-6 p esen s he s a egy men ioned. 4.4.2 CLOSED LOOP STRATEGY The closed-loop bio eedback s a egy was achie ed by he communica ion p o ocol be ween MVN Analyze P o and Uni y, using he eal- ime IC/FC de ec ion algo i hm (Sec ion 4.3.1). This way, he s a egy o i ual scena ios 2 ( i ual doo ) and 3 (na ow spaces) consis ed o p esen ing isual cues on he loo in on o he use in a closed loop. In his case, i was chosen o use oo p in s since hey show ele ance in he gai . The g een was chosen since he eye is mos sensi i e o a yellowish-g een colou unde no mal ligh ing condi ions [35], [36] and, beyond ha , g een is ela ed o “being igh ”, “p oceeding”, unlike de colou ed, as in a ic ligh s. Thus, when he use places he igh oo on he loo , a igh IC o heel s ike is de ec ed, and he sys em will place a le g een oo p in o se e as a spa ial guideline, indica ing whe e o place he le oo . On he con a y, once he use places he le oo on he loo , a le IC o heel s ike is de ec ed, and he igh g een oo p in is displayed. The e o e, a mo e luid and con inuous gai is mo i a ed. Figu e 4-7 shows a g een oo p in o he igh oo a e de ec ion o a heel s ike om he le oo . Figu e 4-6 - Bio eedback s a egy o scena io 1. 53 4.5 CONCLUSIONS This chap e has p esen ed he me hods used o implemen a modula , use -cus omised, mixed eali y-based echnology solu ion ha (1) imme ses pa ien s in en i onmen s ha cause PD- ela ed gai impai men s; and (2) imme ses pa ien s in en i onmen s ha help o e come hese impai men s wi h he aid o HoloLens 2, Xsens and bio eedback s a egies (RQ2). The i ual en i onmen s and asks we e de ined, ha ing de eloped h ee di e en en i onmen s ha aimed o ep esen he eal-li e si ua ions ha mos cause PD-gai disabili ies in PD pa ien s, namely (1) u ning (scena io 1); (2) walking h ough doo s (scena io 2); and (3) walking in na ow spaces (scena io 3). Rega ding bio eedback s a egies, a ows and oo p in s we e added o he i ual en i onmen s, and i would be expec ed ha he use would ollow hese isual cues. The oo p in s we e p o ided in closed loop, i.e. as he use walks, mo e oo p in s will appea which a e ac i a ed by he occu ence o ICs. Thus, when he use pu s one oo on he g ound, he HoloLens p ojec s he oo p in ela i e o he opposi e oo in o de o p omo e a mo e con inuous and luid gai . To make his possible, an algo i hm was de eloped o de ec ICs and FCs in eal ime, based on adap i e h esholds. Fu he mo e, a MATLAB code was de eloped o he es ima ion o spa io empo al me ics in o line, so ha i was possible o e alua e he mo o pe o mance a e exposu e o he i ual en i onmen s and a e exposu e o he bio eedback s a egies in hese pa ien s. Figu e 4-7 – Display o he igh g een oo p in . 5 SOLUTION VALIDATION 55 This chap e desc ibes he me hodologies o alida ing he solu ion. Fi s ly, he p o ocol used in he alida ion o he mixed eali y s a egies wi h indi iduals wi h PD is p esen ed. This alida ion p o ocol ollowed a p e-pos expe imen al s udy design aiming o e alua e he subjec s’ mo o pe o mance. The pa icipan s and hei cha ac e is ics a e speci ied, as well as he inclusion and exclusion c i e ia. Nex , he ma e ials used in he in e en ion a e desc ibed, ollowed by he da a acquisi ion me hods and he s udy a iables. The da a p ocessing conduc ed o achie e he in ended ou comes measu es is exposed, as well as he s a is ical analysis pe o med. Finally, he esul s ob ained a e p esen ed, as well as a de ailed discussion o hem, answe ing RQ3. 5.1 INTRODUCTION 5.1.1 HYPOTHESIS, RESEARCH QUESTION AND STUDY DESIGN PD is cu en ly incu able, so i s ea men consis s o applying in e en ions ha slow down he apid p og ession o he disease. The mixed eali y s a egies de eloped in his disse a ion aim o b ing he day- o-day eali y o he pa ien close o he medical appoin men . In ac , i becomes c i ical ha hese pa ien s a e co ec ly and objec i ely assessed, since disease p og ession is e y as . In addi ion, i is in ended o e i y whe he his s a egy could complemen ehabili a ion sessions, h ough a mo e un and disease- ocused aining, using isual cues, wi h he aim o imp o ing hei mobili y and au onomy. In ha sense, he esea ch ques ion o be answe ed is RQ3: “How does he implemen ed modula echnological solu ion, based on mixed eali y in eg a ed wi h a mo ion acking sys em and wi h bio eedback s a egies, a ec he mo o pe o mance o PD pa ien s du ing assessmen and aining?” The s udy in ques ion consis ed o a c oss-sec ional s udy as an obse a ion o a de ined popula ion was conduc ed a a single poin in ime. Exposu e o he in e en ion and ou come we e de e mined simul aneously [40]. 5.2 METHODOLOGY The alida ion p o ocol wi h pa hological end-use s was conduc ed in Hospi al o B aga, wi h he collabo a ion o he physicians om 2CA-B aga, ollowing he p inciples o he Decla a ion o Helsinki and he O iedo Con en ion, in acco dance wi h he e hical guidelines o he E hics Commi ee in Li e and Heal h Sciences (CEICVS 147/2021). All pa icipan s illed ou an in o med consen o pa icipa e in he cu en esea ch. 56 5.2.1 PARTICIPANTS Ele en subjec s (six emales and i e males) we e ec ui ed and accep ed o pa icipa e in his da a collec ion. A lis o inclusion and exclusion c i e ia was ou lined in o de o selec he pa icipan s. Pa icipan s we e ec ui ed i hey had: I) diagnosis o PD acco ding o he UK Pa kinson’s Disease Socie y B ain bank c i e ia; II) p esence o eezing o gai ; III) Hoehn and Yah s age be ween 1 and 4; IV) age be ween 45 and 85 yea s old; and V) able o walk wi hou assis ance. Exclusion c i e ia we e: I) p esence o como bid diso de s likely o a ec gai , including s oke, o hopaedic disease, heuma ologic disease, o he neu ological and musculoskele al diso de s, ca dio ascula and pulmona y diseases; II) signi ica i e cogni i e impai men (MMSE<24); III) ob ious mo o impai men s; IV) isual acui y de ici s; V) audiome ic de ici s; VI) pain ha may a ec walking; and VII) inabili y o pe o m a 180º u n wi hou assis ance. Table 5-1 p esen s he pa icipan s’ de ailed clinical cha ac e is ics and an h opome ics. Table 5-1 - Demog aphic in o ma ion abou he PD pa icipan s Pa icipan ID Gende (M/F) Age (yea s) Body heigh (cm) Body mass (kg) Clinical S a e NFoG-Q UPDRS, Pa III H&Y PD- 01 F 74 160 62 ON 27 66 4 PD- 02 F 48 166 65 ON 24 4 PD- 03 F 67 164 73 ON 24 71 3 PD 04 F 59 155 71 ON 0 18 2 PD 05 M 75 168 80 ON 0 12 2 PD- 06 M 71 163 85 ON 24 19 2 PD- 07 M 65 172 60 ON 25 15 1 PD- 08 F 70 163 77 ON 13 22 2 PD- 09 F 47 169 64 ON 12 14 1 PD- 10 M 83 160 75 ON 26 57 4 PD- 11 M 84 162 62 ON 26 42 3 Mean (±STD) - 67.55 (± 11.71) 163.82 (± 4.55) 70.36 (± 7.96) - 18.28 (± 9.88) 31.86 (±22.03) 2.29 (±1.08) ID: iden i ica ion; PD: indi idual wi h Pa kinson's disease; PD- : indi idual wi h Pa kinson's disease and eezing o gai ; M: male; F: emale; NFoG-Q: New eezing o gai ques ionnai e; UPDRS-III: Uni ied Pa kinson’s disease a ing scale – pa III; H&Y: Hoehn and Yah scale; STD: s anda d de ia ion. 5.2.2 MATERIALS The ma e ials used in he alida ion phase we e he HoloLens 2 mixed eali y sma glasses and he Xsens mo ion acking sys em. In addi ion, a documen eco ding he pa icipan s' demog aphic and clinical in o ma ion was also used. HoloLens 2 was used o display he i ual en i onmen s and o p o ide he isual cues (a ows and g een oo p in s). In u n, Xsens was used o s eam ine ial da a o Uni y a 63 5.2.6 STATISTICAL ANALYSIS The s a is ical analysis was pe o med h ough IBM SPSS so wa e e sion 25.0 ( o Windows) (IMP Co p, A monk, NY, USA). Fi s ly, desc ip i e s a is ics we e ob ained o summa ise he esul s (means and s anda d de ia ions) o each g oup and he da a no mali y was assessed using Shapi o- Wilk es . The popula ion was conside ed no mally dis ibu ed i he signi icance alue was highe han 0.05. This s udy p esen s pai ed samples as he samples a e om he same pa icipan s, Table 5-8. In his sense, o compa e wo pai ed g oups, pai ed es was pe o med o he pa ame ic me ics and he Wilcoxon signed- ank es was applied o a iables whe e he assump ion o no mali y was no e i ied. When mo e han wo pai ed g oups we e o be compa ed, epea ed-measu es ANOVA was used o no mal popula ions, on he con a y, he F iedman es was used o non-no mal a iables. All s a is ical es s we e execu ed conside ing a con idence le el o 95% (α = 0.05). The s a is ical es s we e conduc ed o e alua e he ollowing null hypo hesis (H0): “ he e a e s a is ically signi ican di e ences be ween in e en ions”. I p - alue<0.05, he H0 is accep ed. Table 5-8 - Pa ame ic and non-pa ame ic es s Type o da a Goal Measu emen (o no mal popula ions) O de , esul o measu e (o non-no mal popula ions) Compa e wo pa ed samples Pai ed es Wilcoxon es Compa e mo e han wo pa ed samples Repea ed-measu es ANOVA F iedman es 5.3 RESULTS This subchap e aims o p esen he esul s om he alida ion p o ocol wi h PD pa ien s. The esul s a e di ided in o h ee sec ions. Fi s ly, he esul s conce ning he mo o assessmen a e p esen ed, ollowed by he esul s ela ed o he mo o aining. In addi ion, he esul s o eezing gai episodes (numbe and du a ion) pe es a e shown. Finally, he answe s o he use ’s expe ience e alua ion es s a e depic ed. Ele en pa ien s unde wen he es s, wi h en pa ien s comple ing all es s. Pa ien 10 d opped ou due o a igue/no in e es . Mos pa icipan s success ully comple ed all he es s p o ided, in an a e age du a ion o 40 minu es. 64 5.3.1 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR ASSESSMENT Fi s ly, he aim was o check whe he wea ing he HoloLens 2 OFF would ha e any in luence on he mo o unc ion o hese pa ien s. To his end, he TC1 and TC2 es s we e compa ed. Rega dless o he ou come o he compa ison o hese es s, he TC1 was selec ed o be compa ed o he o he es s o he pu pose o esul s analysis. Nex , i was aimed o s udy he po en ial o mixed eali y o cause PD-gai disabili ies by compa ing he TC1 wi h he M2 and, la e , he TC1 wi h he M3. Thus, he i s s ep o he s a is ical analysis was o pe o m a desc ip i e analysis by es . The esul s o his s ep a e shown in Table 5-10, o con ol es s 1 and 2 (TC1, TC2) and moni o ing es s 2 and 3 (M2 and M3). A e wa ds, he no mali y o he ea u es was e i ied using he Shapi o-Wilk es . As mos o he ea u es did no show a no mal dis ibu ion, he Wilcoxon es was chosen o compa ison o wo pai ed samples. The esul s o his es a e also shown in Table 5-10, in which he di e ence o a ce ain ea u e be ween he se e al es s is conside ed signi ican i he signi icance alue is smalle han 0.05 and hese a e in bold. Rega ding con ol es s TC1 and TC2, only a ew me ics, namely, s ep and s ide leng h, eloci y and AS swing ime, p esen ed s a is ically signi ican di e ences, since he p- alue < 0.05, co obo a ing he null hypo hesis. A e wa ds, he compa ison be ween TC1 and M2 showed s a is ically signi ican di e ences in he me ics s ep and s ide du a ion, s ance, swing and double suppo phase, eloci y, SD s ep and s ide du a ion, SD s ance and swing phase, SD s ep leng h, SD cadence and AS s ance and swing ime, co obo a ing he null hypo hesis. Finally, he compa ison be ween TC1 and M3 showed s a is ically signi ican di e ences in almos all me ics, so he null hypo hesis is co obo a ed. All hese me ics a e in bold. Th oughou da a acquisi ion, he esea che s isually assessed he exis ence o gai eezing episodes. La e , du ing da a p ocessing, hese episodes we e excluded and hei numbe , a e age and o al du a ion we e coun ed pe es . The esul s a e p esen ed in Table 5-9. Figu e 5-5 p esen s wo QR codes showing ideos o pa ien 7 pe o ming es s M2 and M3. 65 Table 5-9 - Numbe , a e age and o al du a ion os eezing o gai episodes in con ol es s and moni o ing es s 2 and 3 Tes Numbe o episodes A e age du a ion (s) To al du a ion (s) TC1 0 0 0 TC2 0 0 0 M2 4 6.73 74 M3 0 0 0 Figu e 5-5 – Videos o pa icipan 7 pe o ming he M2 and M3 es s. 66 Table 5-10 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o con ol es s and scena ios 2 and 3 TC1 TC2 Wilcoxon Tes (sig) (TC1-TC2) M2 Wilcoxon Tes (sig) (TC1-M2) M3 Wilcoxon Tes (sig) (TC1-M3) Me ic Mean S d de ia ion Shapi o- Wilk (sig) Mean S d de ia ion Shapi o- Wilk (sig) Mean S d de ia ion Shapi o- Wilk (sig) Mean S d de ia ion Shapi o- Wilk (sig) S ep du a ion 0.618 0.087 0.298 0.618 0.082 0.144 0.811 0.740 0.233 0.254 0.022 0.812 0.360 0.001 0.003 S ide du a ion 1.236 0.176 0.345 1.235 0.165 0.122 0.868 1.478 0.472 0.218 0.025 1.615 0.722 0.001 0.005 S ance phase 62.057 3.047 0.168 62.688 3.295 0.081 0.053 67.056 5.887 0.383 0.000 67.360 7.848 0.002 0.000 Swing phase 37.943 3.047 0.168 37.312 3.295 0.081 0.053 32.914 5.937 0.361 0.000 32.640 7.848 0.002 0.000 Double suppo phase 24.196 6.104 0.170 25.508 6.550 0.102 0.058 34.562 12.255 0.247 0.000 34.442 16.290 0.001 0.000 S ep leng h 0.568 0.096 0.378 0.519 0.135 0.022 0.004 0.509 0.165 0.075 0.053 0.500 0.127 0.613 0.003 S ide leng h 1.143 0.199 0.265 1.046 0.257 0.033 0.005 1.019 0.344 0.108 0.053 0.994 0.258 0.596 0.004 Veloci y 0.934 0.198 0.226 0.853 0.218 0.047 0.012 0.750 0.275 0.128 0.002 0.702 0.215 0.273 0.000 Cadence 99.395 13.881 0.149 99.988 13.399 0.035 0.744 95.866 27.003 0.399 0.396 89.104 21.928 0.042 0.006 SD s ep ime 0.040 0.022 0.000 0.055 0.059 0.000 0.616 0.222 0.274 0.000 0.001 0.240 0.370 0.000 0.002 SD s ide ime 0.055 0.037 0.000 0.076 0.090 0.000 0.828 0.307 0.370 0.000 0.002 0.317 0.447 0.000 0.003 SD s ance ime 0.045 0.026 0.000 0.068 0.092 0.000 0.616 0.280 0.344 0.000 0.001 0.259 0.396 0.000 0.002 SD swing ime 0.031 0.022 0.000 0.037 0.033 0.000 0.557 0.101 0.101 0.000 0.005 0.147 0.202 0.000 0.003 SD s ep leng h 0.139 0.067 0.049 0.119 0.052 0.043 0.184 0.180 0.102 0.043 0.039 0.159 0.105 0.001 0.420 SD s ide leng h 0.182 0.121 0.002 0.134 0.089 0.000 0.145 0.252 0.166 0.072 0.133 0.211 0.171 0.002 0.528 SD eloci y 0.229 0.119 0.008 0.191 0.084 0.087 0.170 0.247 0.126 0.259 0.396 0.204 0.125 0.004 0.231 SD cadence 6.170 2.530 0.002 7.453 5.207 0.001 0.500 19.854 19.948 0.000 0.004 13.895 15.022 0.000 0.039 AS s ep ime 0.034 0.023 0.092 0.032 0.022 0.299 0.695 0.114 0.179 0.000 0.231 0.136 0.173 0.000 0.017 AS s ide ime 0.009 0.011 0.000 0.005 0.006 0.001 0.316 0.026 0.051 0.000 0.446 0.033 0.068 0.000 0.758 AS s ance ime 0.018 0.016 0.038 0.027 0.030 0.002 0.085 0.048 0.057 0.000 0.031 0.069 0.119 0.000 0.078 AS swing ime 0.011 0.016 0.000 0.031 0.033 0.000 0.004 0.047 0.062 0.000 0.011 0.094 0.140 0.000 0.008 AS s ep leng h 0.112 0.079 0.053 0.121 0.092 0.044 0.777 0.102 0.071 0.080 0.845 0.109 0.077 0.058 0.983 AS s ide leng h 0.043 0.043 0.002 0.026 0.031 0.001 0.085 0.038 0.040 0.003 0.586 0.039 0.043 0.002 0.472 AS eloci y 0.176 0.130 0.023 0.184 0.132 0.111 0.349 0.159 0.090 0.103 0.913 0.173 0.104 0.158 0.616 AS cadence 5.546 3.910 0.241 5.768 3.701 0.176 0.557 7.072 8.293 0.000 0.845 9.726 7.091 0.006 0.071 67 5.3.2 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR TRAINING This subchap e p esen s he esul s o he bio eedback con ibu ion o mo o pe o mance, i.e., es s M1 wi h T1 (dice), M2 wi h T2 (doo ) and M3 wi h T3 (na ow spaces) we e compa ed. Thus, a desc ip i e analysis pe es was i s pe o med and hen he no mali y o he ea u es was s udied, using Shapi o-Wilk es . As mos o he me ics did no p esen a no mal dis ibu ion, he Wilcoxon es was used o compa ison o wo pai ed samples. The esul s o hese es s a e shown in Table 5-12, Table 5-13 and Table 5-14, in which he di e ence o a ce ain ea u e be ween he se e al es s is conside ed signi ican i he signi icance alue is smalle han 0.05. Rega ding scena io 1, no me ics showed s a is ically signi ican di e ences, ejec ing he null hypo hesis. In u n, scena io 2 showed ha only he me ics s ep du a ion, s ide du a ion, eloci y, cadence, and SD eloci y, p esen ed s a is ically signi ican di e ences, since he p- alue < 0.05, co obo a ing he null hypo hesis. Finally, scena io 3 had wo spa io empo al me ics ha showed s a is ically signi ican di e ences, namely s ep leng h and cadence. Thus, he null hypo hesis is co obo a ed. All hese me ics a e in bold. Th oughou da a acquisi ion, he esea che s isually assessed he exis ence o gai eezing episodes. La e , du ing da a p ocessing, hese episodes we e excluded and hei numbe , a e age and o al du a ion we e coun ed pe es . The esul s a e p esen ed in Table 5-11. Figu e 5-6 p esen s six QR codes showing ideos o pa ien 7 pe o ming es s M1, T1, M2, T2, M3 and T3. Table 5-11 - Numbe , a e age and o al du a ion os eezing o gai episodes in moni o ing es s 1, 2, and 3 and aining es s 1, 2 and 3 Tes Numbe o episodes A e age du a ion (s) To al du a ion (s) M1 0 0 0 T1 0 0 0 M2 4 6.73 74 T2 6 9.78 107.6 M3 0 0 0 T3 4 3.13 34.4 68 Table 5-12 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o scena io 1 M1 T1 Wilcoxon Tes (sig) Me ic Mean S d de ia ion Shapi o-Wilk (sig) Mean S d de ia ion Shapi o-Wilk (sig) ROM X (+) 3.630 1.427 0.049 3.472 1.269 0.002 0.390 ROM Y (+) 3.616 1.279 0.050 3.241 0.851 0.266 0.372 ROM Z (+) 4.720 3.046 0.000 3.985 3.422 0.000 0.168 RMS X (+) 0.478 0.217 0.003 0.508 0.214 0.018 0.178 RMS Y (+) 0.480 0.180 0.195 0.491 0.149 0.279 0.615 RMS Z (+) 0.380 0.157 0.092 0.383 0.222 0.001 0.833 JERK X (-) 0.002 0.006 0.007 0.001 0.005 0.005 0.158 JERK Y (-) 0.002 0.004 0.013 0.000 0.003 0.502 0.123 JERK Z (-) 0.000 0.004 0.054 0.001 .003 0.041 0.661 Table 5-13 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o scena io 2 M2 T2 Wilcoxon Tes (sig) Me ic Mean S d de ia ion Shapi o-Wilk (sig) Mean S d de ia ion Shapi o-Wilk (sig) S ep du a ion 0.758 0.228 0.274 0.848 0.298 0.000 0.031 S ide du a ion 1.513 0.463 0.230 1.696 0.606 0.000 0.028 S ance phase 67.517 5.724 0.536 69.522 5.243 0.393 0.064 Swing phase 32.451 5.775 0.505 30.478 5.243 0.393 0.071 Double suppo phase 35.519 11.918 0.252 38.728 10.415 0.524 0.170 S ep leng h 0.518 0.165 0.093 0.499 0.189 0.135 0.948 S ide leng h 1.038 0.345 0.135 0.991 0.408 0.045 0.777 Veloci y 0.747 0.283 0.118 0.630 0.268 0.008 0.016 Cadence 93.068 25.000 0.319 79.108 15.118 0.282 0.002 SD s ep ime 0.227 0.282 0.000 0.209 0.257 0.000 0.446 SD s ide ime 0.315 0.380 0.000 0.285 0.278 0.000 0.327 SD s ance ime 0.292 0.351 0.000 0.261 0.255 0.000 0.332 SD swing ime 0.098 0.103 0.000 0.118 0.146 0.000 0.231 SD s ep leng h 0.182 0.104 0.070 0.168 0.092 0.006 0.647 SD s ide leng h 0.254 0.171 0.087 0.210 0.135 0.004 0.586 SD eloci y 0.246 0.130 0.181 0.197 0.127 0.000 0.028 SD cadence 18.870 20.107 0.000 13.568 7.317 0.003 0.557 AS s ep ime 0.118 0.183 0.000 0.090 0.075 0.044 0.616 AS s ide ime 0.027 0.053 0.000 0.025 0.029 0.002 0.845 Figu e 5-6 - Videos o pa icipan 7 pe o ming es s M1, T1, M2, T2, M3 and T3. 69 AS s ance ime 0.048 0.058 0.000 0.064 0.102 0.000 0.983 AS swing ime 0.047 0.063 0.000 0.060 0.119 0.000 0.286 AS s ep leng h 0.104 0.072 0.145 0.134 0.100 0.133 0.184 AS s ide leng h 0.040 0.040 0.005 0.039 0.044 0.000 0.647 AS eloci y 0.158 0.092 0.077 0.159 0.113 0.241 0.586 AS cadence 7.044 8.547 0.000 7.023 4.985 0.032 0.913 Table 5-14 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o scena io 3 M3 T3 Wilcoxon es (sig) Me ic Mean S d de ia ion Shapi o-Wilk (sig) Mean S d de ia ion Shapi o-Wilk (sig) S ep du a ion (-) 0.762 0.229 0.022 0.821 0.398 0.000 0.147 S ide du a ion (-) 1.512 0.449 0.023 1.627 0.746 0.000 0.147 S ance phase 67.644 7.198 0.001 66.531 3.850 0.223 0.520 Swing phase 32.356 7.198 0.001 33.469 3.850 0.223 0.520 Double suppo phase 35.295 15.152 0.001 33.065 8.002 0.232 0.314 S ep leng h (+) 0.483 0.107 0.418 0.544 0.128 0.229 0.044 S ide leng h (+) 0.960 0.217 0.539 1.071 0.263 0.146 0.070 Veloci y (+) 0.693 0.191 0.652 0.744 0.263 0.686 0.841 Cadence 89.330 16.968 0.027 83.140 19.546 0.290 0.024 SD s ep ime 0.192 0.303 0.000 0.161 0.236 0.000 0.811 SD s ide ime 0.277 0.393 0.000 0.199 0.239 0.000 0.809 SD s ance ime 0.220 0.343 0.000 0.203 0.324 0.000 0.936 SD swing ime 0.122 0.164 0.000 0.123 0.215 0.000 0.841 SD s ep leng h 0.144 0.098 0.000 0.162 0.106 0.001 0.421 SD s ide leng h 0.184 0.163 0.000 0.212 0.178 0.000 0.314 SD eloci y 0.190 0.108 0.003 0.209 0.135 0.001 0.904 SD cadence 13.360 15.084 0.000 9.254 4.855 0.087 0.445 AS s ep ime 0.124 0.174 0.000 0.136 0.294 0.000 0.421 AS s ide ime 0.030 0.070 0.000 0.037 0.103 0.000 0.557 AS s ance ime 0.047 0.078 0.000 0.142 0.403 0.000 0.445 AS swing ime 0.069 0.097 0.000 0.115 0.300 0.000 0.825 AS s ep leng h 0.118 0.073 0.111 0.111 0.104 0.013 0.398 AS s ide leng h 0.027 0.028 0.005 0.049 0.077 0.000 0.227 AS eloci y 0.173 0.103 0.273 0.140 0.107 0.008 0.122 AS cadence 8.772 6.882 0.000 7.226 6.754 0.008 0.277 5.3.3 SSQ AND IMI QUESTIONNAIRES A he end o da a acquisi ion, pa icipan s comple ed he a o emen ioned accep abili y ques ionnai es, SSQ and IMI (Appendix B – Subjec i e ques ionnai es). Table 5-15 p esen s he esul s o hese ques ionnai es. 70 Table 5-15 - SSQ and IMI ques ionnai es esul s Pa icipan ID SSQ IMI In e es /Enjoymen subscale Value/Use ulness subscale PD- 01 4 5.67 5.33 PD- 02 3 5.67 5.33 PD- 03 4 7 6.33 PD 04 0 7 5.67 PD 05 0 7 7 PD- 06 4 5.33 7 PD- 07 0 7 7 PD- 08 3 6.67 6.67 PD- 09 1 7 7 PD- 10 1 7 6.67 PD- 11 1 7 6.33 Mean (±STD) 1.91 (± 1.62) 6.58 (± 0.64) 6.39 (± 0.63) 6.49 (± 0.64) 5.4 DISCUSSION This subsec ion discusses he esul s ob ained in he p e ious subsec ion. The analysis is made o bo h “mo o assessmen ” and “mo o aining” sepa a ely. In addi ion, a b ie discussion is elabo a ed ega ding he occu ence o FoG episodes and also he esul s o he accep abili y ques ionnai es. 5.4.1 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR ASSESSMENT Looking a he mean alues o he me ics o he wo con ol es s (TC1 and TC2), i can be seen ha hey ha dly a ied, wi h he excep ion o s ep and s ide leng h and eloci y. Ac ually, he e e ed me ics plus AS swing ime p esen ed s a is ically signi ican di e ences. Thus, i is concluded ha he use o HoloLens in luences mo o pe o mance in he men ioned me ics, e en i hey a e swi ched o . This may be due o he p esence o he HoloLens lenses ha a e no comple ely anspa en , seeing some e lec ions. Vi ual en i onmen 2 (doo s) showed an inc ease in he mean alues o s ep and s ide du a ion, and a dec ease in s ep and s ide leng h, eloci y, and cadence. These me ics beha ed as expec ed (jus like [2]) wi h he excep ion o cadence, which should ha e inc eased. In ac , his i ual en i onmen showed ou een me ics ou o wen y- i e (s ep and s ide du a ion, s ance, swing and double suppo phase, eloci y, SD s ep and s ide du a ion, SD s ance and swing phase, SD s ep leng h, SD cadence and AS s ance and swing ime) wi h s a is ically signi ican di e ences 71 be ween he con ol and moni o ing es s. This means ha he MR echnology eally dis u bed he mo o pe o mance o he pa ien s in he way ha was expec ed, due o he p esence o i ual objec s and especially he i ual doo ha was able o ec ea e a eal doo . In u n, i ual en i onmen 3 (na ow spaces) showed an inc ease in he mean alues o s ep and s ide du a ion and a dec ease in s ep and s ide leng h, eloci y, and cadence. Ac ually, hese me ics beha ed as expec ed wi h he excep ion o cadence, which should ha e inc eased. Obse ing he esul s o he Wilcoxon es , six een ou o wen y- i e me ics showed s a is ically signi ican di e ences, demons a ing ha he MR echnology may in ac ha e igge ed PD-gai ela ed disabili ies. This may be due o he p esence o he a ious i ual objec s ha c ea ed a na owe co ido han he eal co ido , ac ing as obs acles o he pa icipan , making him ake smalle and slowe s eps. Wi h ega d o he occu ence o FoG episodes, he moni o ing es s should ha e inc eased hei numbe and du a ion. Howe e , a e analysing he esul s, i was ound ha moni o ing es 2 was he only one ha caused hese episodes. This may be due o he ac ha : (1) disease may be “masked” by medica ion, causing pa icipan s no o su e om FoG, since da a acquisi ion was pe o med 1h a e medica ion in ake on a e age, i.e., in “ON” phase; (2) he e ogenei y in he o igin o FoG, since some pa icipan s epo ed ha hey su e om hese episodes in s ess ul si ua ions, o he pa ien s su e igh a e waking up, as well as, o he pa ien s su e when hey a e in c owded places. 5.4.2 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR TRAINING Rega ding i ual en i onmen 1 (dices), one would expec he a e age ROM and RMS alues o inc ease and he a e age JERK alues o dec ease wi h he use o he bio eedback s a egy. Howe e , by analysing he esul s one no ices ha he a e age ROM alues o he h ee axes dec eased, in u n he a e age RMS alues o all axes inc eased and he same happened o he a e age JERK alues. Thus, only he RMS beha ed as expec ed. F om he esul s o he Wilcoxon es , no s a is ically signi ican di e ences we e ound, making i possible o men ion ha he isual bio eedback s a egy, namely he colou ed a ows, had no e iden impac . This may ha e occu ed because he sma glasses do no ha e a su icien ield o iew (FOV), o cing pa ien s o look in he di ec ion o he loo . Ano he eason could be ha he e was li le ime in con ac wi h he bio eedback s a egy. Vi ual en i onmen 2 (doo s) showed an inc ease in he mean alues o s ep and s ide du a ion, a dec ease in he mean alues o s ep and s ide leng h as well as a dec ease in he mean alues o eloci y and cadence. Ne e heless, s ep and s ide du a ion and cadence we e expec ed o dec ease, 72 s ep and s ide leng h we e expec ed o inc ease, jus as eloci y, since his bio eedback s a egy aims o imp o e mo o pe o mance, ob aining la ge and as e s eps. In ac , his i ual en i onmen showed some me ics (s ep and s ide du a ion, eloci y, SD eloci y and cadence) wi h s a is ically signi ican di e ences be ween he moni o ing and aining es s. Thus, bio eedback may ha e nega i ely a ec ed hese me ics once he mean alues beha ed con a y o wha was expec ed. This may be due o (1) he pa icipan was le wai ing o he oo p in s, and (2) he eal- ime IC and FC de ec ion algo i hm was no sui able o hese pa ien s' gai causing he oo p in s no o appea igh away, inc easing hei eac ion ime. On he o he hand, he cadence alues beha ed as expec ed. This may be due o he ac ha he oo p in s indica e spa ial in o ma ion o he pa icipan , i.e., whe e o place he nex oo . In u n, i ual en i onmen 3 (na ow spaces) showed an inc ease in he mean alues o s ep and s ide du a ion as well as s ep, s ide leng h and eloci y. On he o he hand, he cadence dec eased i s mean alue wi h he use o he oo p in s. Ac ually, s ep and s ide leng h, eloci y and cadence beha ed as i was expec ed. On he con a y, s ep and s ide du a ion should ha e dec eased. The e a e wo spa io empo al me ics ha showed s a is ically signi ican di e ences, namely s ep leng h and cadence. This may be due o he in en ion o he oo p in s o " o ce" he pa icipan o be awa e o hem and o help planning whe e o place his ee , guiding him o he inish line. In addi ion, he oo p in s had a p e-de ined dis ance be ween hem (dependen on he heigh o he pa icipan ), in luencing he pa icipan o ollow and imi a e he isual cues. In wha conce ns he occu ence o eezing o gai episodes, he numbe and du a ion o hese episodes would be expec ed o educe o disappea in he bio eedback aining ials. Howe e , his did no occu and may be due o (1) un amilia i y wi h isual cues, as pa icipan s epo ed ha hey had ne e in e ac ed wi h hese; (2) educed ield o iew o he HoloLens 2 causing pa icipan s o some imes ail o see isual cues. 5.4.3 SSQ AND IMI QUESTIONNAIRES A e analysing he esul s ob ained o he SSQ i was concluded ha hey did no e lec any symp oms (nausea, diso ien a ion, and oculomo o ) a e exposu e o he i ual en i onmen s. Fu he mo e, no pa icipan s e bally indica ed ha hey had symp oms o simula o sickness. In u n, he esul s o he IMI ques ionnai e showed ha he sys em ecei ed high a ings on he in e es and alue subscales. The pa icipan s we e always happy and willing o pa icipa e in he es s, 79 [1] Y. W. Wang, C. H. Chen, and Y. C. Lin, “Balance Rehabili a ion Sys em o Pa kinson’s Disease Pa ien s based on Augmen ed Reali y,” 2nd IEEE Eu asia Con e ence on IOT, Communica ion and Enginee ing 2020, ECICE 2020 , pp. 191–194, Oc . 2020, doi: 10.1109/ECICE50847.2020.9302018. [2] L. I. Gómez-Jo dana, J. S a o d, C. E. Pepe , and C. M. C aig, “C ossing Vi ual Doo s: A New Me hod o S udy Gai Impai men s and F eezing o Gai in Pa kinson’s Disease,” Pa kinsons Dis , ol. 2018, 2018, doi: 10.1155/2018/2957427. [3] A. Mi elman e al. , “Gai impai men s in Pa kinson’s disease,” Lance Neu ol , ol. 18, no. 7, pp. 697–708, Jul. 2019, doi: 10.1016/S1474-4422(19)30044-4. [4] J. Nonnekes, A. H. Snijde s, J. G. Nu , G. Deuschl, N. Giladi, and B. R. Bloem, “F eezing o gai : a p ac ical app oach o managemen ,” Lance Neu ol , ol. 14, no. 7, pp. 768–778, Jul. 2015, doi: 10.1016/S1474-4422(15)00041-1. [5] M. Mancini, K. Smulde s, G. Ha ke , S. S ua , and J. G. Nu , “Assessmen o he abili y o open- and closed-loop cueing o imp o e u ning and eezing in people wi h Pa kinson’s disease,” Scien i ic Repo s 2018 8:1 , ol. 8, no. 1, pp. 1–9, Aug. 2018, doi: 10.1038/s41598-018-31156- 4. [6] P. H. Chen, R. L. Wang, D. J. Liou, and J. S. Shaw, “Gai Diso de s in Pa kinson’s Disease: Assessmen and Managemen ,” In J Ge on ol , ol. 7, no. 4, pp. 189–193, Dec. 2013, doi: 10.1016/J.IJGE.2013.03.005. [7] R. Iansek, F. Huxham, and J. McGinley, “The sequence e ec and gai es ina ion in Pa kinson disease: con ibu o s o eezing o gai ?,” Mo Diso d , ol. 21, no. 9, pp. 1419–1424, Sep. 2006, doi: 10.1002/MDS.20998. [8] R. Chong e al. , “Closed-Loop VR-Based In e ac ion o Imp o e Walking in Pa kinson’s Disease,” Jou nal o No el Physio he apies 2011 1:1 , ol. 1, no. 1, pp. 1–7, Dec. 2011, doi: 10.4172/2165- 7025.1000101. [9] A. L. Sil a de Lima e al. , “F eezing o gai and all de ec ion in Pa kinson’s disease using wea able senso s: a sys ema ic e iew,” J Neu ol , ol. 264, no. 8, pp. 1642–1654, Aug. 2017, doi: 10.1007/S00415-017-8424-0. [10] S. M. Rissanen, M. Koi u, P. Ha ikainen, and E. Pekkonen, “Ambula o y su ace elec omyog aphy wi h accele ome y o e alua ing daily mo o luc ua ions in Pa kinson’s disease,” Clin Neu ophysiol , ol. 132, no. 2, pp. 469–479, Feb. 2021, doi: 10.1016/J.CLINPH.2020.11.039. 80 [11] A. Lheu eux e al. , “Imme si e Vi ual Reali y o Res o e Na u al Long-Range Au oco ela ions in Pa kinson’s Disease Pa ien s’ Gai Du ing T eadmill Walking,” F on Physiol , ol. 11, p. 1207, Sep. 2020, doi: 10.3389/FPHYS.2020.572063/BIBTEX. [12] M. Yamagami e al. , “E ec s o i ual eali y en i onmen s on o e g ound walking in people wi h Pa kinson disease and eezing o gai ,” Disabil Rehabil Assis Technol , 2020, doi: 10.1080/17483107.2020.1842920. [13] Y. Zhao e al. , “Feasibili y o ex e nal hy hmic cueing wi h he Google Glass o imp o ing gai in people wi h Pa kinson’s disease,” J Neu ol , ol. 263, no. 6, p. 1156, Jun. 2016, doi: 10.1007/S00415-016-8115-2. [14] S. Janssen e al. , “Usabili y o h ee-dimensional augmen ed isual cues deli e ed by sma glasses on (F eezing o ) gai in pa kinson’s disease,” F on Neu ol , ol. 8, no. JUN, p. 279, Jun. 2017, doi: 10.3389/FNEUR.2017.00279/BIBTEX. [15] S. Janssen e al. , “The E ec s o Augmen ed Reali y Visual Cues on Tu ning in Place in Pa kinson’s Disease Pa ien s Wi h F eezing o Gai ,” F on Neu ol , ol. 11, p. 185, Ma . 2020, doi: 10.3389/FNEUR.2020.00185/FULL. [16] D. J. Moo e, A. B. Wes , V. L. Dawson, and T. M. Dawson, “Molecula pa hophysiology o Pa kinson’s disease,” Annual Re iew o Neu oscience , ol. 28. pp. 57–87, 2005. doi: 10.1146/annu e .neu o.28.061604.135718. [17] L. I. Gómez-Jo dana, J. S a o d, C. E. Pepe , and C. M. C aig, “Vi ual Foo p in s Can Imp o e Walking Pe o mance in People Wi h Pa kinson’s Disease,” F on Neu ol , ol. 9, no. AUG, Aug. 2018, doi: 10.3389/FNEUR.2018.00681. [18] D. J. Gee se, B. Coolen, J. J. an Hil en, and M. Roe dink, “Holocue: A Wea able Holog aphic Cueing Applica ion o Alle ia ing F eezing o Gai in Pa kinson’s Disease,” F on Neu ol , ol. 12, Jan. 2022, doi: 10.3389/ neu .2021.628388. [19] J. Janko ic, “Pa kinson’s disease: Clinical ea u es and diagnosis,” Jou nal o Neu ology, Neu osu ge y and Psychia y , ol. 79, no. 4. BMJ Publishing G oup, pp. 368–376, 2008. doi: 10.1136/jnnp.2007.131045. [20] A. Besha a e al. , “Vi ual eali y doo way and hallway en i onmen s al e gai kinema ics in people wi h Pa kinson disease and eezing,” Gai Pos u e , ol. 92, pp. 442–448, Feb. 2022, doi: 10.1016/j.gai pos .2021.12.013. 81 [21] S. Bada ny, J. Aha on-Pe e z, Z. Susel, G. Habib, and Y. Ba am, “Vi ual eali y eedback cues o imp o emen o gai in pa ien s wi h Pa kinson’s disease,” T emo O he Hype kine Mo (N Y) , ol. 4, no. 0, p. 225, Ap . 2014, doi: 10.7916/D8V69GM4. [22] Y. W. Wang, C. H. Chen, and Y. C. Lin, “Balance Rehabili a ion Sys em o Pa kinson’s Disease Pa ien s based on Augmen ed Reali y,” 2nd IEEE Eu asia Con e ence on IOT, Communica ion and Enginee ing 2020, ECICE 2020 , pp. 191–194, Oc . 2020, doi: 10.1109/ECICE50847.2020.9302018. [23] T. Tunu , A. DeBlois, E. Ya es-Ho on, K. Rick o d, and L. A. Columna, “Augmen ed eali y-based dance in e en ion o indi iduals wi h Pa kinson’s disease: A pilo s udy,” Disabil Heal h J , ol. 13, no. 2, Ap . 2020, doi: 10.1016/J.DHJO.2019.100848. [24] R. S. Calab ò e al. , “Imp o ing mo o pe o mance in Pa kinson’s disease: a p elimina y s udy on he p omising use o he compu e assis ed i ual eali y en i onmen (CAREN),” Neu ological Sciences , ol. 41, no. 4, pp. 933–941, Ap . 2020, doi: 10.1007/S10072-019-04194-7. [25] J. M. Finley, M. Go sis, V. Lympou idis, S. Jain, A. Kim, and B. E. Fishe , “Design and De elopmen o a Vi ual Reali y-Based Mobili y T aining Game o People Wi h Pa kinson’s Disease,” F on Neu ol , ol. 11, Jan. 2021, doi: 10.3389/FNEUR.2020.577713. [26] P. Campo-P ie o e al. , “Can Imme si e Vi ual Reali y Videogames Help Pa kinson’s Disease Pa ien s? A Case S udy,” Senso s 2021, Vol. 21, Page 4825 , ol. 21, no. 14, p. 4825, Jul. 2021, doi: 10.3390/S21144825. [27] Y. Wang e al. , “E icacy o C-Mill gai aining o imp o ing walking adap abili y in ea ly and middle s ages o Pa kinson’s disease,” Gai Pos u e , ol. 91, pp. 79–85, Jan. 2022, doi: 10.1016/J.GAITPOST.2021.10.010. [28] A. Kim, N. Da akjian, and J. M. Finley, “Walking in ully imme si e i ual en i onmen s: an e alua ion o po en ial ad e se e ec s in olde adul s and indi iduals wi h Pa kinson’s disease,” J Neu oeng Rehabil , ol. 14, no. 1, Feb. 2017, doi: 10.1186/S12984-017-0225-2. [29] H. R. Gonçal es, A. M. Rod igues, and C. P. San os, “Vib o ac ile bio eedback de ices in Pa kinson’s disease: a na a i e e iew,” Med Biol Eng Compu , ol. 59, no. 6, pp. 1185–1199, Jun. 2021, doi: 10.1007/S11517-021-02365-3. [30] “HoloLens 2—O e iew, Fea u es, and Specs | Mic oso HoloLens.” h ps://www.mic oso .com/en-us/hololens/ha dwa e# (accessed Oc . 05, 2022). [31] “Hololens Sys em Requi emen s – Vangos P e neas.” h ps://p e neas.com/2016/03/31/hololens-sys em- equi emen s/ (accessed Oc . 05, 2022). 82 [32] “MVN Use Manual,” 2005. [33] “MTw_Awinda_Use _Manual”. [34] “JMU Upda ed o elease MVN Analyze/Anima e 4.0 (new HW + suppo o TCP p o ocol),” 2013, Accessed: Oc . 07, 2022. [Online]. A ailable: h ps://www.asse s o e.uni y3d.com/en/#!/con en /11338 [35] “Nondes uc i e E alua ion Techniques : Pene an .” h ps://www.nde- ed.o g/NDETechniques/Pene an Tes /In oduc ion/ isualacui y.xh ml (accessed Oc . 14, 2022). [36] R. Mo ei a, H. Goncal es, A. Rod igues, and C. San os, “The Po en ial o Visual Cues o O e come F eezing o Gai in Pa kinson’s Disease*; The Po en ial o Visual Cues o O e come F eezing o Gai in Pa kinson’s Disease*,” 2019. [Online]. A ailable: h ps://www.ninds.nih.go /Diso de s/All-Diso de s/Pa kinsons- [37] J. Figuei edo, P. Félix, L. Cos a, J. C. Mo eno, and C. P. San os, “Gai E en De ec ion in Con olled and Real-Li e Si ua ions: Repea ed Measu es om Heal hy Subjec s,” IEEE T ansac ions on Neu al Sys ems and Rehabili a ion Enginee ing , ol. 26, no. 10, pp. 1945–1956, Oc . 2018, doi: 10.1109/TNSRE.2018.2868094. [38] H. R. Gonçal es, A. Rod igues, and C. P. San os, “Gai moni o ing sys em o pa ien s wi h Pa kinson’s disease,” Expe Sys Appl , ol. 185, Dec. 2021, doi: 10.1016/j.eswa.2021.115653. [39] J. Figuei edo, J. C. Mo eno, and C. P. San os, “A u o ial on human locomo ion o de eloping sma gai ehabili a ion de ices.” [40] J. Glogowski, “GSU Lib a y Resea ch Guides: Li e a u e Re iews: Types o Clinical S udy Designs”, Accessed: Oc . 15, 2022. [Online]. A ailable: h ps:// esea ch.lib a y.gsu.edu/c.php?g=115595&p=755213 [41] R. S. Kennedy, N. E. Lane, K. S. Be baum, and M. G. Lilien hal, “Simula o Sickness Ques ionnai e: An Enhanced Me hod o Quan i ying Simula o Sickness,” In J A ia Psychol , ol. 3, no. 3, pp. 203–220, 1993, doi: 10.1207/s15327108ijap0303_3. [42] “In insic Mo i a ion In en o y (IMI) Scale Desc ip ion.” APPENDICES 84 APPENDIX A – EXPERIMENTAL PROTOCOL 85 86 87 88