senso s
Re iew
Comp ehensi e Re iew o Vision-Based Fall De ec ion Sys ems
Jesús Gu ié ez 1,*, Víc o Rod íguez 2and Se gio Ma in 1


Ci a ion: Gu ié ez, J.; Rod íguez, V.;
Ma in, S. Comp ehensi e Re iew o
Vision-Based Fall De ec ion Sys ems.
Senso s 2021,21, 947. h ps://
doi.o g/10.3390/s21030947
Recei ed: 18 Decembe 2020
Accep ed: 25 Janua y 2021
Published: 1 Feb ua y 2021
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2021 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
1Uni e sidad Nacional de Educación a Dis ancia, Juan Rosal 12, 28040 Mad id, Spain; [email p o ec ed]
2EduQTech, E.U. Poli écnica, Ma ia Lluna 3, 50018 Za agoza, Spain; [email p o ec ed]
*Co espondence: jgu ie [email p o ec ed]
Abs ac :
Vision-based all de ec ion sys ems ha e expe ienced as de elopmen o e he las yea s.
To de e mine he cou se o i s e olu ion and help new esea che s, he main audience o his pape ,
a comp ehensi e e ision o all published a icles in he main scien i ic da abases ega ding his
a ea du ing he las i e yea s has been made. A e a selec ion p ocess, de ailed in he Ma e ials
and Me hods Sec ion, eigh y-one sys ems we e ho oughly e iewed. Thei cha ac e iza ion and
classi ica ion echniques we e analyzed and ca ego ized. Thei pe o mance da a we e also s udied,
and compa isons we e made o de e mine which classi ying me hods bes wo k in his ield. The
e olu ion o a i icial ision echnology, e y posi i ely in luenced by he inco po a ion o a i icial
neu al ne wo ks, has allowed all cha ac e iza ion o become mo e esis an o noise esul an om
illumina ion phenomena o occlusion. The classi ica ion has also aken ad an age o hese ne wo ks,
and he ield s a s using obo s o make hese sys ems mobile. Howe e , da ase s used o ain hem
lack eal-wo ld da a, aising doub s abou hei pe o mances acing eal elde ly alls. In addi ion,
he e is no e idence o s ong connec ions be ween he elde ly and he communi ies o esea che s.
Keywo ds:
a i icial ision; neu al ne wo ks; all de ec ion; all cha ac e iza ion; all classi ica ion;
all da ase
1. In oduc ion
In acco dance wi h he UN epo on he aging popula ion [
1
], he global popula ion
aged o e 60 doubled i s numbe in 2017 compa ed o 1980. I is expec ed o double again
by 2050 when hey exceed he 2 billion ma k. By his ime, hei numbe will be g ea e
han he numbe o eenage s and youngs e s aged 10 o 24.
The phenomenon o popula ion aging is a global one, mo e ad anced in he de eloped
coun ies, bu also p esen in he de eloping ones, whe e wo- hi ds o he wo lds olde
people li e, a numbe which is ising as .
Wi h his pe spec i e, he amoun o esou ces de o ed o elde ly heal h ca e is
inc easingly high and could, in he non-dis an u u e, become one o he mos ele an
wo ld economic sec o s. Because o his, all elde ly heal h- ela ed a eas ha e a ac ed
g ea esea ch a en ion o e he las decades.
One o he a eas imme sed in his body o esea ch has been human all de ec ion, as,
o his communi y, o e 30% o alls cause impo an inju ies, anging om hip ac u e o
b ain concussion, and a good numbe o hem end up causing dea h [2].
The numbe o echnologies used o de ec alls is wide, and a huge numbe o sys ems
able o wo k wi h hem ha e been de eloped by esea che s. These sys ems, in b oad
e ms, can be classi ied as wea able, ambien and came a-based ones [3].
The i s block, he wea able sys ems, inco po a e senso s ca ied by he su eilled
indi idual. The echnologies used by his g oup o sys ems a e nume ous, anging om
accele ome e s o p essu e senso s, including inclinome e s, gy oscopes o mic ophones,
among o he senso s. R. Rucco e al. [
4
] ho oughly e iew hese sys ems and s udy hem
in-dep h. In his a icle, sys ems a e classi ied in acco dance wi h he numbe and ype
o senso s, hei placemen and he cha ac e is ics o he s udy made du ing he sys em
Senso s 2021,21, 947. h ps://doi.o g/10.3390/s21030947 h ps://www.mdpi.com/jou nal/senso s
Senso s 2021,21, 947 2 o 50
e alua ion phase concluding ha mos sys ems inco po a e one o wo accele ome ic
senso s a ached o he unk.
The second block includes sys ems whose senso s a e placed a ound he moni o ed
pe son and include p essu e, acous ic, in a- ed, and adio- equency senso s. The las
block, he objec o his e iew, g oups sys ems able o iden i y alls h ough a i icial ision.
In pa allel, o e he las yea s, a i icial ision has expe ienced as de elopmen ,
mainly due o he use o a i icial neu al ne wo ks and hei abili y o ecognize objec s
and ac ions.
This a i icial ision de elopmen applied o human ac i i y ecogni ion in gene al,
and human all de ec ion in pa icula , has gi en e y ui ul ou comes in he las decade.
Howe e , up o whe e we know, no sys ema ic e iews on he speci ic a ea o ision-
based de ec ion sys ems ha e been made, as all e e ences o his ield ha e been included
in gene ic all de ec ion sys em e iews.
This e iew in ends o shed some ligh on he p ocess o de elopmen ollowed by
ision-based all de ec ion sys ems, so esea che s ge a clea image o wha has been done
in his ield du ing he las i e yea s ha help hem in hei in es iga ion p ocess. In his
s udy, au ho s in end o show he main ad an ages and disad an ages o all p ocesses
and algo i hms used in he e iewed sys ems so new de elope s ge a clea pic u e o he
s a e o he a in he ield o human all de ec ion h ough a i icial ision, an a ea ha
could signi ican ly imp o e li ing s anda ds o he dependen communi y and ha e a
high impac on hei day- o-day li es.
The a icle is o ganized as ollows: In Sec ion 2, Ma e ials and Me hods, cha ac e iza-
ion and classi ica ion echniques a e desc ibed and applied o he p eselec ed sys ems, so
a numbe o hem a e inally decla ed as eligible o be included in his e iew. In Sec ion 3,
Resul s, hose sys ems a e p esen ed and oughly desc ibed, he da abases used o hei
alida ion a e p esen ed, and some pe o mance compa isons a e made. In he nex Sec ion,
Discussion, he algo i hms and p ocesses used by he sys ems a e desc ibed and, in he
las pa o he e iew, Sec ion 5, conclusions a e ex ac ed based on all he p e iously
p esen ed in o ma ion.
2. Ma e ials and Me hods
In his pape , we ocus on a i icial ision sys ems able o de ec human alls. To ul ill
his pu pose, we ha e pe o med a deep e iew o all published pape s p esen in public
da abases o esea ch documen a ion (ScienceDi ec , IEEE Explo e , Senso s da abase).
This documen al sea ch was based on di e en ex s ing sea ches and was execu ed om
May 2020 o Decembe 2020. The ime ame o publica ion was es ablished be ween 2015
and 2020, so he las de elopmen s in he ield can be iden i ied, and he s udy se es o
o ien a e new esea che s. The e ms used in he bibliog aphical Boolean explo a ion we e
“ all de ec ion” and “ ision”. A seconda y sea ch was ca ied ou o comple e he i s one
by using o he sea ch engines o schola ly li e a u e ocused on heal h (PubMed, MedLine).
All sea ches ha e been limi ed o a icles and publica ions in English, language used by
mos a ea esea che s.
A e an ini ial analysis o pape s ul illing hese sea ching c i e ia 81 a icles, desc ib-
ing he same numbe o sys ems we e selec ed. They illus a e how all de ec ion sys ems
based on a i icial ision ha e e ol ed in he las i e yea s.
The selec ion p ocess included an ini ial sc eening made h ough e e ence manage-
men so wa e o gua an ee no duplica ion, and a manual sc eening, whose objec i e was
making su e he a icle co e ed he ield, did no all wi hin he ield o he all p e en ion
o human ac i i y ecogni ion (HAR), did no mix ision echnologies wi h o he ones and
we e no s udies in ending o classi y he human gai as an indica o o all p obabili y.
This way, he e iew is pu ely cen e ed on a i icial ision all de ec ion.
The en i e p ocess is summa ized in he low diag am shown in Figu e 1.
Senso s 2021,21, 947 3 o 50
Senso s 2020, 20, x FOR PEER REVIEW 3 o 50
Figu e 1. Flow diag am o adop ed sea ch and selec ion s a egy o pape selec ion.
All selec ed sys ems we e s udied one-by-one o de e mine hei cha ac e iza ion and
classi ica ion echniques, desc ibing hem in-dep h in he Discussion (Sec ion 4), so a ull
axonomy can be made based on hei cha ac e is ics. In addi ion, pe o mance compa i-
sons a e also included, so conclusions on which ones a e he mos sui able sys ems can be
eached.
3. Resul s
The a icle sea ch and selec ion p ocess s a ed wi h an ini ial iden i ica ion o 929
po en ial a icles. Duplica ed ones and hose whose i le clea ly did no ma ch he equi ed
con en we e disca ded, lea ing 430 a icles ha we e assessed o eligibili y. These a i-
cles we e hen e iewed, and hose ela ed o HAR, all p e en ion, mixed echnologies,
gai s udies and he ones which did no co e he a ea o ision-based all de ec ion we e
disca ded, so; inally, 81 a icles a e conside ed in he e iew.
The selec ed sys ems we e ho oughly e ised and classi ied in acco dance wi h he
used cha ac e iza ion and classi ica ion me hods, as well as he employed ype o signal.
The used da ase o pe o mance de e mina ion and i s indica o s alues ha e also been
s udied. All his in o ma ion is included in Table 1.
Sys em compa ison da a we e used o de elop Table 2, and inally, all main cha ac-
e is ics o publicly accessible da ase s used by any o he sys ems a e included in Table 3.
Figu e 1. Flow diag am o adop ed sea ch and selec ion s a egy o pape selec ion.
All selec ed sys ems we e s udied one-by-one o de e mine hei cha ac e iza ion
and classi ica ion echniques, desc ibing hem in-dep h in he Discussion (Sec ion 4), so
a ull axonomy can be made based on hei cha ac e is ics. In addi ion, pe o mance
compa isons a e also included, so conclusions on which ones a e he mos sui able sys ems
can be eached.
3. Resul s
The a icle sea ch and selec ion p ocess s a ed wi h an ini ial iden i ica ion o 929 po-
en ial a icles. Duplica ed ones and hose whose i le clea ly did no ma ch he equi ed
con en we e disca ded, lea ing 430 a icles ha we e assessed o eligibili y. These a i-
cles we e hen e iewed, and hose ela ed o HAR, all p e en ion, mixed echnologies,
gai s udies and he ones which did no co e he a ea o ision-based all de ec ion we e
disca ded, so; inally, 81 a icles a e conside ed in he e iew.
The selec ed sys ems we e ho oughly e ised and classi ied in acco dance wi h he
used cha ac e iza ion and classi ica ion me hods, as well as he employed ype o signal.
The used da ase o pe o mance de e mina ion and i s indica o s alues ha e also been
s udied. All his in o ma ion is included in Table 1.
Sys em compa ison da a we e used o de elop Table 2, and inally, all main cha ac e -
is ics o publicly accessible da ase s used by any o he sys ems a e included in Table 3.
Senso s 2021,21, 947 4 o 50
Table 1. Vision-based all de ec ion sys ems published 2015–2020.
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
A. Yajai
e al. [5]2015
Skele on join acking model p o ided
by MS Kinec ®is used o ack join s and
build a 2D and 3D bounding box a ound
he body/dep h cha ac e iza ion
Fea u e- h eshold-based.
•
Heigh /wid h a io o he bounding
box
•cen e o g a i y (CG) posi ion in
ela ion o suppo polygon (de ined
by ankle join s)
Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 98.43%
Speci ici y 98.75%
Recall 98.12%
C. -J. Chong
e al. [6]2015 Pixel clus e ing and backg ound
(Ho p ase )/global cha ac e iza ion
Fea u e- h eshold-based.
Me hod 1:
•Bounding box (BB) aspec a io
•CG posi ion
Me hod 2:
•Ellipse o ien a ion and aspec a io
•Mo ion his o y image (MHI)
Red-
g een-
blue
(RGB)
Speci ic ideo da ase —no public
access a e ision ime
Me hod 1
Sensi i i y 66.7%
Speci ici y 80%
Me hod 2
Sensi i i y 72.2%
Speci ici y 90%
H. Rajabi
e al. [7]2015
Fo eg ound ex ac ion h ough
backg ound sub ac ion (Gaussian mixed
models—GMM) and Sobel il e
applica ion/ global cha ac e iza ion
Fea u e- h eshold-based.
•BB o ien a ion angle
•Change o CG wid h
•Heigh /wid h ela ion o con ou
•Hu momen in a ian s
RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Fall de ec ion success
a e 81%
L. H. Juang
e al. [8]2015
Fo eg ound ex ac ion h ough
backg ound sub ac ion (op ical
low-based) and human join s
iden i ied/global cha ac e iza ion
Suppo ec o machine (SVM) RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy up o 100%
M. A. Mousse
e al. [9]2015
Fo eg ound ex ac ion h ough pixel
colo and b igh ness dis o ion
de e mina ion and in eg a ion o
o eg ound maps h ough
homog aphy/global cha ac e iza ion
Fea u e- h eshold-based.
Ra io obse ed silhoue e a ea/silhoue e
a ea p ojec ed on he g ound plane
RGB—2
OR-
THOGO-
NAL
VIEWS
Mul icam Fall Da ase [10]Sensi i i y 95.8%
Speci ici y 100%
Muza e
Aslan
e al. [11]
2015
Human silhoue e is segmen ed using
dep h in o ma ion, and cu a u e scale
space (CSS) is calcula ed and encoded in
a Fishe ec o /dep h cha ac e iza ion
SVM Dep h SDUFall [12]A e age accu acy
88.01%
Senso s 2021,21, 947 5 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
Z. Bian
e al. [13]2015
Silhoue e ex ac ion by using dep h
in o ma ion. Human body join s
iden i ied and acked wi h o so
o a ion/dep h cha ac e iza ion
SVM Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Sensi i i y 95.8%
Speci ici y 100%
C. Lin
e al. [14]2016
Fo eg ound ex ac ion h ough
backg ound sub ac ion (GMM)/global
cha ac e iza ion
Fea u e- h eshold-based.
•Ellipse o ien a ion
•Linea and angula accele a ion
•MHI
RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
No published
F. Me ouche
e al. [15]2016
Fo eg ound ex ac ion by using he
di e ence be ween dep h ames and
head acking h ough pa icle
il e /dep h cha ac e iza ion
Fea u e- h eshold-based.
•Ra io head e ical posi ion/pe son
heigh
•CG eloci y
Dep h SDUFall [12]
Sensi i i y 90.76%
Speci ici y 93.52%
Accu acy 92.98%
K. G. Gunale
e al. [16]2016
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/global cha ac e iza ion
K-nea es neighbo (KNN) RGB Chu e da ase —no public access a
e ision ime
Accu acy
Fall 90%
No all 100%
K. R. Bha ya
e al. [17]2016
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/global cha ac e iza ion +
op ical low (OF)/global cha ac e iza ion
KNN on MHI and OF ea u es RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
No published
Kun Wang
e al. [18]2016
Segmen a ion h ough ibe [19] and
his og am o o ien ed g adien s (HOG)
and local bina y pa e n (LBP)/global
cha ac e iza ion + ea u e maps ob ained
h ough con olu ional neu al ne wo k
(CNN)/ local cha ac e iza ion
SVM-linea ke nel RGB
Mul icam Fall Da ase [10] and
SIMPLE Fall De ec ion Da ase [20]
and This sys em-speci ic ideo
da ase —no public access a
e ision ime
Sensi i i y 93.7%
Speci ici y 92%
U. P a ap
e al. [21]2016
Fo eg ound ex ac ion h ough
backg ound sub ac ion (GMM)/global
cha ac e iza ion
Fea u e- h eshold-based.
•Silhoue e CG s a iona y o e a
h eshold ime limi
RGB Speci ic ideo da ase s—no public
access a e ision ime
Fall de ec ion a e
92%
False ala m a e
6.25%
X. Wang
e al. [22]2016
Segmen a ion h ough ibe [19] and
uppe body da abase popula ed and
spa se OF de e mined/global
cha ac e iza ion
Fea u e- h eshold-based.
•Body a io wid h/heigh
•Ve ical eloci y de i ed om OF
•Uppe body posi ion his o y
RGB LE2I [23]A e age p ecision
81.55%

Senso s 2021,21, 947 6 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
A. Y. Alaoui
e al. [24]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/global cha ac e iza ion +
OF/global cha ac e iza ion
No classi ica ion algo i hm epo ed RGB CHARFI2012 Da ase [25]P ecision 91%
Sensi i i y 86.66%
Apiche Yajai
e al. [26]2017 Skele on join acking model p o ided
by MS Kinec ®/dep h cha ac e iza ion
Fea u e- h eshold-based.
Aspec a ios:
•Bounding box
•CoG
•Bounding box diagonal s. max.
heigh
•Bounding box heigh s. max.
heigh
Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 98.15%
Sensi i i y 97.75%
Speci ici y 98.25%
B.
Lewandowski
e al. [27]
2017
oxels a ound he poin cloud a e
calcula ed. The ones classi ied as human
a e clus e ed, and IRON ea u es a e
calcula ed/local cha ac e iza ion
Fea u e- h eshold-based.
•Mahalanobis dis ance be ween
clus e IRON ea u es and he
dis ibu ion o IRON ea u es om
allen bodies
Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Sensi i i y in
ope a ional
en i onmen s 99%
F. Ha ou
e al. [28]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/dep h cha ac e iza ion
Mul i a ia e exponen ially weigh ed
mo ing a e age (MEWMA)-SVM
KNN
A i icial neu al ne wo k (ANN)
Naï e Bayes (NB)
RGB UR Fall De ec ion [29] &
Fall De ec ion Da ase [30]
Accu acy
KNN 91.94%
ANN 95.15%
NB 93.55%
NEWMA-SVM
96.66%
G. M.
Basa a aj
e al. [31]
2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (median)/global
cha ac e iza ion
Fea u e- h eshold-based.
•Ellipse eccen ici y and o ien a ion
•MHI
RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy
Fall 86.66%
Non- all 90%
K. Adhika i
e al. [30]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison) using bo h RGB echniques
and dep h ones and Fea u e maps
ob ained h ough CNN/local and dep h
cha ac e iza ion
So max based on ea u es ec o om
CNN Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
O e all, accu acy
74%
Sys em sensi i i y o
lying pose 99%
Senso s 2021,21, 947 7 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
Koldo De
Miguel
e al. [32]
2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (GMM) + Spa se
OF de e mined/global cha ac e iza ion
KNN on silhoue e and OF ea u es RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 96.9%
Sensi i i y 96%
Speci ici y 97.6%
Leiyue Yao
e al. [33]2017 Skele on join acking model p o ided
by MS Kinec ®/dep h cha ac e iza ion
Fea u e- h eshold-based
•To so angle
•Cen oid heigh
Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 97.5%
T ue posi i e a e
98%
T ue nega i e a e
97%
M. An onello
e al. [34]2017
oxels a ound he poin cloud a e
calcula ed. Then hey a e segmen ed in
homogeneous pa ches and he ones
classi ied as human a e ga he ed and
classi ied o no as a human lying
body/dep h cha ac e iza ion
SVM— adial-based ke nel Dep h IASLAB-RGBD allen pe son
Da ase [35]
Se A
Accu acy: single
iew (SV)
0.87/SV+map
e i ica ion (MV)
0.92
P ecision: SV
0.73/SV+MV 0.85
Recall: SV
0.85/SV+MV 0.85
Se B
Accu acy: SV
0.88/SV+MV 0.9
P ecision: SV
0.8/SV+MV 0.87
Recall: SV
0.86/SV+MV 0.81
M. N. H.
Mohd
e al. [36]
2017
Skele on join acking model p o ided
by MS Kinec ®is used o de e mine join
posi ions and speeds/dep h
cha ac e iza ion
SVM based on join s speeds and
ule-based decision-based on join s
posi ion in ela ion o knees
Dep h
TST Fall De ec ion [37] and UR Fall
De ec ion [
29
] and Falling De ec ion
[38]
Accu acy 97.39%
Speci ici y 96.61%
Sensi i i y 100%
N. B. Joshi
e al. [39]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (GMM)/global
cha ac e iza ion
Fea u e- h eshold-based.
•BB wid h/heigh a io
•CG posi ion
•O ien a ion
•Hu momen s
RGB LE2I [23]Speci ici y 92.98%
Accu acy 91.89%
Senso s 2021,21, 947 8 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
N. O anasap
e al. [40]2017 Skele on join acking model p o ided
by MS Kinec ®/dep h cha ac e iza ion
Fea u e- h eshold-based.
•Head eloci y
•CG posi ion in ela ion o ankle
join s
Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Sensi i i y 97%
Accu acy 100%
Q. Feng
e al. [41]2017
CNN is used o de ec and ack people,
and Sub-MHI a e co ela ed o each
pe son BB/local cha ac e iza ion
SVM RGB UR Fall De ec ion [29]
P ecision 96.8%
Recall 98.1%
F197.4%
S. He nandez-
Mendez
e al. [42]
2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison) and silhoue e acking.
Then cen oid and ea u es a e
de e mined/dep h cha ac e iza ion
Fea u e- h eshold-based.
•Angles and a io heigh /wid h o
he BB
Dep h
Dep h And Accele ome ic Da ase
[43] and his sys em-speci ic ideo
da ase —no public access a
e ision ime
The allen pose is
de ec ed co ec ly on
100% o occasions.
S. Kas u i
e al. [44]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/dep h cha ac e iza ion
SVM Dep h UR Fall De ec ion [29]Sensi i i y 100%
Speci ici y 88.33%
S. Kas u i
e al. [45]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/dep h cha ac e iza ion
SVM Dep h UR Fall De ec ion [29]
Accu acy
To al es ing accu acy
96.34%
S. Pa amase
e al. [46]2017
Body ec o cons uc ion and CG
iden i ica ion aking as s a ing poin 16
pa s o he human body/dep h
cha ac e iza ion
Fea u e- h eshold-based.
•CG accele a ion
•Body ec o / e ical angle
Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 100%
Sajjad
Tagh aei
e al. [47]
2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion/dep h
cha ac e iza ion
Hidden Ma ko model (HMM) Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 84.72%
Y. M. Gal ão
e al. [48]2017 Median squa e e o (MSE) e e y 3
ames/global cha ac e iza ion
Mul ilaye pe cep on (MLP)
KNN
SVM—polynomial ke nel
RGB UR Fall De ec ion [29]
F1 sco e:
MLP 0.991
KNN 0.988
SVM—polynomial
ke nel 0.988
Senso s 2021,21, 947 9 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
Thanh-Hai
T an e al. [
49
]
2017
Skele on join acking model p o ided
by MS Kinec
®
/dep h cha ac e iza ion o
Mo ion map ex ac ion om RGB images
and g adien ke nel desc ip o
calcula ed/global cha ac e iza ion
Fea u e- h eshold-based.
•Heigh o hip join
•Ve ical body eloci y
O
•SVM classi ica ion
Dep h o
RGB
UR Fall De ec ion [
29
] and LE2I [
23
]
and Mul imodal Mul i iew Da ase
o Human Ac i i ies [50]
UR Da ase
Sensi i i y 100%
Speci ici y 99.23%
LE2I Da ase
Sensi i i y 97.95%
Speci ici y 97.87%
MULTIMODAL
Da ase (A e age)
Sensi i i y 92.62%
Speci ici y 100%
X. Li
e al. [51]2017
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison) and ea u e maps ob ained
h ough CNN/ local cha ac e iza ion
So max based on ea u es ec o om
CNN RGB UR Fall De ec ion [29]
Sensi i i y 100%
Speci ici y 99.98%
Accu acy 99.98%
Yaxiang Fan
e al. [52]2017
Fea u e maps ob ained h ough CNN
om dynamic images/local
cha ac e iza ion
Classi ica ion made by ully connec ed
las laye s o CNNs RGB
Mul icam Fall Da ase [10] & LE2I
[23] and High-Quali y Da ase [53]
and This sys em-speci ic ideo
da ase —no public access a
e ision ime
Sensi i i y
LE2I 98.43%
Mul icam 97.1%
HIGH-QUALITY
FALL SIM 74.2%
SYSTEM Da ase
63.7%
A. Abobak
e al. [54]2018
Silhoue e ex ac ion by using dep h
in o ma ion. A ea u e ec o o di e en
body pixels based on dep h di e ence
be ween pai s o poin s is c ea ed/dep h
cha ac e iza ion
Random decision o es o pose
ecogni ion and SVM o mo emen
iden i ica ion
Dep h
UR Fall De ec ion [29] and CMU
G aphics Lab—mo ion cap u e
lib a y [55]
Accu acy 96%
P ecision 91%
Sensi i i y 100%
Speci ici y 93%
B. Dai
e al. [56]2018
Fo eg ound ex ac ion h ough
backg ound sub ac ion (di ec
compa ison)/global cha ac e iza ion
Fea u e- h eshold-based.
•BB segmen ed a eas occupancy.
•CG/heigh a io
•CG e ical speed
RGB
UR Fall De ec ion [29] and This
sys em-speci ic ideo da ase —no
public access a e ision ime
Sensi i i y 95%
Speci ici y 96.7%
Senso s 2021,21, 947 16 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
Qingzhen Xu
e al. [98]2020
Human keypoin s iden i ied by
OpenPose (con olu ional pose machines
and human body ec o cons uc ion)
and CNN used o ea u e maps
c ea ion/local cha ac e iza ion
So max based on ea u es ec o om
CNN implemen ed in i s las laye RGB
UR Fall De ec ion [29] and
Mul icam Fall Da ase [10] and
NTU RGB+D Da ase [99]
Accu acy a e 91.7%
Swe N. H un
e al. [100]2020
Fo eg ound ex ac ion h ough
backg ound sub ac ion (GMM)/global
cha ac e iza ion
Hidden Ma ko model (HMM) based
onObse able da a:
•Silhoue e su ace
•Cen oid heigh
•Bounding box aspec a io
RGB LE2I [23]
P ecision 99.05%
Recall 98.37%
Accu acy 99.8%
T. Kalinga
e al. [101]2020
Skele on join acking model p o ided
by MS Kinec ®is used o de e mine join
speeds and angles o di e en body
pa s/dep h cha ac e iza ion
Fea u e- h eshold-based.
•Join speeds and angles o body
pa s Dep h
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 92.5%
Sensi i i y 95.45%
Speci ici y 88%
Weiming
Chen
e al. [102]
2020
Human keypoin s iden i ied by
OpenPose (con olu ional pose machines
and human body ec o
cons uc ion)/local cha ac e iza ion
Fea u e- h eshold-based
•Hip e ical eloci y
•Spine/g ound plane angle
•BB aspec a io
RGB
This sys em-speci ic ideo
da ase —no public access a
e ision ime
Accu acy 97%
Sensi i i y 98.3%
Speci ici y 95%
X. Cai
e al. [103]2020
Fea u e maps ob ained h ough hou glass
con olu ional au o-encode (HCAE)
ANN/local cha ac e iza ion
So max based on ea u es ec o om
HCAE RGB UR Fall De ec ion [29]
Sensi i i y 100%
Speci ici y 93%
Accu acy 96.2%
Y. Chen
e al. [104]2020
Fo eg ound ex ac ion h ough CNN and
Bi-LSTM ANN/local cha ac e iza ion
So max based on ea u es ec o om
RNN-Bi-LSTM RGB
UR Fall De ec ion [29] and This
sys em-speci ic ideo da ase —no
public access a e ision ime
URFD
P ecision 0.897
Recall 0.813
F10.852
Speci ic da ase
P ecision 0.981
Recall 0.923
F10.948

Senso s 2021,21, 947 17 o 50
Table 1. Con .
Re e ence Yea Cha ac e iza ion (Global/Local/Dep h) Classi ica ion Inpu
Signal Used Da ase s Pe o mance
Yuxi Chen
e al. [105]2020
Fea u e maps ob ained h ough 3
di e en CNNs (LeNe , AlexNe y
GoogLeNe )/dep h cha ac e iza ion
Classi ica ion made by ully connec ed
las laye s o CNNs Dep h Video da ase de eloped o he
sys em in [84]
A e age alues
Lene
Sensi i i y 82.78%
Speci ici y 98.07%
AlexNe
Sensi i i y 86.84%
Speci ici y 98.41%
GoogLeNe
Sensi i i y 92.87%
Speci ici y 99%
X. Wang
e al. [106]2020
Fea u e maps ob ained h ough
con olu ional laye s o an ANN/local
cha ac e iza ion
Logis ic unc ion o iden i y
ame-by- ame wo classes in he
p edic ion laye (pe son and allen)
RGB UR Fall De ec ion [29] &
Fall De ec ion Da ase [30]
A e age p ecision
(AP) o allen 0.97
mean a e age
p ecision (mAP) o
bo h classes 0.83
Table 2. Sys em pe o mance compa ison.
Re e ence Yea Inpu Signal ANN/Classi ie s and Pe o mance
C. -J. Chong
e al. [6]2015 RGB
Me hod 1 BB aspec a io and CG posi ion
Sensi i i y 66.7%
Speci ici y 80%
Me hod 2 Ellipse o ien a ion and aspec a io + MHI
Sensi i i y 72.2%
Speci ici y 90%
F. Ha ou
e al. [28]2017 RGB
Accu acy Sensi i i y Speci ici y
KNN 91.94% 100% 86.00%
ANN 95.15% 100% 91.00%
NB 93.55% 100% 88.60%
MEWMA-SVM 96.66% 100% 94.93%
Senso s 2021,21, 947 18 o 50
Table 2. Con .
Re e ence Yea Inpu Signal ANN/Classi ie s and Pe o mance
Y. M. Gal ão
e al. [48]2017 RGB
F1 sco e
Mul ilaye pe cep on (MLP) 0.991
K-nea es neighbo s (KNN) 0.988
SVM—polynomial ke nel 0.988
Leila Panahi
e al. [60]2018 Dep h
A e age esul s
SVM
Sensi i i y 98.52%
Speci ici y 97.35%
Th eshold-based decision
Sensi i i y 98.52%
Speci ici y 97.35%
K. Sehai i
e al. [58]2018 RGB
Accu acy
SVM-RBF 99.27%
KNN 98.91%
ANN 99.61%
Chao Ma e al. [
70
]
2019 RGB + IR
Au oencode
Sensi i i y 93.3%
Speci ici y 92.8%
SVM
Sensi i i y 90.8%
Speci ici y 89.6%
F. Ha ou
e al. [74]2019 RGB
Accu acy:
K-NN 91.94%
ANN 95.16%
Naï e Bayes 93.55%
Decision ee 90.48%
SVM 96.66%
Senso s 2021,21, 947 19 o 50
Table 2. Con .
Re e ence Yea Inpu Signal ANN/Classi ie s and Pe o mance
Rica do Espinosa
e al. [79]2019 RGB
Sensi i i y Speci ici y
So max 97.95% 83.08%
SVM 14.10% 90.03%
RF 14.30% 91.26%
MLP 11.03% 93.65%
KNN 14.35% 90.96%
Xiangbo Kong
e al. [84]2019 Dep h
HOG+SVM LeNe AlexNe GoogLeNe ETDA-Ne
A e age accu acy 89.48% 88.28% 93.53% 96.59% 95.66%
A e age speci ici y
95.43% 97.18% 97.56% 98.76% 99.35%
A e age
sensi i i y 83.75% 74.54% 87.10% 88.74% 91.87%
B. Wang e al. [87]2020 RGB
F1 sco e
Falling s a e
GDBT 95.69%
DT 84.85%
RF 95.92%
SVM 96.1%
KNN 93.78%
MLP 97.41%
Fallen s a e
GDBT 95.27%
DT 95.45%
RF 96.8%
SVM 95.22%
KNN 94.22%
MLP 94.46%
Senso s 2021,21, 947 20 o 50
Table 2. Con .
Re e ence Yea Inpu Signal ANN/Classi ie s and Pe o mance
C. Zhong e al. [
89
]
2020 IR
F1 sco e
RBFNN 89.57 (+/−0.62)
SVM 88.74% (+/−1.75)
So max 87.37% (+/−1.4)
DT 88.9% (+/−0.68)
C. Menacho
e al. [88]2020 RGB
Accu acy
VGG-16 87.81%
VGG-19 88.66%
Incep ion V3 92.57%
ResNe 50 92.57%
Xcep ion 92.57%
ANN p oposed in his sys em 88.55%
G. Sun e al. [90]2020 RGB
Sensi i i y Speci ici y
SVM 92.50% 93.70%
KNN 93.80% 92.30%
SVDD 94.60% 93.80%
Yuxi Chen
e al. [105]2020 Dep h
A e age alues
Lene
Sensi i i y 82.78%
Speci ici y 98.07%
AlexNe
Sensi i i y 86.84%
Speci ici y 98.41%
GoogLeNe
Sensi i i y 92.87%
Speci ici y 99%
Senso s 2021,21, 947 21 o 50
Table 3. Sys em pe o mance e alua ion da ase s.
Signal Type Da ase Name Cha ac e is ics
Accele ome ic and
elec oencephalog am (EEG) and RGB
and passi e in a ed (IR)
Up all [80]17 olun ee s execu e alls and ac i i ies o daily li e (ADL) o di e en ypes eco ded by an
accele ome e , EEG, RGB and passi e IR sys ems
Dep h and Accele ome ic
Dep h and accele ome ic da ase [43] Volun ee s execu e se e al ac i i ies, and alls a e eco ded by a dep h sys em and accele ome e s.
TST all de ec ion [37]11 olun ee s execu e 4 all ypes and 4 ADLs eco ded by RGB-dep h (RGB-D) and accele ome e
sys ems
UR all de ec ion [29] 30 alls and 40 ADLs eco ded by RGB-D and accele ome e sys ems
RGB
Cen e o digi al home da a se —MMU [68] 20 ideos, including 31 alls and se e al ADLs
LE2I [23] 191 di e en ac i i ies, including ADLs and 143 alls
Cha i2012 da ase [25]
250 ideo sequences in ou di e en loca ions, 192 con aining alls, and 57 con aining ADLs.
Ac o s, unde di e en ligh condi ions, mo e in en i onmen s whe e occlusion exi s and clu e ed
and ex u ed backg ound is common
High-quali y da ase [53]
I is a all de ec ion da ase ha a emp s o app oach he quali y o a eal-li e all da ase . I has
ealis ic se ings and all scena ios. In de ail, 55 all scena ios and 17 no mal ac i i y scena ios we e
ilmed by i e web-came as in a oom simila o one in a nu sing home
Mul icam all da ase [10]
The ideo da a se is composed o se e al simula ed no mal daily ac i i ies and alls iewed om 8
di e en came as and pe o med by one subjec in 24 scena ios
Simple all de ec ion da ase [20]The da ase con ains 30 daily ac i i ies such as walking, si ing down, squa ing down, and 21 all
ac i i ies such as o wa d alls, backwa d alls and sideway alls
MO da ase [72]
MOT da ase in ends o be a amewo k o he ai e alua ion o mul iple people acking
algo i hms. In his amewo k, he designe s p o ide:
•De ec ions o all he sequences;
•A common e alua ion ool p o iding se e al measu es, om ecall o p ecision o unning
ime;
•An easy way o compa e he pe o mance o s a e-o - he-a acking me hods;
•
Se e al challenges wi h subse s o da a o speci ic asks such as 3D acking and su eillance.
COCO da ase [73]COCO is a la ge-scale objec de ec ion, segmen a ion, and cap ioning da ase designed o show
common objec s in con ex
Pi opo [96]
Mul iple ac i i ies eco ded in wo di e en scena ios wi h bo h con en ional and ish eye came as

Senso s 2021,21, 947 22 o 50
Table 3. Con .
Signal Type Da ase Name Cha ac e is ics
Dep h
IASLAB-RGB allen pe son da ase [35]I consis s o se e al s a ic and dynamic sequences wi h 15 di e en people and 2 di e en
en i onmen s
Mul imodal mul i iew da ase o human
ac i i ies [50]
I consis s o 2 da ase s eco ded simul aneously by 2 Kinec sys ems including ADLs and alls in a
li ing oom equipped wi h a bed, a cupboa d, a chai and su ounding o ice objec s illumina ed by
neon lamps on he ceiling o by sunligh
Sdu all [12] 10 olun ee s de elop 6 ac i i ies eco ded by RGB-D sys ems
Falling de ec ion [38] 6 olun ee s pe o m 26 alls and simila ac i i ies eco ded by RGB-D sys ems.
Fall de ec ion da ase [30] 5 olun ee s execu e 5 di e en ypes o all
NTU RGB+ da ase [99]
I is a la ge-scale da ase o human ac ion ecogni ion.
I con ains 56,880 ac ion samples and includes 4 di e en modali ies o da a o each sample: RGB
ideos, dep h map sequences, 3D skele al da a and IR ideos
Syn he ic Mo emen Da abases CMU G aphics Lab—mo ion cap u e
lib a y [55]Lib a y ha cap u es syn he ic mo emen s h ough mo emen cap u e (MoCap) echnology
Senso s 2021,21, 947 23 o 50
4. Discussion
The s udied sys ems illus a e isual-based all de ec ion e olu ion in he las i e
yea s. These sys ems ollow a pa allel pa h o o he human ac i i y ecogni ion sys ems,
wi h inc easingly in ense use o a i icial neu al ne wo ks (ANN) and a clea endency
owa ds cloud compu ing sys ems, excep o he ones moun ed on obo s.
All s udied sys ems ollow, wi h nuances, a h ee-s ep app oach o all de ec ion
h ough a i icial ision.
The i s s ep, in oduced in Sec ion 4.1 and no always needed, includes ideo signal
p ep ocessing in o de o op imize i as much as possible.
Cha ac e iza ion is he second s ep, s udied in Sec ion 4.2, whe e image ea u es a e
abs ac ed, so wha happens in he images can be exp essed in he o m o desc ip o s ha
will be classi ied in he las s ep o he p ocess.
The hi d p ocess s ep, explained in Sec ion 4.3, in ends o ag he obse ed ac ions,
which main ea u es a e cha ac e ized by abs ac desc ip o s, as a all e en o one which
is no , so measu es can be aken o help he allen pe son as as as possible.
Some o he s udied sys ems ollow a ame-by- ame app oach whe e he sole sys em
goal is classi ying human pose as allen o no , lea ing aside he all mo ion i sel . Fo hose
sys ems ying o de e mine i a speci ic mo emen may be a all, silhoue e acking is a
basic suppo ope a ion de eloped h ough di e en p ocesses. T acking echniques used
by he s udied sys ems a e explained in Sec ion 4.4.
Finally, a compa ison in classi ying algo i hm pe o mance and alida ion da ase s is
p esen ed in Sec ions 4.5 and 4.6.
4.1. P ep ocessing
The inal objec i e o his phase is ei he dis o ion and noise educ ion o o ma adap-
a ion, so downs eam sys em blocks can ex ac cha ac e is ic ea u es wi h classi ica ion
pu poses. Image complexi y educ ion could also be an objec i e du ing he p ep ocessing
phase in some sys ems, so he compu a ional cos can be educed, o ideo s eaming
bandwid h use can be diminished.
The echniques g ouped in his Sec ion o dec easing noise a e nume ous and
ange om Gaussian smoo hing used in [
31
] o he mo phological ope a ions execu ed
in
[17,31,74]
o [
24
]. They a e in oduced in subsequen Sec ion as a pa o he o eg ound
segmen a ion p ocess.
Fo ma adap a ion p ocesses a e p esen in se e al o he s udied sys ems, as is he
case in [
48
], whe e images a e con e ed o g ayscale and ha e hei his og ams equalized
be o e being ans e ed o he cha ac e iza ion p ocess.
Image bina iza ion, as in [
89
], is also in oduced as a pa o he sys ema ic e o o
educe noise du ing he segmen a ion p ocess, while some o he sys ems, like he one p e-
sen ed in [
56
], pu sue image complexi y dec easing by ans o ming ideo signals om ed,
g een and blue (RGB) o black and whi e and hen applying a median il e , an algo i hm
which assigns new alues o image pixels based on he median o he su ounding ones.
Image complexi y educ ion is a goal pu sued by some sys ems, as he one p oposed
in [
91
], which in oduces comp essed sensing (CS), an algo i hm i s p oposed by Donoho
e al. [
107
] used in signal p ocessing o acqui e and econs uc a signal. Th ough his
echnique, signals, spa se in some domain, a e sampled a a es much lowe han equi ed
by he Nyquis –Shannon sampling heo em. The sys em uses a h ee-laye ed app oach
o CS by applying i o ideo signals, which allows p i acy p ese a ion and bandwid h
use educ ion. This echnique, howe e , in oduces noise and o e -smoo hs edges, espe-
cially hose in low con as egions, leading o in o ma ion loss and image low- esolu ion.
The e o e, image complexi y educ ion ea u e cha ac e iza ion o en becomes a challenge.
4.2. Cha ac e iza ion
The second p ocess s ep in ends o exp ess human pose and/o human mo ion as
abs ac ea u es in a quali a i e app oach, o quan i y hei in ensi y in an ul e io quan i y
Senso s 2021,21, 947 24 o 50
app oach. These quan i ied ea u es a e hen used wi h classi ying pu poses in he las s ep
o he all de ec ion sys em.
These abs ac pose/ac ion desc ip o s can globally be classi ied in o h ee main
g oups: global, local and dep h.
Global desc ip o s analyze images as a block, segmen ing o eg ound om back-
g ound, ex ac ing desc ip o s ha de ine i and encoding hem as a whole.
Local desc ip o s app oach he abs ac ion p oblem om a di e en pe spec i e and, in-
s ead o segmen ing he block o in e es , p ocess he images as a collec ion o
local desc ip o s
.
Dep h cha ac e iza ion is an al e na i e way o de ine desc ip o s om images con-
aining dep h in o ma ion by ei he using dep h maps o skele on da a ex ac ed om a
join acking p ocess.
4.2.1. Global
Global desc ip o s y o ex ac abs ac in o ma ion om he o eg ound once i has
been segmen ed om he backg ound and encode i as a whole.
This kind o ac i i y desc ip o s was e y commonly used in a i icial ision ap-
p oaches o human ac i i y ecogni ion in gene al and o all de ec ion in pa icula . How-
e e , o e ime, hey ha e been displaced by local desc ip o s o used in combina ion wi h
hem, as hese ones a e less sensi i e o noise, occlusions and iewpoin changes.
Fo eg ound segmen a ion is execu ed in a numbe o di e en ways. Some app oaches
o his concep es ablish a speci ic backg ound and sub ac i om he o iginal image;
some o he s loca e egions o in e es by iden i ying he silhoue e edges o use he op ical
low, gene a ed as a consequence o body mo emen s, as a desc ip o . Some global cha ac-
e iza ion me hods segmen he human silhoue e o e ime o o m a space– ime olume
which cha ac e izes he mo emen . Some o he me hods ex ac ea u es om images in
a di ec way, as in he case o he sys em desc ibed in [
48
], whe e e e y h ee ames, he
mean squa e e o (MSE) is de e mined and used as an indica o o image simila i y.
Silhoue e Segmen a ion
Human shape segmen a ion can be execu ed h ough a numbe o echniques, bu
all o hem equi e backg ound iden i ica ion and sub ac ion. This p ocess, known as
backg ound ex ac ion, is p obably he mos isually in ui i e one, as i s p oduc is a
human silhoue e.
Backg ound es ima ion is he mos impo an s ep o he p ocess, and i is add essed
in di e en ways.
In [
17
,
24
,
56
,
74
], as he backg ound is supposed cons an , an image o i is aken
du ing sys em ini ializa ion, and a di ec compa ison allows segmen a ion o any new
objec p esen in he ideo. This echnique is easy and powe ul; howe e , i is ex emely
sensi i e o ligh changes. To mi iga e his law, he sys em desc ibed in [
31
], whe e he
backg ound is also supposed s able, a median h oughou ime is calcula ed o e e y pixel
posi ion in e e y colo channel. Then, i is di ec ly sub ac ed om he obse ed image
ame-by- ame.
Despi e e e y hing, he ob ained p oduc s ill con ains a subs an ial amoun o noise
associa ed wi h shadows and illumina ion. To educe i , mo phological ope a o s can be
used as in [
17
,
24
,
31
,
74
]. Dila ion and/o e osion ope a ions a e pe o med by p obing
he image a all possible places wi h a s uc u ing elemen . In he dila ion ope a ion, his
elemen wo ks as a local maximum il e and, he e o e, adds a laye o pixels o bo h inne
and ou e bounda y a eas. In e osion ope a ions, he elemen wo ks as a local minimum
il e and, as a consequence, s ips away a laye o pixels om bo h egions. Noise educ ion
a e segmen a ion can also be pe o med h ough Kalman il e ing, as in [
92
], whe e his
il e ing me hod is success ully used wi h his pu pose.
An al e na i e op ion o backg ound es ima ion and sub ac ion is he applica ion o
Gaussian mix u e models (GMM), a echnique used in [
7
,
11
,
14
,
78
,
92
], among o he s, ha
models he alues associa ed wi h speci ic pixels as a mix o Gaussian dis ibu ions.
Senso s 2021,21, 947 25 o 50
A di e en app oach is used in [
6
], whe e he Ho p ase me hod [
108
] is applied o
backg ound sub ac ion. I uses a compu a ional colo model ha sepa a es he b igh ness
om he ch oma ici y componen . By doing i , i is possible o segmen he o eg ound
much mo e e icien ly when ligh dis u bances a e p esen han wi h p e ious me hods,
diminishing his way ligh change sensi i eness. In his pa icula sys em, pixels a e also
clus e ed by simila i y, so compu a ional complexi y can be educed.
Some sys ems, like he one p esen ed in [
7
], apply a il e o de e mine silhoue e
con ou s. In his pa icula case, a Sobel il e is used, which de e mines a wo-dimensional
g adien o e e y image pixel.
O he segmen a ion me hods, like ibe [
19
], used in [
22
,
94
], s o e, associa ed wi h
speci ic pixels, p e ious alues o he pixel i sel and i s icini y o de e mine whe he i s
cu en alue should be ca ego ized as o eg ound o backg ound. Then, he backg ound
model is adap ed by andomly choosing which alues should be subs i u ed and which no ,
a clea ly di e en pe spec i e om o he echniques, which gi e p e e ence o new alues.
On op o ha , pixel alues decla ed as backg ound a e p opaga ed in o neighbo ing pixels
pa o he backg ound model.
The sys em in [
8
] segmen s he o eg ound using he echnique p oposed in [
109
],
whe e he op ical low (OF), which a e p esen ed in la e Sec ions, is calcula ed o de e mine
wha objec s a e in mo ion in he image, ea u e used o o eg ound segmen a ion. In
a subsequen s ep, o educe noise, images a e bina ized and mo phological ope a o s
a e applied. Finally, he poin s ma king he cen e o he head and he ee a e linked
by lines composing a iangle whose a ea/heigh a io will be used as he cha ac e is ic
classi ica ion ea u e.
Some algo i hms, like he illumina ion change- esis an independen componen
analysis (ICA), p oposed in [
95
], combine ea u es o di e en segmen a ion echniques,
like GMM and sel -o ganizing maps, a well-known g oup o ANN able o classi y in o
low dimensional classes e y high dimensional ec o s, o o e come he p oblems o
silhoue e segmen a ion associa ed wi h illumina ion phenomena. This algo i hm is able o
success ully ackle segmen a ion e o s associa ed wi h sudden illumina ion changes due
o any kind o ligh sou ce, bo h in images aken wi h omnidi ec ional diop ic came as
and in plain ones.
ICA and ibe a e compa ed in [
94
] by using a da ase speci ically de eloped o ha
sys em wi h be e esul s o he ICA algo i hm.
In [
9
], o eg ound ex ac ion is execu ed in acco dance wi h he p ocedu e desc ibed
in [
110
]. This me hod in eg a es he egion-based in o ma ion on colo and b igh ness in a
codewo d, and he collec ion o all codewo ds a e g ouped in an en i y called codebook.
Pixels a e hen checked in e e y single new ame and, when i s colo o b igh ness does
no ma ch he egion codewo d, which encodes a ea b igh ness and colo bands, i is
decla ed as o eg ound. O he wise, he codewo d is upda ed, and he pixel is decla ed
as a ea backg ound. Once pixels a e agged as o eg ound, hey a e clus e ed oge he ,
and codebooks a e upda ed o each one o hem. Finally, hese egions a e app oxima ed
by polygons.
Some sys ems, like he one in [
9
], use o hogonal came as and use o eg ound maps by
using homog aphy. This way, noise associa ed wi h illumina ion a ia ions and occlusion
is g ea ly educed. The sys em also calcula es he obse ed polygon a ea/g ound p ojec ed
polygon a ea a e as he main ea u e o de e mine whe he a all e en has aken place.
Sel -o ganizing maps is a echnique, well desc ibed in [
111
], used wi h segmen a ion
pu poses in [
58
]. When applied, ini ial backg ound es ima ion is made based on he i s
ame a sys em s a up. E e y pixel o his ini ial image is associa ed wi h a neu on in an
ANN h ough a weigh . Those weigh s a e cons an ly upda ed as new ames low in o he
sys em and, he e o e, he backg ound model changes. Sel -o ganizing maps ha e been
success ully used o sub ac o eg ound om backg ound, and hey ha e p o ed a good
esilience o he ligh a ia ion noise.
Senso s 2021,21, 947 32 o 50
pendulum o a ion ene gy and i s gene alized o ce sequences. These ea u es a e hen
codi ied in a ec o and used o classi ica ion pu poses.
The sys em in [
105
] uses se e al ANNs and selec s he mos sui able one as a unc ion
o he en i onmen and he cha ac e is ics o he acked people. In addi ion, i uploads
w ongly ca ego ized images which a e used o e ain he used models.
4.2.3. Dep h
Desc ip o s based on dep h in o ma ion ha e gained g ound hanks o he de elop-
men o low-cos dep h senso s, such as Mic oso Kinec
®
. This a o dable sys em coun s
wi h a so wa e de elopmen ki (SDK) and applica ions able o de ec and ack join s and
cons uc human body ec o models. These elemen s, oge he wi h he dep h in o ma-
ion om s e eoscopic scene obse a ion, ha e aised g ea in e es among he a i icial
ision esea ch communi y in gene al and he human all de ec ion sys em de elope s
in pa icula .
A good numbe o he s udied sys ems use dep h in o ma ion, solely o oge he wi h
RGB one, as he da a sou ce in he abs ac ion p ocess leading o image desc ip o cons uc-
ion. These sys ems ha e p o ed o be able o segmen o eg ound, g ea ly diminishing
in e e ence due o illumina ion in e e ences up o he dis ance whe e s e eoscopic ision
p ocedu es a e able o in e dep h da a. Fall de ec ion sys ems use his in o ma ion ei he
as dep h maps o skele on ec o models.
Dep h Map Rep esen a ion
Dep h maps, unlike RGB ideo signals, con ain di ec h ee-dimensional in o ma ion
on objec s in he image. The e o e, dep h map ideo signals in eg a e aw 3D in o ma ion,
so h ee-dimensional cha ac e iza ion ea u es can be di ec ly ex ac ed om hem.
This way, he sys em desc ibed in [
46
] iden i ies 16 egions o he human body ma ked
wi h ed ape and posi ion hem in space h ough s e eoscopic echniques. Taking ha
in o ma ion as a base, he sys em builds he body ec o (aligned wi h spine o ien a ion)
and iden i ies i s cen e o g a i y (CG). Accele a ion o CG and body ec o angle on a
e ical axis will be used as ea u es o classi ica ion.
Fo eg ound segmen a ion o human silhoue e is made by hese sys ems h ough
dep h in o ma ion, by compa ing dep h da a om images and a e e ence es ablished a
sys em s a up. This way, pixels appea ing in an image a a dis ance di e en om he
one s o ed o ha pa icula pixel in he e e ence a e decla ed as o eg ound. This is he
p ocess ollowed by [44] o segmen he human silhoue e. In an ul e io s ep, desc ip o s
based on bounding box, cen oid, a ea and o ien a ion o he silhoue e a e ex ac ed.
O he sys ems, like he one in [
101
], ex ac backg ound by using he same p ocess
and he silhoue e is de e mined as he majo connec ed body in he esul ing image. Then,
an ellipse is es ablished a ound i , and classi ica ion will be made as a unc ion o i s aspec
a io and cen oid posi ion. A simila p ocess is ollowed in [
60
], whe e, a e backg ound
sub ac ion, an ellipse is es ablished a ound he silhoue e, and i s cen oid ele a ion and
eloci y, as well as i s aspec a io, a e used as classi ica ion ea u es.
The sys em in [
57
] uses dep h maps o segmen silhoue es as well and c ea es a
bounding box a ound hem. Box op coo dina es a e used o de e mine he head eloci y
p o ile du ing a all e en , and i s Hausdo dis ance o head ajec o ies eco ded du ing
eal all e en s is used o de e mine whe he a all has aken place. The Hausdo dis ance
quan i ies how a wo subse s o a me ic space a e om each o he . The no el y o his
sys em, lea ing aside he in oduc ion o he Hausdo dis ance as desc ibed in [
130
], is
he use o a mo ing cap u e (MoCap) echnique o d i e a human model using so wa e
o simula e i s mo ion (OpenSim), so p o iles o head e ical eloci ies can be cap u ed
in ADLs, and a da abase can be buil . This da abase is used, by he in oduc ion o he
Hausdo dis ance, o assess alls.
The sys em in [
85
], a e o eg ound ex ac ion by using dep h in o ma ion as in he
p e ious sys ems, ans o ms he image o a black and whi e o ma and, a e de-noising i

Senso s 2021,21, 947 33 o 50
h ough il e ing, calcula es he HOG. To do i , he sys em de e mines he g adien ec o
and i s di ec ion o each image pixel. Then, a his og am is cons uc ed, which in eg a es
all pixels’ in o ma ion. This is he ea u e used o classi ica ion pu poses.
In [
42
], silhoue es a e acked by using a p opo ional-in eg al-di e en ial (PID) con-
olle . A bounding box is c ea ed a ound he silhoue e, and ea u es a e ex ac ed in
acco dance wi h [
131
]. A all will be called i h esholds es ablished o ea u es a e ex-
ceeded. Faces a e sea ched, and when iden i ied, he acking will be biased
owa ds hem.
Some o he sys ems, like he one in [
15
], sub ac s backg ound by di ec use o dep h
in o ma ion con ained in sequen ial images, so he di e ence be ween consecu i e dep h
ames is used o segmen a ion. Then, he head is acked, so he head e ical posi-
ion/pe son heigh a io can be de e mined, which, oge he wi h CG eloci y, is used as a
classi ica ion ea u e.
In [
54
], all backg ound is se o a ixed dep h dis ance. Then, a g oup o 2000 body
pixels is andomly chosen, and o each o hem, a ec o o 2000 alues, calcula ed as
a unc ion o he dep h di e ence be ween pai s o poin s, is c ea ed. These pai s a e
de e mined by es ablishing 2000 pixel o se se s. The ob ained 2000- alue ec o is used as
a cha ac e is ic ea u e o pose classi ica ion.
The sys em in oduced in [
11
], a e he human silhoue e is segmen ed by using
dep h in o ma ion h ough a GMM p ocess, calcula es i s cu a u e scale space (CSS)
ea u es by using he p ocedu es desc ibed in [
12
]. CSS calcula ion me hod con olu es a
pa ame ic ep esen a ion o a plana cu e, silhoue e edge in his case, wi h a Gaussian
unc ion. This way, a ep esen a ion o he a c leng h s. cu a u e is ob ained. Then,
silhoue es ea u es a e encoded, oge he wi h he Gaussian mix u e model used in he
a o emen ioned CSS p ocess, in a single Fishe ec o , which will be used, a e being
no malized, o classi ica ion pu poses.
Finally, a block o sys ems c ea es olumes based on no mal dis ibu ions cons uc ed
a ound poin clouds. These dis ibu ions, called oxels, a e g ouped oge he , and desc ip-
o s a e ex ac ed ou o oxel clus e s o de e mine, i s , whe he hey ep esen a human
body and hen o assess i i is in a allen s a e.
This way, he sys em p esen ed in [
27
] i s es ima es he g ound plane by assuming
ha mos o he pixels belonging o e e y ho izon al line a e pa o he g ound plane.
The g ound can hen be es ima ed, line pe line, a ending o he pixel dep h alues as
explained in he p ocedu e desc ibed in [
132
]. To clean up he pic u es, all pixels below he
g ound plane a e disca ded. Then, no mal dis ibu ions ans o m (NDT) maps a e c ea ed
as a cloud o poin s su ounded by no mal dis ibu ions wi h he physical appea ance o
an ellipsoid. These dis ibu ions, c ea ed a ound a minimum numbe o poin s, a e called
oxels and, in his sys em, a e gi en ixed dimensions. Then, ea u es ha desc ibe he local
cu a u e and shape o he local neighbo hood a e ex ac ed om he dis ibu ions. These
ea u es, known as IRON [
133
], allow oxel classi ica ion as being pa o a human body
o no and, his way, oxels agged as human a e clus e ed oge he . IRON ea u es a e
hen calcula ed o he clus e ep esen ing a human body, and he Mahalanobis dis ance
be ween ha ec o and he dis ibu ion associa ed wi h allen bodies is calcula ed. I he
dis ance is below a h eshold, he all s a e is decla ed.
A simila p ocess is used in [
34
], whe e, a e he poin cloud is unca ed by emo ing
all poin s no con ained in he a ea in be ween he g ound plane and a pa allel one 0.7 m
o e i by applying he RANSAC p ocedu e [
134
], NDTs a e c ea ed and hen segmen ed
in pa ches o equal dimensions. A suppo ec o machine (SVM) classi ie de e mines
which ones o hose pa ches belong o a human body as a unc ion o hei geome ic
cha ac e is ics. Close pa ches agged as humans a e clus e ed, and a bounding box is
c ea ed a ound. A second SVM de e mines whe he clus e s should be decla ed as a allen
pe son. This classi ica ion is e ined, aking da a om a da abase o obs acles o he a ea,
so i he clus e is decla ed as a allen pe son, bu i is con ained in he obs acle da abase,
he decla a ion is skipped.
Senso s 2021,21, 947 34 o 50
Skele on Rep esen a ion
Sys ems implemen ing his ep esen a ion a e able o de ec and ack join s and, based
on ha in o ma ion, hey can build a human body ec o model. This block o echniques,
as he p e ious one, s ongly diminishes he noise associa ed wi h illumina ion bu ha e
p oblems o build a co ec model when occlusion appea s, bo h he one gene a ed by
obs acles and he one p oduc o pe spec i e au o-occlusions.
A good numbe o hese sys ems a e buil o e he Mic oso Kinec
®
sys em and
ake ad an age o bo h de SDK and he applica ions de eloped o i . This is he case
o he sys em in oduced in [
40
], whe e h ee Kinec
®
sys ems co e he same a ea om
di e en pe spec i es, and join s a e, he e o e, ollowed om di e en angles, educing
his way he acking p oblems associa ed wi h occlusion. In his sys em, human mo emen
is cha ac e ized h ough wo main ea u es, head speed and CG si ua ion e e enced o
ankles posi ion.
The Kinec
®
sys em is also used in [
65
] o ollow join s and es ima e he e ical
dis ance o he g ound plane. Then, he angle be ween he e ical and he o so ec o ,
which links he neck and spine base, is de e mined and used o iden i y a s a key ame
(SKF), whe e a all s a s, and an end key ame (EKF), whe e i ends. Du ing his pe iod,
e ical dis ance o he g ound plane and e ical eloci y o ollowed uppe join s will be
he inpu o classi ica ion. A e y simila app oach is ollowed in [
33
], whe e o so/ e ical
angle and cen oid heigh a e he key ea u es used o classi ica ion.
This sys em is used as well in [
5
] o build, a ound iden i ied join s, bo h 2D and
3D bounding boxes aligned wi h he spine di ec ion. Then, he a io wid h/heigh is
de e mined, and he ela ion H
CG
/P
CG
, being he o me de ele a ion o he CG o e he
g ound plane and he la e de dis ance be ween he CG p ojec ion on he g ound and he
suppo polygon de ined by ankles posi ion, is calcula ed. Those ea u es will be he base
o e en classi ica ion.
In [
135
], human body key poin s a e iden i ied by a CNN whose inpu is a 2D RGB
ideo signal complemen ed by dep h in o ma ion. Based on hose key poin s, he sys em
builds a human body ec o model. A il e was de eloped o gene a e digi al e ain
models om da a cap u ed by ai bo ne sys ems [
136
], and he dep h da a we e hen used
o es ima e he g ound plane. The sys em uses all ha in o ma ion o calcula e he dis ance
om he body CG and he body egion o e he shoulde s o he g ound. These dis ances
will se e o cha ac e ize he human pose.
A CNN is also used in [
61
] o gene a e ea u e maps ou o he dep h images. This
ne wo k s acks con olu ion laye s o ex ac ea u es and pooling laye s o educe map
complexi y, wi h a philosophy iden ical o he one used in he RGB local cha ac e iza ion.
The ou pu map goes h ough wo laye s o ully connec ed laye s o classi y he eco ded
ac i i y, and a So max unc ion is implemen ed in he las laye o he ANN, which
de e mines whe he a all has aken place.
In [84], p io o inpu images in a CNN o gene a e ea u e maps, which will be used
o classi ica ion, he backg ound is sub ac ed h ough an algo i hm ha combines dep h
maps and 2D images o enhance segmen a ion pe o mance. This way, i he pixels o he
segmen ed 2D silhoue e expe imen sha p changes, bu pixels in he dep h map do no ,
pixels subjec o hose changes a e ega ded as noise. The sys em mixes in o ma ion om
bo h sou ces, allowing a be e ack on segmen ed silhoue es and a quick ack egain in
case i is los .
The sys em in [
13
]—a e iden i ying human body join s as he key ea u es whose
ajec o y will be used o de e mine whe he a alling e en has aken place—p oposes
o a ing he o so so i is always e ical. This way, join ex ac ion becomes pose in a ian , a
echnique used in he sys em wi h posi i e esul s in o de o deal wi h he noise associa ed
wi h join iden i ica ion as a esul o apid mo emen and occlusion, cha ac e is ic o alls.
Senso s 2021,21, 947 35 o 50
4.3. Classi ica ion
Once pose/mo emen abs ac desc ip o s ha e been ex ac ed om ideo images,
he nex s ep o he all de ec ion p ocess is classi ica ion. In b oad e ms, du ing his phase,
he sys em classi ies mo emen and o pose as a all o a allen s a e h ough an algo i hm
ha is pa o one o hese wo ca ego ies; gene a i e o disc imina i e models.
Disc imina i e models a e able o de e mine bounda ies be ween classes, ei he by
explici ly being gi en hose bounda ies o by se ing hem hemsel es using se s o p e-
classi ied desc ip o s.
Gene a i e models app oach he classi ica ion p oblem in a o ally di e en way, as
hey explici ly model he dis ibu ion o each class and hen use he Bayes heo em o link
desc ip o s o he mos likely class, which, in his case, can only be a all o a no all s a e.
4.3.1. Disc imina i e Models
The inal goal o any classi ie is assigning a class o a gi en se o desc ip o s. The
disc imina i e models a e able o es ablish he bounda ies sepa a ing classes, so he p oba-
bili y o a desc ip o belonging o a speci ic class can be gi en. In o he e ms, gi en
α
as a
class, and [A] as he ma ix o desc ip o alues associa ed wi h a pose o mo emen , his
amily o classi ie s is able o de e mine he p obabili y P(α|[A]).
Fea u e-Th eshold-Based
Fea u e- h eshold-based classi ica ion models a e b oadly used in he s udied sys ems.
This app oach is easy and in ui i e, as he esea che es ablishes h eshold alues o he
desc ip o s, so hei associa ed e en s can be assigned o a speci ic class in case hose
h esholds a e exceeded.
This is he case o he sys em p oposed in [
31
]. I classi ies he ac ion as a all o a
non- all in acco dance wi h a double a ionale. On one hand, i es ablishes h esholds
o ellipse ea u es o es ima e whe he he pose i s a allen s a e; on he o he , an MHI
ea u e exceeding a ce ain alue indica es a as mo emen and, he e o e, a po en ial all.
The sys em p oposed in [
14
] adds accele a ion o he o me ea u es and, in [
40
], head
speed o e a ce ain h eshold and CG posi ion ou o he segmen de ined by ankles a e
indica i es o a all.
Simila app oaches, whe e h eshold alues a e de e mined by sys em de elope s
based on p e ious expe imen a ion, a e implemen ed in a good numbe o he s udied
sys ems, as hey a e simple, in ui i e and compu a ionally inexpensi e.
Mul i a ia e Exponen ially Weigh ed Mo ing A e age
Mul i a ia e exponen ially weigh ed mo ing a e age (MEWMA) is a s a is ical p ocess
con ol o moni o a iables ha use he en i e his o y o alues o a se o a iables. This
echnique allows designe s o gi e a weigh ing alue o all eco ded a iable ou pu s, so
he mos ecen ones a e gi en highe weigh alues, and he olde ones a e weigh ed
ligh e . This way, he las alue is weigh ed
λ
(being
λ
a numbe be ween 0 and 1) and
p e ious
β
alues a e weigh ed
λβ
. Limi s o he alue o ha weigh ed ou pu a e
es ablished, aking as a basis he expec ed mean and s anda d de ia ion o he p ocess.
Ce ain sys ems, like [
28
], use his echnique o classi ica ion pu poses. Howe e , as i is
unable o dis inguish be ween alling e en s and o he simila ones, e en s agged as all
by he MEWMA classi ie need o go h ough an ul e io suppo ec o machine classi ie .
Suppo Vec o Machines
Suppo ec o machines (SVM) a e a se o supe ised lea ning algo i hms i s
in oduced by Vapnik e al. [137].
SVMs a e used o eg ession and classi ica ion p oblems. They c ea e hype planes in
high dimension spaces ha sepa a e classes nonlinea ly. To ul ill his ask, SVMs, simila
o a i icial neu al ne wo ks, use ke nel unc ions o di e en ypes.
A s anda d SVM bounda y de ini ion is shown in Figu e 4.
Senso s 2021,21, 947 36 o 50
Senso s 2021, 21, x FOR PEER REVIEW 36 o 50
Figu e 4. Suppo ec o machine bounda y de ini ion.
In [74], linea , polynomial, and adial ke nels a e used o ob ain he hype planes; in
[67], adial ones a e implemen ed, and in [48], polynomial ke nels a e used o achie e
nonlinea classi ica ions.
The suppo ec o da a desc ip ion (SVDD), in oduced by Tax e al. [138], is a clas-
si ying algo i hm inspi ed by he suppo ec o machine classi ie , able o ob ain a sphe -
ically shaped bounda y a ound a da ase and, analogously o SVMs, i can use di e en
ke nel unc ions. The me hod is made obus agains ou lie s in he aining se and is
capable o igh ening classi ica ion by using nega i e examples. SVDDs classi ying algo-
i hms a e used in [90].
SVMs ha e been e y used in he s udied sys ems as hey ha e p oo ed o be e y
e ec i e; howe e , hey equi e high compu a ional loads, some hing inapp op ia e o
edge compu ing sys ems.
K-Nea es Neighbo
K-nea es neighbo (KNN) is an algo i hm able o model he condi ional p obabili y
o a sample belonging o a speci ic class. I is used o classi ica ion pu poses in
[16,17,48,74] among o he s.
KNNs assume ha classi ica ion can be success ully made based on he class o he
nea es neighbo s. This way, i o a speci ic ea u e, all µ closes sample neighbo s a e
pa o a de e mined class, he p obabili y o he sample being pa o ha class will be
assessed as e y high. This s udy is epea ed o e e y ea u e con ained in he desc ip o ,
so a inal assessmen based on all ea u es can be made. The algo i hm usually gi es di -
e en weigh s o he neighbo s, and hea ie weigh s a e assigned o he closes ones. On
op o ha , i also assigns di e en weigh s o e e y ea u e. This way, he ones assessed
as mos ele an ge hea ie weigh s.
Decision T ee
Decision ees (DT) a e algo i hms used bo h in eg ession and classi ica ion. I is an
in ui i e ool o make decisions and explici ly ep esen s decision-making. Classi ica ion
DTs use ca ego ical a iables associa ed wi h classes. T ees a e buil by using lea es,
which ep esen class labels, and b anches, which ep esen cha ac e is ic ea u es o hose
Figu e 4. Suppo ec o machine bounda y de ini ion.
In [
74
], linea , polynomial, and adial ke nels a e used o ob ain he hype planes;
in [
67
], adial ones a e implemen ed, and in [
48
], polynomial ke nels a e used o achie e
nonlinea classi ica ions.
The suppo ec o da a desc ip ion (SVDD), in oduced by Tax e al. [
138
], is a
classi ying algo i hm inspi ed by he suppo ec o machine classi ie , able o ob ain
a sphe ically shaped bounda y a ound a da ase and, analogously o SVMs, i can use
di e en ke nel unc ions. The me hod is made obus agains ou lie s in he aining se
and is capable o igh ening classi ica ion by using nega i e examples. SVDDs classi ying
algo i hms a e used in [90].
SVMs ha e been e y used in he s udied sys ems as hey ha e p oo ed o be e y
e ec i e; howe e , hey equi e high compu a ional loads, some hing inapp op ia e o
edge compu ing sys ems.
K-Nea es Neighbo
K-nea es neighbo (KNN) is an algo i hm able o model he condi ional p obabili y o
a sample belonging o a speci ic class. I is used o classi ica ion pu poses in [
16
,
17
,
48
,
74
]
among o he s.
KNNs assume ha classi ica ion can be success ully made based on he class o he
nea es neighbo s. This way, i o a speci ic ea u e, all
µ
closes sample neighbo s a e pa
o a de e mined class, he p obabili y o he sample being pa o ha class will be assessed
as e y high. This s udy is epea ed o e e y ea u e con ained in he desc ip o , so a
inal assessmen based on all ea u es can be made. The algo i hm usually gi es di e en
weigh s o he neighbo s, and hea ie weigh s a e assigned o he closes ones. On op o
ha , i also assigns di e en weigh s o e e y ea u e. This way, he ones assessed as mos
ele an ge hea ie weigh s.
Decision T ee
Decision ees (DT) a e algo i hms used bo h in eg ession and classi ica ion. I is an
in ui i e ool o make decisions and explici ly ep esen s decision-making. Classi ica ion
DTs use ca ego ical a iables associa ed wi h classes. T ees a e buil by using lea es,
which ep esen class labels, and b anches, which ep esen cha ac e is ic ea u es o hose
classes. DTs buil p ocess is i e a i e, wi h a selec ion o ea u es co ec ly o de ed o
de e mine he spli poin s ha minimize a cos unc ion ha measu es he compu a ional
Senso s 2021,21, 947 37 o 50
equi emen s o he algo i hm. These algo i hms a e p one o o e i ing, as se ing he
co ec numbe o b anches pe lea is usually e y challenging. To educe he complexi y
o he ees, and he e o e, hei compu a ional cos , b anches a e p uned when he ela ion
cos -sa ing/accu acy loss is sa is ac o y. This ype o classi ie is used in [87,89].
Random o es (RF), like he one used in [
54
,
87
], is an agg ega ion echnique o DT,
in oduced by B aiman [
139
], which main objec i e is a oiding o e i ing. To accomplish
his ask, he aining da ase is di ided in o subg oups, and he e o e, a inal numbe o
DTs, equal o he numbe o da ase subg oups, is ob ained. All o hem a e used in he
p ocess, so he inal classi ica ion decision is ac ually a combina ion o he classi ica ion o
all DTs.
G adien boos ing decision ees (GBDT) is ano he DT agg ega ion echnique whose
algo i hm was i s in oduced by F iedman [
140
] whe e simple DTs a e buil and, o each
one o hem, a classi ica ion e o in aining ime is de e mined. An e o unc ion based
on calcula ed indi idual e o s is de e mined, and i s g adien is minimized by combining
indi idual DT classi ica ions in a p ope way. This agg ega ion echnique, speci ically
de eloped o DTs, is ac ually pa o a b oade amily ha will be mo e ex ensi ely
p esen ed in he nex sec ion.
Bo h echniques, RF and GBDT, a e used in [87].
Boos Classi ie
Boos classi ie algo i hms a e a amily o classi ie building echniques ha c ea e
s ong classi ie s by g ouping weak ones. I is done by adding up models c ea ed om he
aining da a un il he sys em is pe ec ly p edic ed o a maximum numbe o models is
eached.
This is done by building a model om he aining da a. Then, a second model is
c ea ed o co ec he e o s om he i s one. Models a e added un il he aining se is
well p edic ed o a maximum numbe o hem is added. Du ing he boos ing p ocess, he
i s model is ained on he en i e da abase while he es a e i ed o he esiduals o he
p e ious ones.
Adaboos , used in [
23
], can be u ilized o inc ease pe o mances wi h any classi ica ion
echnique, bu i is mos commonly used wi h one-le el decision ees.
In [
64
], boos ing echniques a e used on a J48 algo i hm, a ee-based echnique, simila
o andom o es , which is used o c ea e uni a ia e decision ees.
Spa se Rep esen a ion Classi ie
Spa se ep esen a ions classi ica ion (SRC) is a echnique used o image classi ica ion
wi h a e y good deg ee o pe o mance.
Na u al images a e usually ich in ex u e and o he s uc u es ha end o be ecu en .
Fo his eason, spa se ep esen a ion can be success ully applied o image p ocessing. This
phenomenon is known as pa ch ecu ence and, because o i , eal-wo ld digi al images
can be ecognized by p ope ly ained dic iona ies.
SRCs a e able o ecognize hose pa ches, as hey can be exp essed as a linea combi-
na ion o a limi ed numbe o elemen s ha a e con ained in he classi ie dic iona ies.
This is he case o he SRC p esen ed in [24].
Logis ic Reg ession
Logis ic eg ession is a s a is ical model used o classi ica ion. I is able o implemen
a bina y classi ie , like he one needed o decide whe he a all e en has aken place. Fo
such a pu pose, a logis ic unc ion is used. I can be adjus ed by using classi ying ea u es
associa ed wi h e en s agged as all o no all.
This me hod is used in sys ems like [
93
], whe e a logis ic classi ying algo i hm is
employed o classi y e en s as all o no a all, based on a ec o ha encodes he empo al
se ies o o a ion ene gy and gene alized o ce.

Senso s 2021,21, 947 38 o 50
Some a i icial neu al ne wo ks implemen a logis ic eg ession unc ion o classi i-
ca ion, like he one desc ibed in [
106
], whe e a CNN uses his unc ion o de e mine he
de ec ion p obabili y o each de ined class.
Deep Lea ning Models
In [
83
], he las laye s o he ANN implemen a So max unc ion, a gene aliza ion
o he logis ic unc ion used o mul inomial logis ic eg ession. This unc ion is used as
he ac i a ion unc ion o he nodes o he las laye o a neu al ne wo k, so i s ou pu is
no malized o a p obabili y dis ibu ion o e he di e en ou pu classes. So max is also
implemen ed in he las laye s o he a i icial neu al ne wo ks used in [
75
,
103
], among
o he s udied sys ems.
Mul ilaye pe cep on (MLP) is a ype o mul ilaye ed ANN wi h hidden laye s
be ween he en ance and he exi ones able o so ou classes non linea ly sepa able. Each
node o his ne wo k is a neu on ha uses a nonlinea ac i a ion unc ion, and i is used o
classi ica ion pu poses in [48,87].
Radial basis unc ion neu al ne wo ks (RBFNN) a e used in he las laye o [
89
] o
classi y he ea u e ec o s coming om p e ious CNN laye s. This ANN is cha ac e ized
by using adial basis unc ions as ac i a ion unc ions and yields be e gene aliza ion
capabili ies han o he a chi ec u es, such as So max, as i is ained ia minimizing he
gene alized e o es ima ed by a localized-gene aliza ion e o model (L-GEM).
O en, he las laye s o ANN a chi ec u es a e ully connec ed ones, as in [
58
,
76
,
86
],
whe e all nodes o a laye a e connec ed o all nodes in he nex one. In hese s uc u es,
he inpu laye is used o la en ou pu s om p e ious laye s and ans o m hem in o a
single ec o , while subsequen laye s apply weigh s o de e mine a p ope agging and,
he e o e, success ully classi y e en s.
Finally, ano he ANN s uc u e use ul o classi ica ion is he au oencode one, used
in [
70
]. Au oencode s a e ANNs ained o gene a e ou pu s equal o inpu s. I s in e nal
s uc u e includes a hidden laye whe e all neu ons a e connec ed o e e y inpu and ou pu
node. This way, au oencode s ge high dimensional ec o s and encode hei ea u es.
Then, hese ea u es a e decoded back. As he numbe o dimensions o he ou pu ec o
may be educed, his kind o ANNs can be used o classi ica ion pu poses by educing he
numbe o ou pu dimensions o he numbe o inal expec ed classes.
4.3.2. Gene a i e Models
The app oach o gene a i e models o he classi ica ion p oblem is comple ely di e en
om he one ollowed by he disc imina i e ones.
Gene a i e models explici ly model he dis ibu ion o each class. This way, gi en
α
as a class, and [A] as he ma ix o desc ip o alues associa ed wi h a pose o mo emen ,
i bo h P ([A]|
α
) and P (
α
) can be de e mined, i will be possible, by di ec applica ion o
he Bayes heo em, o ob ain P (α|[A]), which will sol e he classi ica ion p oblem.
Hidden Ma ko Model
Classi ica ion using he hidden Ma ko model (HMM) algo i hm is one o he h ee
ypical p oblems ha can be sol ed h ough his p ocedu e. I was i s p oposed wi h
his pu pose by Rabine e al. [
141
] o sol e he speech ecogni ion p oblem, and i is used
in [100] o classi y he ea u e ec o s associa ed wi h a silhoue e.
HMMs a e s ochas ic models used o ep esen sys ems whose s a e a iables change
andomly o e ime. Unlike o he s a is ical p ocedu es, like Ma ko chains, which deal
wi h ully obse able sys ems, HMMs ackle pa ially obse able sys ems. This way, he
inal objec i e o he HMM classi ying p oblem esolu ion will be decided, on he basis o
he obse able da a ( ea u e ec o ), whe he a all has occu ed (hidden sys em s a e).
The sys em p oposed in [
100
] de e mines, using an HMM as a classi ie , on he basis
o silhoue e su ace, cen oid posi ion and bounding box aspec a io, whe he a all akes
place o no . To do i , and o ake as a e e ence eco ded alls, a p obabili y is assigned o
Senso s 2021,21, 947 39 o 50
he wo possible sys em s a es ( all/no all) based on alue and a ia ion along he e en
ime ame pe iod o he ea u e ec o . This classi ying echnique is used wi h success in
his sys em, hough in [
142
], a b ie summa y o he nume ous limi a ions o his basic
HMM app oach is p esen ed, and se e al mo e e icien ex ensions o he algo i hm, such
as a iable ansi ion HMM o he hidden semi-Ma ko model, a e in oduced. These
algo i hm a ia ions a e de eloped as he basic HMM p ocess is conside ed ill-sui ed o
modeling sys ems whe e in e ac ing elemen s a e ep esen ed h ough a ec o o single
s a e a iables.
A simila classi ica ion app oach using an HMM classi ie is used in [
47
], whe e u u e
s a es p edic ed by an au o eg essi e-mo ing-a e age (ARMA) algo i hm a e classi ied as
all o no - all e en s. ARMA models a e able o p edic u u e s a es o a sys em based on a
p e ious ime-se ies. The model in eg a es wo modules, an au o eg essi e one, which uses
a linea combina ion o weigh ed p e ious sys em s a e alues, and a mo ing a e age one,
which linea ly combines weigh ed p e ious e o s be ween sys em s a e eal alues and
p edic ed ones. In he model, e o s a e assumed o be andom alues ha i a Gaussian
dis ibu ion o mean 0 and a iance σ2.
4.4. T acking
A good numbe o he e iewed sys ems iden i y objec s h ough ANN o ex ac
silhoue es om he backg ound. Then, ele an ea u es a e associa ed wi h he al eady
segmen ed objec s. This assignmen equi es a cons an upda e, and, he e o e, objec
co ela ion needs o be es ablished om ame- o- ame. This co ela ion is made h ough
objec acking, and a good numbe o di e en echniques a e used o such a pu pose.
4.4.1. Mo ing A e age Fil e
The double mo ing a e age il e used in [
65
] smoo hs e ical dis ance om join s o
he g ound plane. This il e de e mines wice he mean alue o he las n samples, ac ing
his way as a low pass il e , elimina ing high- equency signal componen s associa ed
wi h noise.
4.4.2. PID Fil e
The sys em p oposed in [
42
] uses a p opo ional-in eg al-di e en ial (PID) il e o
main ain acking on silhoue es segmen ed om he backg ound. Cons an s o he il e
o gua an ee smoo h acking, educing o e shoo s and s eady-s a e e o s, a e calcula ed
h ough a gene ic algo i hm. This algo i hm, inspi ed by he heo y o na u al e olu ion,
is a heu is ic sea ch whe e se s o alues a e selec ed o disca ded based on i s abili y o
educe o a minimum he absolu e e o unc ion and, he e o e, minimize o e shoo s and
s eady e o s.
4.4.3. Kalman Fil e
Kalman il e , i s in oduced by R. E. Kalman in [
143
], is a ecu si e algo i hm
ha allows imp o emen s in he de e mina ion o sys em a iable alues by combining
se e al se s o indi ec sys em a iable obse a ions con aining inaccu acies. The esul ing
es ima ion is mo e p ecise han any o he ones which could be in e ed om a single
indi ec obse a ion se .
This way, in [
40
], he acking o join s, ollowed by h ee independen Kinec
®
sys ems,
is used by a Kalman il e . The esul ing join posi ion is es ima ed by in eg a ing in o ma-
ion om he h ee sys ems and is mo e accu a e han one o any o he
indi idual sys ems.
A pa icula a ia ion in he use o Kalman il e ing is he one in [
97
], whe e a p oce-
du e call deep-so , p esen ed in [
129
], is used. In his p ocess, a Kalman algo i hm is used
o es ima e he nex loca ion o he acked pe son, and hen he Mahalanobis dis ance is
calcula ed be ween he de ec ed pe son in he ollowing ame and i s es ima ed posi ion.
By measu ing his dis ance, unce ain y in he ack co ela ion can be quan i ied. This
il e pe o mance is deeply a ec ed by occlusion. To mi iga e his p oblem, he unce ain y
Senso s 2021,21, 947 40 o 50
alue is associa ed wi h he ack desc ip o and, o keep acks a e long occlusion pe iods,
he p ocess sa es hose desc ip o s o 100 ames.
Al hough his il e ing algo i hm wo ks e y well o main ain acks in linea sys ems,
human bodies in ol ed in a all end o beha e nonlinea ly, subs an ially deg ading i s
abili y o main ain acking.
4.4.4. Pa icle Fil e
This me hod, used in [
15
], is a Mon e Ca lo algo i hm used o objec acking in ideo
signals. In oduced in 1993 by Go don [
144
] as a Bayesian ecu si e il e , i is able o
de e mine u u e sys em s a es, in his case, u u e posi ions o he acked objec .
The il e algo i hm ollows an i e a i e app oach. This way, a e a cloud o pa icles,
image pixels, in his case, ha e been selec ed, weigh s a e assigned o hem. Those weigh
alues a e a unc ion o he p obabili y o being pa o he acked objec . Then, he
ini ial pa icle cloud is upda ed by using he weigh alues. Based on objec cinema ic,
i s mo emen is p opaga ed o he pa icle cloud, p edic ing, his way, he u u e objec
si ua ion. The p ocess con inues wi h a new upda e phase o gua an ee he p edic ed cloud
ma ches he acked objec .
This algo i hm, al hough a ec ed by occlusion, has p o en o be highly capable o
main aining acks on objec s mo ing nonlinea ly and, he e o e, he esul is adequa e o
ack human bodies du ing all e en s.
Rao–Blackwellized pa icle il e (RBPF), like he one used in [
63
], is a ype o pa icle
il e acking algo i hm used in linea /nonlinea scena ios whe e a pu ely Gaussian
app oach is inadequa e.
This algo i hm di ides pa icles in o wo se s. Those which can be analy ically e alu-
a ed and hose which canno . This way, he il e ing equa ions a e sepa a ed in o wo se s,
so wo di e en app oaches can be used o calcula e hem. The i s se , which includes
linea mo ing pa icles, is sol ed by using a Kalman il e app oach, while he second one,
whose pa icles mo e nonlinea ly, is sol ed by employing a Mon e Ca lo
sampling me hod.
4.4.5. Fused Images
In [
9
], a using cen e uses images aken om o hogonal iews, and he ob ained
objec is agged wi h a numbe . Objec s iden i ied in he nex ame a e co ela ed o
p e ious ones i hey mee he minimum dis ance es ablished h eshold. This way, he
acking is main ained.
4.4.6. Camshi
This algo i hm, in eg a ed in o OpenCV and used in [
59
], i s con e s images RGB o
hue-sa u a ion- alue (HSV) and, s a ing wi h ames whe e a CNN has c ea ed a bounding
box (BB) a ound a de ec ed pe son, i de e mines he hue his og am in each BB. Then,
mo phological ope a ions a e applied o educe noise associa ed wi h illumina ion. In he
consecu i e ame, he a ea which be e i s he eco ded Hue his og am is es ablished and
compa ed wi h de ec ed BBs. Tha way, a co ela ion can be es ablished and, he e o e, a
ack on a pe son.
4.4.7. Deep Lea ning A chi ec u es
DeepSORT is a CNN used o ack mul iple objec s a he same ime, as shown in [
87
].
The sys em p esen ed in [
71
] acks images using an algo i hm as ollows: Fi s , in
e e y new ame, a YoLO con olu ional a chi ec u e is used o iden i y people. Once all
people in he ame ha e been iden i ied, a Siamese CNN is used o i s de e mine he
cha ac e is ic ea u es o e e y pe son iden i ied in he ame and hen compa e hem
wi h he ones associa ed wi h people iden i ied in p e ious ames, looking o simila i ies.
A he same ime, an LSTM ANN is used o p edic people’s mo ion, so associa ions o
main ain ack o people om ame- o- ame can be made. Based on ea u e simila i y and
mo emen associa ion, a ack can be es ablished on people p esen in consecu i e ideo
Senso s 2021,21, 947 41 o 50
ames o can be s a ed when a new pe son appea s o he i s ime in a ideo sequence.
An almos equal p ocess is used in [
97
] o keep ack o people wi h wo CNNs wo king in
pa allel, a i s one o iden i y people and a second one o ex ac cha ac e is ic ea u es ou
o hem. Tha way, acks can be es ablished.
In [
41
], a CNN is used o de ec people in e e y ame. A BB is es ablished a ound,
and dis ances om cen al poin BBs o consecu i e ames a e de e mined. Boxes mee ing
minimum dis ance c i e ia in consecu i e ames a e co ela ed and, his way, acking
is es ablished.
4.5. Classi ying Algo i hms Pe o mances
A numbe o he e iewed sys ems es ablish compa isons wi h o he ones. Many
o hem base ha compa ison on pe o mance igu es ob ained on di e en da ase s,
while some o he s es ablish a sys em- o-sys em compa ison based on he same da abase.
Howe e , sys ems a e, in b oad e ms, an agg ega ion o wo main blocks, he i s one
whose mission is in e ing desc ip o s om images and a second one ha classi ies hose
ea u es. This way, sys em compa ison, e en on he same da ase , compa es wo agg ega ed
blocks so, compa isons on pe o mances o a speci ic block is di icul o assess, as i is
in luenced by he o he one.
To a oid hese p oblems, hese compa isons ha e been igno ed. The only ones aken
in o conside a ion ha e been hose ha compa e one o he blocks and a e based on he
same da ase . The esul s a e shown in Table 2. In global e ms, SVMs and deep lea ning
classi ie s a e he ones wi h be e pe o mances. The bes wo king classi ying deep
lea ning a chi ec u es a e MLP, au oencode s and hose implemen ing So max algo i hms
like GoogLeNe . I is also ele an ha in acco dance wi h C.J. Chong e al. [
3
], sys ems
whose desc ip o s a e dynamic and, he e o e, include e e ences o he ime a iable,
ha e be e pe o mances han hose o he ones whose desc ip o s do no inco po a e
ha a iable.
4.6. Valida ion Da ase s
The sys ems included in his esea ch ha e been es ed by using da ase s. On many
occasions, hose da ase s ha e been speci ically de eloped by he esea che s o es and
alida e hei sys ems, so hei pe o mances can be de e mined. These da ase s, al hough
b ie ly discussed in he a icles p esen ing he sys ems, a e no usually publicly accessible.
Howe e , he e a e also a g oup o da ase s used in he sys em alida ion and pe o -
mance de e mina ion phases ha a e public. Mos o hem a e also accessible h ough he
In e ne , so de elope s can download and use hem o esea ch pu poses. All he da ase s
belonging o his ca ego y used in he de elopmen o he sys ems con ained in his e iew
a e collec ed in Table 3.
Da ase s associa ed wi h he e iewed sys ems, bo h he publicly accessible ones and
he ones ha a e no , a e eco ded ei he by olun ee s o ac o s young and i enough o
gua an ee ha a simula ed all will no ha m hem. In some o hem, ac o s a e ad ised by
he apis s, so hey can imi a e how an elde ly pe son mo es o alls. Finally, none o he
da abases include elde ly eal alls o daily li e ac i i ies pe o med by elde ly people.
The da ase s a e g ouped by collec ed signal ype, so i e big g oups a e iden i ied.
1.
The i s g oup is in eg a ed by a single da ase . I collec s alls and ac i i ies o daily
li e (ADL) execu ed by olun ee s whose esul s a e eco ded using di e en senso s,
included RGB and IR came as. I is used by a single sys em o alida ion pu poses;
2.
The second g oup, which includes h ee da ase s, inco po a es dep h and accele o-
me ic da a. By i s ele ance and numbe o e iewed sys ems using i in hei
pe o mance e alua ion, one da ase is especially impo an , UR all de ec ion [
29
].
This da ase , employed by o e a hi d o all s udied sys ems, includes 30 alls and
40 ADLs eco ded by wo dep h sys ems, one p o iding on al images and a second
came a eco ding e ical ones. This in o ma ion is accompanied by accele ome ic
da a and was eleased in 2015;
Senso s 2021,21, 947 48 o 50
88.
Menacho, C.; O donez, J. Fall de ec ion based on CNN models implemen ed on a mobile obo . In P oceedings o he 2020 17 h
In e na ional Con e ence on Ubiqui ous Robo s (UR), Kyo o, Japan, 22–26 June 2020; pp. 284–289.
89.
Zhong, C.; Ng, W.W.Y.; Zhang, S.; Nugen , C.; Shewell, C.; Medina-Que o, J. Mul i-occupancy Fall De ec ion using Non-In asi e
The mal Vision Senso . IEEE Sens. J. 2020,21, 1. [C ossRe ]
90.
Sun, G.; Wang, Z. Fall de ec ion algo i hm o he elde ly based on human pos u e es ima ion. In P oceedings o he 2020
Asia-Paci ic Con e ence on Image P ocessing, Elec onics and Compu e s (IPEC), Busan, Ko ea, 13–16 Oc obe 2020; pp. 172–176.
91.
Liu, J.-X.; Tan, R.; Sun, N.; Han, G.; Li, X.-F. Fall De ec ion unde P i acy P o ec ion Using Mul i-laye Comp essed Sensing. In
P oceedings o he 2020 3 d In e na ional Con e ence on A i icial In elligence and Big Da a (ICAIBD), Chengdu, China, 28–31
May 2020; pp. 247–251.
92.
Thummala, J.; Pum in, S. Fall De ec ion using Mo ion His o y Image and Shape De o ma ion. In P oceedings o he 2020 8 h
In e na ional Elec ical Enginee ing Cong ess (iEECON), Chiang Mai, Thailand, 4–6 Ma ch 2020; pp. 1–4. [C ossRe ]
93.
Zhang, J.; Wu, C.; Wang, Y. Human Fall De ec ion Based on Body Pos u e Spa io-Tempo al E olu ion. Senso s
2020
,20, 946.
[C ossRe ]
94.
Ko a i, K.N.; Delibasis, K.; Maglogiannis, I. Real-Time Fall De ec ion Using Uncalib a ed Fisheye Came as. IEEE T ans. Cogn.
De . Sys . 2019,12, 588–600. [C ossRe ]
95.
Delibasis, K.; Goudas, T.; Maglogiannis, I. A no el obus app oach o handling illumina ion changes in ideo segmen a ion.
Eng. Appl. A i . In ell. 2016,49, 43–60. [C ossRe ]
96.
PIROPO (People in Indoo ROoms wi h Pe spec i e and Omnidi ec ional Came as). A ailable online: h ps://www.g i.ss .upm.
es/ esea ch/g i-da a/da abases (accessed on 27 Janua y 2020).
97.
Feng, Q.; Gao, C.; Wang, L.; Zhao, Y.; Song, T.; Li, Q. Spa io- empo al all e en de ec ion in complex scenes using a en ion
guided LSTM. Pa e n Recogni . Le . 2020,130, 242–249. [C ossRe ]
98.
Xu, Q.; Huang, G.; Yu, M.; Guo, Y.; Huang, G. Fall p edic ion based on key poin s o human bones. Phys. A S a . Mech. i s Appl.
2020,540, 123205. [C ossRe ]
99.
Shah oudy, A.; Liu, J.; Ng, T.-T.; Wang, G. NTU RGB+D: A La ge Scale Da ase o 3D Human Ac i i y Analysis. In P oceedings
o he 2016 IEEE Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp.
1010–1019.
100.
H un, S.N.; Zin, T.T.; Tin, P. Image P ocessing Technique and Hidden Ma ko Model o an Elde ly Ca e Moni o ing Sys em. J.
Imaging 2020,6, 49. [C ossRe ]
101.
Kalinga, T.; Si i hunge, C.; Buddhika, A.; Jayaseka a, P.; Pe e a, I. A Fall De ec ion and Eme gency No i ica ion Sys em o Elde ly.
In P oceedings o he 2020 6 h In e na ional Con e ence on Con ol, Au oma ion and Robo ics (ICCAR), Singapo e, 20–23 Ap il
2020; pp. 706–712.
102.
Chen, W.; Jiang, Z.; Guo, H.; Ni, X. Fall De ec ion Based on Key Poin s o Human-Skele on Using OpenPose. Symme y
2020
,12,
744. [C ossRe ]
103.
Cai, X.; Li, S.; Liu, X.; Han, G. Vision-Based Fall De ec ion Wi h Mul i-Task Hou glass Con olu ional Au o-Encode . IEEE Access
2020,8, 44493–44502. [C ossRe ]
104.
Chen, Y.; Li, W.; Wang, L.; Hu, J.; Ye, M. Vision-Based Fall E en De ec ion in Complex Backg ound Using A en ion Guided
Bi-Di ec ional LSTM. IEEE Access 2020,8, 161337–161348. [C ossRe ]
105.
Chen, Y.; Kong, X.; Meng, L.; Tomiyama, H. An Edge Compu ing Based Fall De ec ion Sys em o Elde ly Pe sons. P ocedia
Compu . Sci. 2020,174, 9–14. [C ossRe ]
106.
Wang, X.; Jia, K. Human Fall De ec ion Algo i hm Based on YOLO 3. In P oceedings o he 2020 IEEE 5 h In e na ional
Con e ence on Image, Vision and Compu ing (ICIVC), Qingdao, China, 23–25 July 2020; pp. 50–54.
107. Donoho, D.L. Comp essed sensing. IEEE T ans. In . Theo y 2006,52, 1289–1306. [C ossRe ]
108.
Ho p ase , T.; Ha wood, D.; Da is, L.S. A S a is ical App oach o Real- ime Robus Backg ound Sub ac ion and Shadow
De ec ion. In P oceedings o he IEEE ICCV’99 FRAME-RATE Wo kshop, Ke ky a, G eece, 20 Sep embe 1999.
109.
Mi al, A.; Pa agios, N. Mo ion-based backg ound sub ac ion using adap i e ke nel densi y es ima ion. In P oceedings o he
2004 IEEE Compu e Socie y Con e ence on Compu e Vision and Pa e n Recogni ion, CVPR 2004, Washing on, DC, USA, 27
June–2 July 2004.
110.
Mousse, M.A.; Mo amed, C.; Ezin, E.C. Fas Mo ing Objec De ec ion om O e lapping Came as. In P oceedings o he 12 h
In e na ional Con e ence on In o ma ics in Con ol, Au oma ion and Robo ics, Colma , F ance, 21–23 July 2015; pp. 296–303.
111.
Ma io, I.; Chacon, M.; Se gio, G.D.; Ja ie , V.P. Simpli ied SOM-neu al model o ideo segmen a ion o mo ing objec s. In
P oceedings o he 2009 In e na ional Join Con e ence on Neu al Ne wo ks, A lan a, GA, USA, 14–19 June 2009; pp. 474–480.
112.
Nguyen, V.-T.; Le, T.-L.; T an, T.-H.; Mullo , R.; Cou boulay, V.; Van-Toi, N. A new hand ep esen a ion based on ke nels o
hand pos u e ecogni ion. In P oceedings o he 2015 11 h IEEE In e na ional Con e ence and Wo kshops on Au oma ic Face and
Ges u e Recogni ion (FG), Ljubljana, Slo enia, 4–8 May 2015; Volume 1, pp. 1–6.
113.
Kennedy, J.; Ebe ha , R. Pa icle swa m op imiza ion. In P oceedings o he ICNN’95-In e na ional Con e ence on Neu al
Ne wo ks, Pe h, Aus alia, 27 No embe –1 Decembe 1995; Volume 4, pp. 1942–1948. [C ossRe ]
114.
Bobick, A.F.; Da is, J.W. The ecogni ion o human mo emen using empo al empla es. IEEE T ans. Pa e n Anal. Mach. In ell.
2001,23, 257–267. [C ossRe ]

Senso s 2021,21, 947 49 o 50
115.
Lucas, B.D.; Kanade, T. An i e a i e image egis a ion echnique wi h an applica ion o s e eo ision. In P oceedings o he
Imaging Unde s anding Wo kshop, Vancoube , Canada, 24–28 Augus 1981; pp. 121–130.
116. Shi, J.; Tomasi, C. Good Fea u es o T ack. In P oceedings o he IEEE Con e ence on Compu e Vision and Pa e n Recogni ion,
Sea le, DC, USA, 21–23 June 1994; pp. 593–600.
117. Candès, E.; Li, X.; Ma, Y.; W igh , J. Robus p incipal componen analysis? J. ACM 2011,58, 1–37. [C ossRe ]
118.
Dalal, N.; T iggs, B. His og ams o o ien ed g adien s o human de ec ion. In P oceedings o he 2005 IEEE Compu e Socie y
Con e ence on Compu e Vision and Pa e n Recogni ion, CVPR, San Diego, CA, USA, 20–25 June 2005.
119.
Nizam, Y.; Haji Mohd, M.N.; Abdul Jamil, M.M. A S udy on Human Fall De ec ion Sys ems: Daily Ac i i y Classi ica ion and
Sensing Techniques. In . J. In eg . Eng. 2016,8, 35–43.
120.
Kali a, S.; Ka maka , A.; Haza ika, S.M. E icien ex ac ion o spa ial ela ions o ex ended objec s is-à- is human ac i i y
ecogni ion in ideo. Appl. In ell. 2018,48, 204–219. [C ossRe ]
121.
Rosenbla , F. The pe cep on: A p obabilis ic model o in o ma ion s o age and o ganiza ion in he b ain. Psychol. Re .
1958
,65,
386–408. [C ossRe ] [PubMed]
122.
Hop ield, J.J. Neu al ne wo ks and physical sys ems wi h eme gen collec i e compu a ional abili ies. P oc. Na l. Acad. Sci. USA
1982,79, 2554–2558. [C ossRe ] [PubMed]
123. Hoch ei e , S.; Schmidhube , J. Long Sho -Te m Memo y. Neu al Compu . 1997,9, 1735–1780. [C ossRe ] [PubMed]
124. Su ske e , I.; Vinyals, O.; Le, Q.V. Sequence o sequence lea ning wi h neu al ne wo ks. a Xi 2014, a Xi :1409.3215.
125.
Hubel, D.H.; Wiesel, T.N. Recep i e ields, binocula in e ac ion and unc ional a chi ec u e in he ca ’s isual co ex. J. Physiol.
1962,160, 106–154. [C ossRe ]
126.
Fukushima, K. Neocogni on: A sel -o ganizing neu al ne wo k model o a mechanism o pa e n ecogni ion una ec ed by shi
in posi ion. Biol. Cybe n. 1980,36, 193–202. [C ossRe ]
127.
LeCun, Y.; Bo ou, L.; Bengio, Y.; Ha ne , P. G adien -based lea ning applied o documen ecogni ion. P oc. IEEE
1998
,86,
2278–2324. [C ossRe ]
128.
Tyge , M.; B una, J.; Chin ala, S.; LeCun, Y.; Pian ino, S.; Szlam, A. A Ma hema ical Mo i a ion o Complex-Valued Con olu ional
Ne wo ks. Neu al Compu . 2016,28, 815–825. [C ossRe ]
129.
Wojke, N.; Bewley, A.; Paulus, D. Simple online and eal ime acking wi h a deep associa ion me ic. In P oceedings o he 2017
IEEE In e na ional Con e ence on Image P ocessing (ICIP), Beijing, China, 17–20 Sep embe 2017; pp. 3645–3649.
130. Junejo, I.N.; Fo oosh, H. Euclidean pa h modeling o ideo su eillance. Image Vis. Compu . 2008,26, 512–528. [C ossRe ]
131.
Maldonado, C.; Rios-Figue oa, H.V.; Mezu a-Mon es, E.; Ma in, A.; Ma in-He nandez, A. Fea u e selec ion o de ec allen
pose using dep h images. In P oceedings o he 2016 In e na ional Con e ence on Elec onics, Communica ions and Compu e s
(CONIELECOMP), Cholula, Mexico, 24–26 Feb ua y 2016; pp. 94–100.
132.
Labay ade, R.; Aube , D.; Ta el, J.-P. Real ime obs acle de ec ion in s e eo ision on non la oad geome y h ough " -dispa i y"
ep esen a ion. In P oceedings o he In elligen Vehicle Symposium, Ve sailles, F ance, 17–21 June 2002.
133.
Schmiedel, T.; Einho n, E.; G oss, H.-M. IRON: A as in e es poin desc ip o o obus NDT-map ma ching and i s applica ion
o obo localiza ion. In P oceedings o he 2015 IEEE/RSJ In e na ional Con e ence on In elligen Robo s and Sys ems (IROS),
Hambu g, Ge many, 28 Sep embe –2 Oc obe 2015; pp. 3144–3151.
134.
Fischle , M.A.; Bolles, R.C. Random Sample Consensus: A Pa adigm o Model Fi ing wi h Applica ions o Image Analysis and
Au oma ed Ca og aphy. Read. Compu . Vis. 1987,24, 726–740. [C ossRe ]
135.
Solbach, M.D.; Tso sos, J.K. Vision-Based Fallen Pe son De ec ion o he Elde ly. In P oceedings o he 2017 IEEE In e na ional
Con e ence on Compu e Vision Wo kshops (ICCVW), Venice, I aly, 22–29 Oc obe 2017; pp. 1433–1442.
136.
Zhang, K.; Chen, S.-C.; Whi man, D.; Shyu, M.-L.; Yan, J.; Zhang, C. A p og essi e mo phological il e o emo ing nong ound
measu emen s om ai bo ne LIDAR da a. IEEE T ans. Geosci. Remo e Sens. 2003,41, 872–882. [C ossRe ]
137. Co es, C.; Vapnik, V. Suppo - ec o ne wo ks. Mach. Lea n. 1995,20, 273–297. [C ossRe ]
138. Tax, D.M.J.; Duin, R.P.W. Suppo Vec o Da a Desc ip ion. Mach. Lea n. 2004,54, 45–66. [C ossRe ]
139. B eiman, L. Random Fo es s. Mach. Lea n. 2001,45, 5–32. [C ossRe ]
140. F iedman, J.H. G eedy unc ion app oxima ion: A g adien boos ing machine. Ann. S a . 2011,29, 1189–1232. [C ossRe ]
141. Rabine , L.; Juang, B. An in oduc ion o hidden Ma ko models. IEEE ASSP Mag. 1986,3, 4–16. [C ossRe ]
142.
Na a ajan, P.; Ne a ia, R. Online, Real- ime T acking and Recogni ion o Human Ac ions. In P oceedings o he 2008 IEEE
Wo kshop on Mo ion and ideo Compu ing, Coppe Moun ain, CO, USA, 8–9 Janua y 2008; pp. 1–8. [C ossRe ]
143. Kalman, R.E. A New App oach o Linea Fil e ing and P edic ion P oblems. J. Basic Eng. 1960,82, 35–45. [C ossRe ]
144.
Go don, N. Bayesian Me hods o T acking. Ph.D. Thesis, Ma hema ics Depa men Impe ial College, London, UK, 1993.
A ailable online: h ps://spi al.impe ial.ac.uk/bi s eam/10044/1/7783/1/NeilGo don-1994-PhD-Thesis.pd (accessed on 27
Janua y 2020).
145.
Kangas, M.; Vikman, I.; Nybe g, L.; Ko pelainen, R.; Lindblom, J.; Jämsä, T. Compa ison o eal-li e acciden al alls in olde people
wi h expe imen al alls in middle-aged es subjec s. Gai Pos u e 2012,35, 500–505. [C ossRe ] [PubMed]
146.
Klenk, J.; Becke , C.; Lieken, F.; Nicolai, S.; Mae zle , W.; Al , W.; Zijls a, W.; Hausdo , J.M.; Van Lummel, R.C.; Chia i, L.; e al.
Compa ison o accele a ion signals o simula ed and eal-wo ld backwa d alls. Med. Eng. Phys.
2011
,33, 368–373. [C ossRe ]
[PubMed]
Senso s 2021,21, 947 50 o 50
147.
Thilo, F.J.S.; Hahn, S.; Hal ens, R.; Schols, J.M. Usabili y o a wea able all de ec ion p o o ype om he pe spec i e o olde
people–A eal ield es ing app oach. J. Clin. Nu s. 2018,28, 310–320. [C ossRe ]
148.
Demi is, G.; Chaudhu i, S.; Thompson, H.J. Olde Adul s’ Expe ience wi h a No el Fall De ec ion De ice. Telemed. J. E. Heal h
2016,22, 726–732. [C ossRe ]
149.
Ren, L.; Peng, Y. Resea ch o Fall De ec ion and Fall P e en ion Technologies: A Sys ema ic Re iew. IEEE Access
2019
,7,
77702–77722. [C ossRe ]