Towa ds a cloud‑based au oma ed su eillance sys em using
wi eless echnologies
Ja ie J. Salme ón‑Ga cía1 · Sjoe d an den D ies2 · Fe nando Díaz‑del‑Río1 ·
A u o Mo gado‑Es e ez3 · Jose Luis Se illano‑Ramos1 · M. J. G. an de Moleng a 2
addi ion, his p oo -o -concep claims ha many in e es -
ing oppo uni ies and challenges a ise, o example, when
mobile wa ch obo s and ixed came as would ac as a
eam o ca ying ou complex collabo a i e su eillance
s a egies.
Keywo ds Su eillance · Wo ld modeling · Wi eless
communica ion · Compu a ion o loading · RGBD
came as · Cloud obo ics
1 In oduc ion
Nowadays, he de elopmen o au oma ed su eillance sys-
ems has become mo e han widesp ead [7]. The e a e innu-
me able en i onmen s whe e hei use has become c i ical,
such as ci ies, households and co po a e buildings, no only
o secu i y, bu also o people loca ion o cus ome queue
analy ic [38]. Consequen ly, he e is a need o p ocess, send
and analyze he high olume o senso ial media p oduced
by came as, ada s and o he senso s. This ask in ol es no
only he communica ion in as uc u e bu also he neces-
sa y mul imedia compu ing amewo k. Mo e p ecisely,
in a u u e sma ci y he e would be innume able de ices
(small senso s, compu e s, mobile obo s, amongs o he s)
ha wo king oge he should pe ec ly c ea e an ex emely
sa e en i onmen . Howe e , se e al cu ing-edge su eil-
lance solu ions ely on e y expensi e sensing echnology,
high pe o mance dedica ed ha dwa e and wi ed connec-
i i y, and a e he e o e una o dable o many companies
o scena ios. Hence, he ques ion o be add essed in his
pape would be: un il wha ex en can all hese he e ogene-
ous de ices collabo a e o pe o m a mul i ace ed ask-like
su eillance using cu en low-cos a ailable echnologies?
Abs ac Cloud Compu ing can b ing mul iple bene i s
o Sma Ci ies. I pe mi s he easy c ea ion o cen alized
knowledge bases, hus s aigh o wa dly enabling ha mul-
iple embedded sys ems (such as senso o con ol de ices)
can ha e a collabo a i e, sha ed in elligence. In addi ion o
his, hanks o i s as compu ing powe , complex asks can
be done o e low-spec de ices jus by o loading compu-
a ion o he cloud, wi h he addi ional ad an age o sa -
ing ene gy. In his wo k, cloud’s capabili ies a e exploi ed
o implemen and es a cloud-based su eillance sys-
em. Using a sha ed, 3D symbolic wo ld model, di e en
de ices ha e a comple e knowledge o all he elemen s,
people and in ude s in a ce ain open a ea o inside a build-
ing. The implemen a ion o a olume ic, 3D, objec -o i-
en ed, cloud-based wo ld model (including seman ic in o -
ma ion) is no el as a as we know. Ve y simple de ices
(o ange Pi) can send RGBD s eams (using kinec came as)
o he cloud, whe e all he p ocessing is dis ibu ed and
done hanks o i s inhe en scalabili y. A p oo -o -concep
expe imen is done in his pape in a es ing lab wi h mul-
iple came as connec ed o he cloud wi h 802.11ac wi e-
less echnology. Ou esul s show ha his kind o su eil-
lance sys em is possible cu en ly, and ha ends indica e
ha i can be imp o ed a a sho e m o p oduce high
pe o mance igilance sys em using low-speed de ices. In
*Fe nando Díaz-del-Río
[email p o ec ed]
1 Escuela Técnica Supe io de Ingenie ía In o má ica,
Uni e sidad de Se illa, Se illa, Spain
2 Technische Uni e si ei Eindho en, Eindho en,
The Ne he lands
3 Escuela Supe io de Ingenie ía, Uni e sidad de Cádiz, Cádiz,
Spain
A powe ul ool o ul ill his challenge is he inco po a-
ion o Cloud Compu ing, because many o i s p ope ies
can bene i such su eillance sys ems. Speci ically, he ol-
lowing cha ac e is ics i pe ec ly:
––On-demand esou ce p o ision: The cloud is able o
p o ide bo h compu ing and s o age esou ces. This
is no mally done in he o m o he so-called con ain-
e s. Fo ins ance, should a su eillance o “wa ch”
obo equi e compu ing powe o pe o m a ce -
ain ask, hen i would only ha e o spin up as much
ins ances as equi ed o comple e i . Mos comme cial
cloud pla o ms ( o example, Amazon EC2, Windows
Azu e, Google App Engine, e c.) use a pay-as-you-go
app oach. Ne e heless, his should be much cheape
han all he capi al expendi u e equi ed o a clus e ,
no o men ion all he main enance cos s.
––Dynamic scalabili y: This p ope y ela es o he abil-
i y o he cloud o dynamically adap o he use needs.
A clea example is ha o a mul iple in usion. I , sud-
denly, he numbe o eques s d ama ically inc ease
due o his ansg ession, he se ice migh no be able
o p ocess hem in a easonable ime. To add ess his,
he cloud could inc ease he numbe o compu ing
esou ces de o ed o his ask. Mo eo e , once he h ea
has ceased and he numbe o eques s dec eases, he
cloud could educe he amoun o compu ing esou ces.
This ela es o he “u ili y” concep behind Cloud Com-
pu ing.
In ac , a g ea in e es exis s in applying and de eloping
Cloud Compu ing amewo ks o he su eillance a ea [17,
26]. Howe e , many challenges s ill emain. Fo ins ance,
he inco po a ion o he cloud pa adigm o ces de elop-
e s o e hink hei so wa e a chi ec u es so, o ins ance,
he dynamic scalabili y p ope y could be exploi ed (see
Sec . 4). Ano he majo conce n is ha , as poin ed ou in
[26], he cos o deploying a ideo su eillance sys em
using cloud compu ing is no necessa ily less expensi e
han implemen ing he ha dwa e locally. Fu he mo e, legal
and p i acy issues mus be add essed when using comme -
cial cloud pla o ms.
An in e es ing app oach is he use o a p i a e cloud se -
ing, so ha c i ical su eillance da a is always kep in he
p i a e cloud [17]. This app oach also acili a es he in e-
g a ion o embedded de ices ha can success ully pe o m
collabo a i e asks such as localiza ion, mapping [30], a -
ic-ligh s a us de ec ion [4] o heal hca e [10]. As a conse-
quence, concep s such as “In e ne o Robo s” [13, 37] a e
now possible. Fu he mo e, a p i a e cloud can be comple-
men ed wi h comme cial cloud pla o ms when needed o
op imize deploymen and main enance cos s, an app oach
commonly known as hyb id cloud [17]. The e o e, using
he cloud o a collabo a i e su eillance sys em is ano he
in e es ing applica ion ha we add ess and e alua e in his
wo k.
In a g ea ex en , he answe o educe cos s and deploy
mo e easily a igilance sys em ha in eg a es a numbe o
embedded de ices comes om he use o p i a e/hyb id
Cloud Compu ing and high bandwid h wi eless echnolo-
gies. Mo eo e , he cloud is he mos sui able and cos -
e ec i e candida e o wha is known as compu a ion o -
loading. The idea is o ee an embedded de ice om hea y
compu a ions, so ha an ex e nal pla o m would do hem
and ge he esul s back. This is especially in e es ing as i
allows he de ices o pe o m mo e complex asks, o e -
coming hei ha dwa e limi a ions and hei inhe en di -
icul upg ading. Fu he mo e, o loading CPU-in ensi e
asks implies less powe consump ion in he de ices and
obo s. Following his idea, building e y simple de ices
ha ely on compu a ion o loading will se a end in
o hcoming yea s [31]. On he o he hand, he use o high
bandwid h wi eless echnologies is especially in e es ing
o wo easons. Fi s , i pe mi s he use o mobile obo s
o UAVs (Unmanned Ae ial Vehicles), b inging innume -
able possibili ies in su eillance. Second, he new s anda d
802.11ac p omises bandwid hs o he o de o 1 Gbps [9].
Hence, e en ually, cu en bo lenecks o WiFi connec ions
could be o e come. The e o e, he con enien combina ion
o Cloud Compu ing, wi eless echnologies and he e oge-
neous embedded de ices (depending on he budge ) could
be a pe ec candida e o c ea ing a cos -e ec i e su eil-
lance and in ude de ec ion solu ions.
Besides, he e is a p oblem ha needs o be add essed
o de ec in ude s: he co ec in e p e a ion and sha ing
o in o ma ion o he en i onmen . In his sense, he lack
o seman ic in o ma ion p o okes p oblems in his kind o
asks, o ins ance [5, 11, 22]:
––Some ope a ions such as clea ing could be done mo e
e icien ly i mo e in o ma ion o he s a ic en i onmen
was known.
––Wi hou symbolic in o ma ion, i is e y di icul o he
obo o make p edic ions based on he en i onmen .
The sys em p esen ed he e is a i s p o o ype o a cloud-
based su eillance sys em using a sha ed, 3D symbolic
wo ld model; simple embedded de ices ha can o load
compu a ion o he cloud; mobile de ices including
“wa ch” obo s; high-bandwd h wi eless connec ions; and
Kinec came as as he sensing echnology. The pape is
o ganized as ollows. Fi s , nex sec ion p esen s he sys em
model. In Sec . 3, a b ie e iew o ecen ad ances in he
a ea is p esen ed. Then, in Sec . 4 all he main elemen s o
he p o o ype (wo ld model, in o ma ion s o age, compu a-
ion o loading) a e desc ibed. To p o e he e icacy o his
i s p o o ype, some p elimina y expe imen al esul s a e
included (especially ocused on communica ion and com-
pu a ion pe o mance) in Sec . 5. Finally, u u e lines o
wo k and conclusions a e p esen ed in Sec s. 6 and 7.
2 Sys em model
In his pape , we p opose a amewo k o gi ing suppo
o a su eillance sys em ia an Objec O ien ed Wo ld
Model ha is dis ibu ed among he cloud and he es o
de ices. We use a 3D olume ic, objec -o ien ed wo ld
model, called En i onmen Desc ip o (ED), (de eloped by
he “Con ol Sys ems Technology” in TU/e Uni e si y [5,
11]) ha allows he modules in he sys em o ake he con-
ex o he en i onmen . Such a ully o loaded cloud-based
implemen a ion o a olume ic, 3D, objec -o ien ed wo ld
model (including seman ic in o ma ion) has ne e been
ca ied ou as a as we know.
Figu e 1 shows an applica ion scena io o he p oposed
Cloud-based Wo ld Model pla o m. Su eillance asks can
be sha ed by se e al ixed de ices wi h di e en ha dwa e
con igu a ions and sensing echnologies: s e eo came as,
RGBD came as, ange inde s, lase , amongs o he s. In
addi ion, each de ice can be assigned o di e en asks. The
common case would be ha o came a de ices ins alled
in di e en places. Meanwhile, in addi ion o his, he e
can be mobile “Wa ch obo s”, which na iga e he a ea o
building looking o in ude s. Then, as hey all sha e he
same wo ld model, i a came a de ec ed an in ude , hen
all de ices would know hei exac loca ion. Consequen ly,
“Wa ch obo s” would ans o m hei loca ion coo dina es
o ela e hei own pos u e o ha o he in ude , wi h he
aim o pu suing him/he .
Figu e 2 shows a concep ual diag am o he p oposed
Cloud-based Wo ld Model pla o m. In ou implemen a-
ion, ask o ganiza ion is lexible enough so ha , depend-
ing on he ha dwa e speci ica ions, some de ices can pe -
o m all he compu a ions on hei own (Case 2) o o load
compu a ions o he cloud (Case 1). Non-sensing de ices
ha o m pa o he sys em such as domo ic con olle s
can ecei e an o de and eac acco dingly, o ins ance, by
locking a doo i an in ude we e ound (Case 3). Hence,
using he cloud pa adigm as a na u al cen al in e ace o
many de ices, an ex emely e icien and coope a i e su -
eillance sys em can be de eloped. A de ailed explana-
ion o he blocks o his Fig. 2 will be desc ibed in u he
sec ions.
I is wo h no ing ha in his kind o applica ions,
conce ns such as esponse ime equi emen s o da a
sa e y and p i acy may lead o an al e na i e pa adigm
o Cloud Compu ing called Edge Compu ing [34]. This
e m e e s o p ocessing he da a a he edge o he
ne wo k, ha is, close o he “ hings”, in he IoT sense,
o end de ices. As men ioned be o e, in ou expe imen s
we will use a p i a e cloud which could be conside ed as
a kind o edge-compu ing sys em. The eason is ha his
way we a oid longe and mo e a iable delays and can
ocus on implemen a ion issues such as compu a ion o -
loading o en i onmen sensing and modeling. Howe e ,
along he pape we p e e o use he mo e gene al model
and e minology o Cloud Compu ing. Fu he mo e, he
ex ension o ou su eillance sys em o a la ge Sma
Ci y would bene i om a mo e gene al Cloud Compu -
ing pa adigm, o ins ance, ga he ing sensing da a om
di e en buildings.
3 Rela ed wo k
The e a e nume ous esea ch wo ks in he a ea o au o-
ma ed su eillance. In ela ion wi h his wo k, he mos
ele an can be a anged in hose wo ks ha use he cloud
o in eg a ing senso in o ma ion, hose ha deal wi h he
de ec ion algo i hms, hose ha es se e al senso de ices,
and inally, o he s ha s udy ne wo k echnologies.
The idea o using he cloud o in eg a ing mul iple sen-
so in o ma ion, including su eillance in o ma ion, has
also been explo ed in he li e a u e. Mo eo e , a new e m
has been coined o de ine he applica ion o Cloud Com-
pu ing (in e ms o ubiqui y, s o age, compu a ion, p i acy,
scalabili y, amongs o he s) o his a ea: Video Su eillance
as a Se ice [20, 29]. Fo ins ance, in [27] he au ho s deal
wi h he p oblem o in eg a ing he e ogeneous senso in o -
ma ion by c ea ing a da abase scheme able o in eg a e di -
e en ypes o senso . Howe e , mos o he a ailable solu-
ions ely on wi ed ne wo ks, such as [14, 15, 21, 36]. Only
a ew wo ks ha e de eloped some aspec s o cloud-based
su eillance pla o m using wi eless ne wo ks. Au ho s o
[38] p esen ed Vigil, whe e some de ices wi h came as
execu e ace ecogni ion algo i hms and send he esul s o
he cloud. They also in oduce he concep o in a-clus e
p ocessing, when he e is a se o came as wi h o e lap-
ping isions. Compa ed wi h Vigil, he wo k p esen ed in
his pape akes in o conside a ion he use o e y simple
de ices hanks o compu a ion o loading. In [35] a Cloud
Ga eway o il e ing he in o ma ion om mul iple senso s
in Wi eless Senso Ne wo ks was implemen ed. This il e -
ing is applied o a basic cloud su eillance sys em ob ain-
ing an 83% accu acy.
Mos o he cloud-based su eillance sys ems ha e he
ollowing a chi ec u e: mul iple came as s eam o he
cloud, which is esponsible o analyzing he oo age in
sea ch o e en s. Only in he case o [14] some p e-p o-
cessing is done be o e sending o he cloud. Compa ed o
hese wo ks, his pape o e s a mo e lexible app oach, as
he de ices can decide whe he o o load o no depend-
ing on hei needs.
Ano he impo an elemen o Cloud Compu ing is
Quali y-o -Se ice (QoS) managemen . In his sense,
in [3] a se ice QoS-adap i e con igu a ion amewo k
o op imizing his ac o in su eillance sys ems is
p oposed.
Ano he s udy elemen co esponds o image analysis
o de ec in ude s. Se e al algo i hms ha e been de ised,
such as La en Seman ic Analysis (LSA), Ke nel-based
Fig. 1 Example o applica ion in a building
Online Anomaly De ec ion (KOAD) and Ke nel Es ima-
ion-based Anomaly De ec ion (KEAD), Ke nel P incipal
Componen Analysis (KPCA) and One-Class Neighbo
Machine (OCNM) [2], using machine lea ning echniques
as well.
In [25], ins ead o egula ideo came as, omnidi ec-
ional came as a e used, which pe mi he su eillance o a
whole oom o a bigge open space wi h only one came a.
In [12], he senso used is ha o Mic oso Kinec , wi h
sa is ac o y esul s. In his sense, he sys em p esen ed he e
also makes use o Kinec Senso s which, a a easonable
p ice, p o ide a dep h channel, ex emely impo an o
objec de ec ion and acking.
As i can be seen, mos o he su eillance sys ems a ail-
able in he li e a u e no mally use ixed came as. In his
sense, one o he majo asse s o his wo k is he collabo-
a ion o di e en he e ogeneous de ices (such as mobile
“wa ch” obo s) hanks o he use o a sha ed wo ld model
and he adap a ion o he cloud o each obo ’s needs.
Mo eo e , he use o mobile obo s b ings some in e es -
ing oppo uni ies and challenges o mo e complex su eil-
lance s a egies and sys ems.
The app oach o using an edge-compu ing pla o m and
wi eless echnologies is becoming inc easingly a ac i e
o esea che s on su eillance sys ems [6]. Fo ins ance,
[16] shows ha an edge node can educe he eac ion ime
be ween edge and senso s, and can imp o e ne wo k s a-
bili y as well by uploading only necessa y da a. Also, [23]
imp o es he op imal bandwid h dis ibu ion and alloca ion
in a ideo su eillance sys em using up- o-da e wi eless
echnologies. In he p esen pape , we ollow his app oach
wi h he addi ional con ibu ions o implemen ing a 3D,
objec -o ien ed, cloud-based wo ld model ha includes
seman ic in o ma ion, as well as allowing he use o sim-
ple embedded de ices and mobile “wa ch” obo s h ough
compu a ion o loading.
4 Implemen a ion o he pla o m
In his sec ion, he implemen a ion o he Cloud Wo ld
Model based Su eillance Pla o m is de ailed. The imple-
men a ion o he Su eillance Pla o m consis s o he ol-
lowing pa s:
–– De ini ion o he elemen s o Wo ld Model (Sec . 4.1)
––De ini ion and deploymen o a Wo ld Model S o age
Sys em (Sec . 4.2)
––Implemen a ion o he Senso In eg a ion Module,
esponsible o p ocessing he image s eams (Sec . 4.3).
4.1 Elemen s o he wo ld model
Wo ld modeling is he p ocess o c ea ing a model o an
en i onmen (wi h i s inhe en complexi y), which is a
key ask o in elligen sys ems. The mo e accu a e he
ep esen a ion o he en i onmen , he mo e success ully
a ask will be pe o med. Hence, as s a ed in Sec . 1, add-
ing seman ic in o ma ion o each elemen o he en i on-
men implies a d ama ic inc ease in accu acy. In he imple-
men a ion shown in his pape , he main elemen s o he
wo ld model (simpli ying om [5, 11]) a e en i ies and
ans o ms.
En i ies: An en i y is an elemen o he wo ld. I has he
ollowing p ope ies:
Fig. 2 Concep ual diag am o
he cloud-based su eillance
sys em

––ID: Unique iden i ie o an en i y. No o he en i y can
ha e he same iden i ie , e en i he en i y was emo ed.
––Type: Speci ies he ype o he en i y. This allows o
make classi ica ions o c ea e subse s o en i ies o di -
e en ac ions. Fo example, se e al ac ions can be done
depending i he en i y is a pe son, a u ni u e o a wall.
––Pose: Es ablishes he posi ion and o ien a ion o he
en i y in he wo ld model. These coo dina es a e abso-
lu e.
––Shape o con ex hull: Adds olume ic in o ma ion o
he en i y. This shape can ei he be eal o i ual. An
en i y can ha e bo h.
––Measu emen s: In o ma ion ob ained by senso s ha is
associa ed o an en i y in he wo ld model. These meas-
u emen s hold he senso da a and he ime-s amp.
T ans o ms: de ines he spa ial ela ionship be ween wo
di e en en i ies in he wo ld model. Mul iple ans o -
ma ions o each pa en –child pai can be s o ed o di -
e en imes amps which esul s in a g aph o en i ies and
ans o ms.
4.2 Cloud wo ld model s o age
As s a ed in p e ious sec ions, a collabo a i ely buil wo ld
model o e s se e al ad an ages. Howe e , he e a e some
bo lenecks ha he cloud solu ion mus add ess.
Fi s , i he numbe o obo s inc eases (and he e o e,
he numbe o Da abase ead/w i e ope a ions), he cloud
migh no be able o sa is y all que ies in a easonable ime.
To make ma e s wo se, i he numbe o en i ies inc eases,
he cloud esou ces assigned o he da abase may un ou .
The e o e, one o he main objec i es o he cloud solu ion
is o adap compu ing esou ces a un ime depending on
he needs.
To sa is y hese equi emen s, Dis ibu ed Da abase
Managemen Sys ems (DDBMS) a e he pe ec candi-
da es o s o ing all he in o ma ion, hanks o hei mas-
si ely pa allel na u e, ease o use and po abili y. Among
all he a ailable op ions, one o he bes DDBMS pa a-
digms is ha o Big able. I s main ad an age is ha i
uses a spa se, dis ibu ed and mul idimensional so ed
map, using he ollowing da a o ganiza ion [8]:
( ow:s ing, column:s ing, ag:s ing, ime:in 64)
→
s ingHence, he e would only be one (o wo)
big ables ha can be pe ec ly sca e ed among
di e en con aine s in he cloud. The e o e, i he
a o emen ioned bo lenecks occu , hen mo e con-
aine s can be spun up a un ime. On he con a y,
i he da abase is less loaded, hen he cloud can
scale back, hus sa ing esou ces o o he asks.
Ha ing in mind his DB model, he nex schema was
designed o i wi h he needs o a su eillance sys em.
The idea is no only o s o e he cu en in o ma ion
abou en i ies, bu also hei his o y. In his sense, we use
he concep o del a, ha is, a change in a ce ain en i y
( ollowing he so-called e en -d i en mechanism [17]).
To cla i y he concep , an example ollows (depic ed in
Fig. 3): suppose wo obo s A, B and an en i y E. Bo h
obo s ha e synch onized hei local wo ld models a an
ins an , inding ha he mos ecen change was done
a
−1
(“Las ime” in Fig. 3). The e o e, hey know all
he in o ma ion abou E (shape, pose, ype, e c.). A
+1
,
B egis e s changes in he pose o E, and he e o e will
upda e he wo ld model wi h his new in o ma ion. I
no hing bu he pose has changed, e-sending he in o -
ma ion abou he shape would be a was e o bandwid h.
The e o e, B will only send he change ( ha is, he del a)
o he pose. This same easoning can be applied o A,
which is only in e es ed in he mos ecen changes (del-
as) since da abase’s mos ecen ime-s amp
−1
( o
sa e esou ces). Thanks o he use o imes amps, a obo
can que y all he new del as since a gi en ime .
Taking his in o accoun , he e will be a able o del as
in he da abase wi h he ollowing columns: pose, ype,
shape, con ex hull and dele ed (whe he he en i y was
clea ed o no ). The in o ma ion will be s o ed in he
JSON (Ja aSc ip Objec No a ion) o ma . The ow ID
will co espond o he en i y’s ID.
Among he a ailable implemen a ions o he Big able
model, Hype able has been chosen because i is w i en
on C++ and p omises highe pe o mance esul s com-
pa ed o compe i o s such as HBase [1]. This is o g ea
impo ance when wo king wi h obo s, and in some cases,
nea eal- ime condi ions. The elemen s o a Hype able
sys em ha we e con igu ed o ou sys em a e [19]:
ange se e ( o handling he ead and w i e ope a ions
in he able), Mas e node ( o c ea ing/dele ing ables),
Th i B oke (in e ace be ween he da abase and he cli-
en s) and FS B oke (no malized ilesys em in e ace).
Figu e 4 on he le shows he implemen a ion o he
wo ld model using Hype able ( he FS B oke is no
shown, as i is being used in all he nodes). On he igh ,
an example o adap a ion when a bigge numbe o obo s
demands mo e esou ces has been depic ed. In his case,
mo e ins ances o Range se e s and Th i B oke s a e
spin up by he cloud o sa is y his new load.
4.3 Senso in eg a ion modules
The senso in eg a ion o loading implemen ed he e is
a simpli ied e sion o [5, 11]. The wo k low is depic ed
in Fig. 5. All modules a e implemen ed using he Robo -
ics Ope a ing Sys em (ROS) F amewo k. ROS is an Open
Fig. 3 Example o a del a upda e
Fig. 4 Implemen a ion o
he cloud Wo ld model using
hype able
Sou ce Robo ics So wa e F amewo k (using C++ and
Py hon) ha allows he implemen a ion o ad anced dis-
ibu ed a chi ec u es, e y simila o Mul i-Agen sys ems
[18]. Thanks o he use o opics, se ices, pee - o-pee
communica ions and comp ession echniques, ROS makes
he o loading p ocess simple and scalable. Main p ocesses
(nodes in ROS con en ion) co espond o hose modules o
Fig. 5, while connec ing links o hese modules ha e been
implemen ed using ROS opics and se ices.
––The di e en ope a ions done in he senso in eg a ion
a e he ollowing:RGBD Masking: This p ocess will
only use he D channel o he s eam. I is esponsible
o de ec ing en i ies using ea u e ex ac ion (p oduc-
ing 3D Poin Clouds), associa ion (compa ing wi h a
ende ed wo ld model) and segmen a ion (c ea ing new
en i ies om he unassocia ed da a) [5, 11]. Figu e 6
shows an example. This module is one o he mos com-
pu a ionally demanding. In ou case, as we a e dealing
wi h su eillance cases, his module will disca d hose
en i ies whose size is no big enough o be a pe son.
––Senso usion: This p ocess ecei es he RGBD s eam
and he a o emen ioned en i ies wi h mask measu es.
The idea is o use he p e iously unused RGB channel
and ex ac o he p ope ies om he en i ies. The e-
o e, he module mus bu e he RGBD s eam un il i s
ela ed masked measu emen a i es. Figu e 7, using he
mask om Fig. 6, now ob ains an RGBD measu emen
(which, o example, con ains colo in o ma ion).
––Pe cep ion: This p ocess ecei es he RGBD measu e-
men s (c ea ed by he Senso Fusion module) and pe -
o ms an analysis o disco e he ype o he en i y. This
will be he esponsible module o de ec ing whe he an
en i y is an in ude o no . As ou aim is demons a -
ing he cloud wo king and easibili y, in he expe imen s
p esen ed in his pape he sys em only de ec s whe he
he en i y is wea ing o no an uni o m. I he pe son
was no wea ing i , hen i would no be an au ho ized
pe sonnel. On comme cial su eillance sys ems, i is
ob ious ha a knowledge base (wi h empla es) mus
be used. Fo he p e ious example, Fig. 8 analyzes he
RGBD measu emen and concludes ha he en i y is an
au ho ized pe son.
The p esen ed pla o m is aimed o be as much lexible
as possible. In his sense, a obo may no ha e enough
p ocessing powe o do all he senso in eg a ion on-boa d
bu su icien o do pa o his p ocessing. The e o e, he
obo should be able o choose which pa s o he senso
in eg a ion a e o loaded. To implemen his ea u e, he
sys em was implemen ed using modules. Fo ins ance, one
ins ance may only ha e he Pe cep ion module (and hus
Fig. 5 Wo k low o he Senso
In eg a ion model
Fig. 6 Example o a mask
measu emen
only pe o m ecogni ion ope a ions), whe eas ano he
node may ha e all modules. As a consequence, se e al o -
loading op ions can be aken in o conside a ion.
5 P elimina y esul s
These p elimina y es s a e aimed o show he e icacy and
iabili y o no only ha ing a sha ed wo ld model, bu also
o using wi eless echnologies. The es s a e di ided in o
h ee ca ego ies:
1. Real expe imen in a es ing en i onmen .
2. Que ies in he wo ld model.
3. Senso in eg a ion o loading.
All he es s we e done in a p i a e Cloud wi h 5 nodes (1
on -end node and 4 compu ing nodes). Each node has
a AMD FX-8320 oc a-co e CPU (wi h i ual ex ensions
enabled) and 8 GB o RAM. They a e all connec ed in e -
nally using Gigabi E he ne bandwid h. Fo he cloud
middlewa e, we decided o make use o Kube ne es +
Docke ( ecen ly adop ed by majo cloud endo s, such
as Google Con aine Engine o Amazon EC2 Con aine
Engine) ins ead o adi ional VM-based p i a e clouds
(such as Opens ack o Eucalyp us). One o he main
easons behind his decision was o a oid he o e heads
de i ed om he i ualiza ion o ha dwa e and ope a ing
sys em laye s [33].
5.1 P oo ‑o ‑concep expe imen
A ew p elimina y expe imen s we e de ised o check
whe he he whole sys em wo ks p ope ly. Fo his pu pose,
only a ew se o objec s we e modeled o an indoo en i-
onmen ; mainly ough shapes o he u ni u e and wo peo-
ple (one in ude and one au ho ized pe son). In one o he
o ices, wo came a de ices we e ins alled o de ec in ude s
Fig. 7 Example o a RGBD
measu emen
Fig. 8 Example o pe cep ion