A hyb id ugosi y mesos uc u e (HRM)
o ende ing ine hap ic de ail
Víc o Theok is o1Ma a Fai én Isabel Na azo
Depa amen de Llengua ges i Sis emes In o mà ics
Uni e si a Poli ècnica de Ca alunya, Ba celona, Spain
Abs ac
The hap ic ende ing o su ace mesos uc u e ( ine elie ea u es) in dense iangle meshes
equi es special s uc u es, equipmen , and high sampling a es o de ailed pe cep ion o
ugged models. Some app oaches simula e hap ic ex u e a a lowe p ocessing cos , bu a
he expense o ideli y o pe cep ion. We p opose a be e me hod o ende ing ine su -
ace de ail by using image-based Hyb id Rugosi y Mesos uc u es (HRMs), composed o
pai ed maps o piece-wise heigh ield displacemen s and co esponding no mals, which a e
laye ed on op o a less complex mesh, adding g ea e su ace de ail han he one ac ually
p esen in he geome y. The co e o he algo i hm ende s su ace ea u es by modula -
ing he hap ic p obe’s o ce esponse using a blended HRM coa . The p oposed me hod
sol es ypical p oblems a ising a edge c ossings, conca e oldings and smoo hing ex-
u e s i ching ansi ions ac oss edges. By es ablishing a common se o specially de ised
meshes, HRM mesos uc u es, and a ba e y o pe o mance es s, we build a usabili y es -
ing amewo k ha allows a ai and balanced expe imen al p ocedu e o compa ing hap ic
ende ing app oaches. The ial esul s and use es ing e alua ions show he goodness o
he p oposed HRM echnique in he accu a e ende ing o high 3D su ace de ail a low
p ocessing cos s, de i ing use ul modeling and pe cep ion h esholds o his echnique.
Key wo ds: Hap ic Rende ing; Mesos uc u e; Displacemen mapping
1 In oduc ion
Hap ic sys ems p o ide unique and bidi ec ional communica ion channels be ween humans and
i ual en i onmen s in a manne much close o pe sonal physical manipula ion. Hap ic in e -
aces enable di ec in e ac ion wi h compu e -gene a ed objec s, and when coupled wi h an in u-
i i e isual display o complex da a aise applica ions o new le els; hese applica ions include
molecula docking, nanoma e ials manipula ion, su gical aining, i ual p o o yping, machine
assembly and digi al sculp ing.
Email add ess: { heok, m ai en, isabel}@lsi.upc.edu (Isabel Na azo).
1On lea e om Uni e sidad Simón Bolí a , Ca acas, Venezuela
Fig. 1. Sensing a mesos uc u e coa placed on op o a egula mesh
Hap ics d i es he de elopmen o new algo i hms o objec ’s olume and su ace modeling,
olume p ocessing, and he isualiza ion o no el da a s uc u es able o encode shape and ma e-
ial p ope ies. F om single one-poin based, single pe son ope a ion o mul i-poin , mul i-hand,
and mul i-pe son in e ac ion scena ios, i s en icingly ich in e ac i i y is wi hin each o many
compu e g aphics applica ions.
One o he in e es ing applica ions o hap ic pe cep ion is o be able o eel, ho ough a hap ic
de ice, a ia ions in ex u e, oughness and de ail o he su ace being con ac ed. Al hough some
esea ch has been o ien ed in his di ec ion, he e is no con as ing s udy compa ing esul s o
applying di e en echniques o he same models and ex u es. We su mise ha algo i hms ha
ely solely on a dense geome ic ep esen a ion o hap ic collision de ec ion may ac ually de-
g ade accu a e pe cep ion because o inhe en ly dec easing sampling a es. The gene al idea, as
in isualiza ion, would be o keep sampling a es high by simula ing oughness and o he su ace
ea u es wi hou inc easing he geome ic densi y o he model
In his esea ch, we desc ibe ou solu ion o hap ic ende ing o bo h high equency and low
equency de ail, allowing a comple e pe cep ion anging om ine su ace ex u e o majo o-
pog aphic ea u es. We analyze i s ad an ages and disad an ages agains o he hap ic ende ing
echniques. Ou main con ibu ions a e:
(i) A speci ic model and algo i hm o ende ing image-based mesos uc u e su ace de ails map-
ping pai ed displacemen and no mal maps on o unde lying simpli ied geome ies (Algo-
i hm 2 in sec ion 4);
(ii) A blending unc ion o smoo hing heigh /no mal compu a ion a olding edges (in subsec-
ion 4.1) and mesos uc u e ansi ions (in subsec ion 4.2); and
(iii) A ba e y o usabili y es s o e a chosen se o meshes and mesos uc u es, allowing he
measu ing o ea u e quali y pe cep ion a a ying esolu ions (in subsec ion 5).
We achie e accu a e co espondence be ween he isualiza ion o su ace de ail and he hap ic
pe cep ion o i s ine ea u es, wi hou comp omising ende ing a es o ideli y o ouch.
The a icle is o ganized as ollows: in sec ion 2 we p esen ela ed wo k ecen ly done in hap ic
ende ing. Sec ion 3 desc ibes he app oach, i s algo i hm, sui abili y, ad an ages and disad an-
ages agains he wo o he models. In sec ion 5 we de ail he es ing p o ocol o measu ing
use s’ pe cep ion o hap ic p ope ies, desc ibe he es ing meshes and ial mesos uc u es, and
summa ize he esul s. Finally we p esen ele an conclusions and delinea e u u e wo k owa ds
ob aining a gene alized model o highly de ailed mesos uc u e pe cep ion in e y dense meshes
2
wi h e y low pe o mance penal y.
2 Rela ed wo k
The e m hap ic ende ing as de ined by Zilles and Salisbu y [1] is applied o he eal ime compu-
a ion and gene a ion o o ce esponses o use s in e ac ions wi h i ual objec s. Al hough he e
has been some wo k in pseudo hap ics in simula ing su ace p ope ies using common compu e
mice [2], i is mo e common he use o a specialized hap ic in e ace ha exe s a o ce- eedback
esponse. This esul ing o ce is compu ed om a combina ion o o ces and o ques o a gi en
posi ion and o ien a ion o he in e ac ion de ice. Use s can manually na iga e, explo e and eel
he shape and su ace de ails o i ual objec s in he g owing ield o compu e hap ics [3].
Wi h de ice sampling a es s anda dizing in he 1000 Hz ange, as in he Phan om o HAPTIC-
Mas e de ice [4], e icien hap ic-in e ac ion echniques may go beyond he simple de ec ion
o geome ic p imi i es, owa ds allowing eal- ime ende ing o a bi a y su aces o i egula
de ail, con eying spa ial and ma e ial p ope ies. All his wi hou o ge ing i s o he ole as an
use -in e ac ion de ice o high le el e en acquisi ion, ecogni ion o ac ile “icons” and gene al
hap ic use in e aces o HUIs [5].
When used as an aid o na iga ing a space, i s sho ange each equi es space explo a ion
s a egies, such as a mo ing bubble o na iga ion [6], a wo kspace d i con ol allowing pe -
cep ion disc epancies be ween isual and hap ic space [7], o a o ce- illed cons aining mo e-
men [8].Collabo a ionac oss ne wo ksallows simula iono eal- imeac i i ies suchas s e che -
ca ying [9], bu i b ings i s own se o la ency and simul anei y p oblems ha may cause oscil-
la ions in he in e ac ion.
The mos simple hap ic model uses simple su aces based on iangles, a poin -based de ice
and collisions de ec ion, based on Zilles and Salisbu y cons ain s-based hap ic ende ing [1].
A hap ic cu so ep esen ing a o ce- eedback de ice is placed in o a 3D en i onmen , and a
high p io i y e en loop checks whe he i collides wi h he su ace o an objec , a e which
i p oduces a epulsing o ce o a ying di ec ion and magni ude, which physically combines
wi h he o ce exe ed by he use in he hap ic de ice, co ec ing any pene a ion uled ou by
he objec ’s geome y [10]. Using a ay-based ende ing algo i hm wi h he same se up allows
pe cei ing o que and o ce- o que eedback mechanisms [11], while using a hi d objec as a
ex ended p obe allows also ex u e di e en ia ion and shape pe cep ion [12].
The e is also he issue o pe cei ing se e al o he impo an physical p ope ies besides geo-
me y. De ec ing ic ion among objec s is achie ed by ubbing simula ed known ma e ials [13]
agains each o he and hen compu ing he expec ed ic ion o ce using common physical mod-
els. Su ace so ness o elas ici y may be ep esen ed using an a ay o o ce pins unde a lexible
pla e [14] o by modeling i ual mass sp ings a selec ed mesh poin s [15]. Fo ces a e mapped o
he p og ammable pin a ay and he pla e bends acco dingly when p essed, allowing pe cep ion
o ubbe y o spongy su aces. In modeling de o mable o up u ing 3D medical olumes [16],
he hap ic p obe di ec ly modi ies meshed geome ies ep esen ing so issue su aces, ei he by
poin displacemen , ca ing o spli ing. All hese allow using he hap ic de ice as in e ac ion
ool, o explo e 3D medical images [17], o as na iga ional aids o blind use s [18].
3
These e o s choose among se e al al e na i es o modeling and ende ing su aces. G ego y e
all’s H-Collide [19] uses hyb id hie a chical ep esen a ion, consis ing a hash able o uni o m
g ids and ees o igh - i ing o ien ed bounding boxes, whe eas Johnson [20] uses a pu e geo-
me ic app oach o hap ically ende a bi a y polygons using neighbo hood p oximi ies in o de
o educe compu a ional load. Some hap ic echniques and app oaches a e de i ed om analog
isualiza ion echniques, ea ing he hap ic p obe as a “con ac came a” sys em. Mo genbesse
and S ini asan in [21] p opose he al e na i e me hod o o ce shading, wi h oo s in Phong shad-
ing and bump-mapping in isualiza ion. I is de ined in his con ex as modula ing o ce esponse
in he di ec ion o a no mal ec o sampled om a map. I succeeds in p oducing sensa ions o
bumpy eelings o ib a ions in la su aces, bu i is unable o elici accu a e geome ic pe cep-
ion.
A i s e o o measu e hap ic disc imina ion o basic 2D ex u es was he Sandpape Sys em
by Minsky and Lede man [22]. Use s manipula ed a o ce- eedback joys ick o a e se ac oss
sc een pa ches wi h se e al sample ex u es and epo quali a i e oughness di e ences. An a -
bi a y pa ame ic model was used o model he o ce esponse. Sii a and Pai [23] inco po a e
an s ochas ic model o ac ual physically co ec su ace p ope ies o p oduce he app op ia e
ex u al eel, including ic ion and la e al o ces. Cos a and Cu kosky [24] gene a e ac al u-
gosi y p ocedu ally on la su aces and measu e pe cep ion h esholds. A model o measu ing
hap ic p ope ies o eal su aces h ough a poin p obe is de eloped by Kla zky and Lede -
man [25], es ing pe cep ion quali y a ying hap ic p obe sphe ical adius, a e sal speed and
exe ed o ce.
A global p ocedu e o mapping a g ay-scale image as a displacemen map o poin -based hap-
ic ende ing using s anda d ex u e mapping echniques [26] is gi en by Ho e al [27]. I wo ks
only o pu ely con ex objec s o genus 0 (wi h no holes), wi hou any assessmen o ouch e -
ec i eness, sensa ion ideli y o usabili y measu es. Jagnow [28] modi ies mesh su aces using
geome ic displacemen s. Each iangle o a decima ed mesh is enclosed in a squa e slab con ain-
ing a bilinea pa ch. Each pa ch con ols a ine submesh ha is displaced when he hap ic p obe
p esses (o pinches) he bilinea pa ch. The o ce, as usual, is exe ed in he opposi e di ec ion
o he slab’s no mal, app op ia ely in e pola ed ou o i s main e ices. Inadequa e modeling o
subop imal ende ing p oduce ins abili ies in he o ce esponses, as shown in he wo k o Choi
and Tan [29, 30]. I also de ec s addi ional e ec s such as buzzing (high equency esonance
ib a ions due o i s con ac ) and ali eness (pe cep ion o su ace mo emen in igid su aces).
Collisions a e de ec ed agains a coa se geome y and hen agains a second mic ogeome y laye .
The p oblem o inco ec ende ings when a e sing conca e oldings (due o inc us a ions o
adjoining mac ogeome ies) is iden i ied bu no add essed.
A simila app oach o pain ing and sculp ing ex u es on o geome y wi h a hap ic s ylus is
used by Kim e al [31], in which a 2D ex u e is used o p oduce geome y changes in he
unde lying mesh. I also inco po a es su ace o ces such as ic ion and magne ic a ac ion o a
o ce shading p ocedu e ha s i es o keep he hap ic p obe in con ac wi h he su ace.
Po e e al [32] p o ides a simple model o pe cei e he hap ic a ia ion o la ge heigh ield
e ains, e ec ing collisions agains bilinea in e pola ion pa ches co e ing a la ge e ain da ase
(a big single- aced objec ), bu does no add ess objec s wi h many ace s. In a di e en app oach,
O aduy e al [33] use an objec as p obe o sample ano he objec ’s ela i e ic ion by a e aging
he mul iple con ac a eas p oduced when pa ame e ized isosu aces collide. Pene a ion dep hs
a e compu ed om he in e sec ing isosu aces, and a epulsing o ce is compu ed p opo ionally
4
o he highes di e ence.
Fo a mo e ho ough unde s anding o all issues in ol ed in he pe cep ion o hap ic p ope ies,
an ex ended su ey o cu en hap ic ende ing echniques can be ound in Laycock and Day [34].
I should be no ed om he la e e iew ha mos hap ic ende ing app oaches on meshes ha e
elied ei he on s aigh o wa d collisions agains he mesh’s iangles o collisions agains a
NURBS pa ame e iza ion o he mesh, wi h o wi hou o ce shading. As a as shown, he e
has been no sys ema ic ea men o he issues su ounding he use o heigh ield displacemen s
o hap ic ende ing, such as conca e a eas ea men and edge-c ossing smoo hing. Mo eo e ,
he e is a lack o a uni ied es ing amewo k o measu ing quan i a i e and quali a i e di e -
ences among ende ing app oaches using pe cep ion and usabili y ials on s anda d models and
su aces.
In he ollowing sec ions we de elop a new ea men o hap ic pe cep ion o ine de ail, pos ula -
ing a me hod ha d esses iangle meshes wi h image-based composi e mesos uc u es “coa s”,
buil ou o heigh ield displacemen ex u es and no mal maps. These mesos uc u e coa s a e
used o c ea e, enhance o subs i u e su ace ea u es in low, mid and highe equencies, adding
non-exis en de ail a a e y low p ocessing cos . We hen del e in o explaining he se o usabil-
i y es s ha allow us a ai compa ison o ende ing echniques using he same mesh models and
su ace de ails. This allows us o measu e quan i a i e di e ences on pe cep ion, pe o mance,
and sui abili y o su ace ine de ail, and also o de e mine he limi s in which he p oposed
solu ion se es i s pu pose.
3 Models o hap ic pe cep ion o su ace de ails
The simula ion o su ace de ails in e y complex models has no been a p oblem om he
isualiza ion poin o iew since he la e 70’s. Algo i hms such as he use o colo ed ex u es o
bump-mapping a e well known in he li e a u e [35].
In he case o hap ic pe cep ion, as we ha e seen in sec ion 2, any simula ion algo i hm should
be e icien enough o achie e he high equency upda es equi ed by he human sense o ouch
o pe cei e a con inuous su ace.
Ou objec i e is o ind an algo i hm o allow hap ic pe cep ion o su ace de ails in objec s
ep esen ed by iangle meshes, and o compa e i o o he known solu ions.
Based on all p e ious wo ks we can summa ize a axonomy o hap ic de ail ende ing, which
de e mines he pa icula algo i hm o be used.
•Geome ic De ail, ende ing he su ace as de ailed polygonal meshes (Figs. 2(a) and 2(d)),
NURBS, o poin clouds, and de ec ing collisions agains he su aces.
•Su ace Relie De ail, in which a hap ic ex u e is sampled in lieu o he ac ual su ace. On i s
own, hap ic ex u es may be based on no mal o ce maps (Figs. 2(b) and 2(e)) o heigh ields
(Figs. 2(c) and 2( )).
The o ce shading algo i hm [3] uses he no mal ec o a disc e ized su ace poin s o calcula e
he o ce di ec ion and magni ude o be applied o he hap ic de ice when i collides wi h he
5

(a) Rings - Geome y (b) Rings - Fo ce shading (c) Rings - Heigh ield dis-
placemen
(d) C oss - Geome y (e) C oss - Fo ce shading ( ) C oss - Heigh ield dis-
placemen
Fig. 2. App oaches o simula ing su ace de ails in hap ic pe cep ion
iangle [ igu es 2(b) and 2(e)]. By using his hap ic pe cep ion algo i hm and implemen ing i s
bump-mapping isualiza ion as a GPU shade , one can achie e a co ec pe cep ion o su ace
oughness o small heigh di e ences. Since he collision is always de ec ed agains he iangle
o he mesh, an upwa d/downwa d pe cep ion o displacemen om he iangle su ace is no
possible.
We o e below a b ie summa y o an ea lie app oach we de eloped o ende ing indi idual
su ace de ail ou o an unde lying iangle mesh, by building a cons ain -based o ce esponse
agains local heigh ield displacemen s modula ing 6 DoF sp ing/dampe objec s. The me hod,
shown he e as Algo i hm 1, compa es a o ably agains a o ce shading implemen a ion o
ende ing/pe cei ing he same models using equi alen no mal o ce maps o ex u e pe cep ion.
The comple e model and p ocedu e can be ound in [36].
The p ocedu e used o his app oach wo ked as ollows: A sea ch in 3D space o he exac
p obe’s collision coo dina es agains some small ace is subs i u ed by a p ocedu e ha iden i-
ies a collision agains a much la ge iangle, ollowed by a 2D mapping/sea ch o he hap ic
p obe’s posi ion in o he closes su ace de ail in ha iangle. The algo i hm s a s by de e min-
ing, quickly and a a low compu a ional cos , he base iangle being po en ially collided by he
hap ic p obe, gi ing he hap ic ende algo i hm ample ime o sample he app op ia e heigh ield
al i ude, de e mine whe he he e is an ac ual collision poin ( he hap ic in e ac ion p obe is be-
low ha heigh ), and i ha is he case, exe he app op ia e epulsing o ce using penal y-based
o ce compu a ion model.
A bounded p ism is c ea ed o each mesh base iangle Tk=hVk,0,Vk,1,Vk,2i, wi h equal displace-
men s up and down a maximum dis ance mh along each e ex no mal, con aining all possible
heigh ield alues (see igu e 3(a)). The 8 iangles hus c ea ed (2 o each o he 3 quad ila e al
sides, plus he op and bo om iangula lids) sha e he same agging label o he o iginal base
6
Algo i hm 1 Hap ic heigh ield-displacemen ende ing
1: loop
2: Sample hap ic p obe posi ion PH= (xH,yH,zH);
3: De ec po en ial collision wi h a iangle in he mesh;
4: i (∃collision wi h some iangle p ism T) hen
5: {hap ic p obe PHis inside T’s p ism}
6: P ojec PHagains Tob aining su ace poin P;
7: Compu e 2D ex u e coo ds (s, )o Po e T;
8: Sample he heigh ield displacemen Z=H(s, );
9: i (pene a ion =Z−dis ance(PH,P)>0) hen
10: {Posi i e pene a ion, a eal collision}
11: Calcula e o ce F(pene a ion);
12: Apply Fin he no mal −→
No Ta he de ice;
13: end i
14: end i
15: end loop
p2
3w2
w3
i = pi – mh N
wi = pi + mh N
N2
N3
N1
1 2
p3
p1
w1
mh
mh
god-objec ’s posi ion & o ien a ion
(a) T iangle p ism collision a ea
p2
N2
N3
N1
p3
p1
god-objec ’s posi ion & o ien a ion
P obe
p ojec ed poin
Pene a ion
HF Displacemen
(b) Collision poin compu a ion
Fig. 3. Heigh ield collision mapping
iangle, so he iden i ica ion o he ele an mesh iangle is immedia e a e hi ing any side o
he p ism.
As shown in Algo i hm 1, any collision agains a p ism’s ace igge s he hap ic ende ing o
a co esponding heigh ield su ace displacemen map. I a any ime he p obe ele a ion’s om
he iangle descends wi hin he compu ed heigh a ha poin , a epelling o ce is applied o he
hap ic p obe along he ace no mal a he p ojec ed poin , p opo ional o he heigh di e ence (o
pene a ion). This o ces he god-objec (a cons ained p oxy o he hap ic p obe) o con inually
mo e owa ds he su ace, a which poin he o ce ceases o be (see igu e 3(b)). The hap ic
p obe and he god-objec a e kep in sync when allowed by he cons ain sys em.
Heigh ield displacemen s we e con i ed o ha e al i ude ze o on he edges o he base iangle
mesh o insu e C0-con inui y on he edges. This a oids he p oblem o ha ing ex eme heigh
jumps a he iangles’ edge. In he case o con ex olds, simple no mal in e pola ion may a oid
possible ins abili ies when c uising nea he edges, bu his was shown inadequa e o smoo h
ansi ions when sizable heigh di e ences exis ac oss ace bounda ies, and o ally w ong o
holes and conca e olds.
7
4 Mesos uc u e model o hap ic ende ing
We p oceed now o elabo a e on a me hod ha p oposes a global solu ion o he a o e men ioned
p oblems. Ins ead o applying he o ce in he no mal di ec ion o he base iangle Tk, a much
mo e accu a e ende ing app oach is applying he epulsing o ce in he exac di ec ion o he
no mal a he speci ic impac ed su ace poin . In [33] an app oxima e no mal is compu ed om
he pene a ion g adien , which depends on he applied o ce, o que and cu en p obe 6-DoF
S a e. In ou p esen app oach we p ecompu e no mals di ec ly om he heigh ield displacemen
ex u e and s o e i as a no mal map ex u e, c ea ing wha we call a Hyb id Rugosi y Mesos uc-
u e o HRM.
Taking in o accoun he a e sal di ec ion when ouching a su ace, he hap ic poin is pushed in
hedi ec ion o he no mal, and a cons ain sys em combines his epulsionwi h he o ce exe ed
by he use a he p obe, p oducing a change o posi ion and o ien a ion. I is by using heigh ields
in hap ic ende ing ha he pe cep ion o displacemen o e he base iangle can be achie ed. In
ha manne , we enable accu a e hap ic ende ing wi hou incu ing lagged esponses o p ecision
educ ions. This allows o a y su ace sensa ion explo a ion by “coa ing” o “d essing” a mesh
and ende i wi h se e al su ace equencies and elie s. The e o e, ou inpu da a meshes (all
bu one) a e buil o simila -sized iangles, o ocus on he elie pe cep ion pa . The excep ion
is a mesh ha has iangles o di e en sizes o explo ing he limi s o hap ic pe cep ion.
The gene al p ocedu e, shown as Algo i hm 2, uses he al eady explained p isms, wi h an added
wis . The HRM o no mal and heigh ield displacemen uples = [−→
N(s, ),H(s, )] co espond
o one o mo e RGB-α ex u es, wi h he no mal −→
N(s, )=hNx,Ny,Nziha ing he h ,g,bicoo -
dina es, and he heigh ield displacemen alue H(s, )ha ing he hαi anspa ency coo dina e.
The heigh ield-no mal uples may be p o ided as s a ic o p ocedu al 2D, 3D o 4D (+ ime) ex-
u es, allowing o an e en g ea e complexi y o hap ic pe cep ion. The isual pa is ende ed
by mapping he displacemen s using he same heigh ield and no mals, so he e is comple e co -
espondence be ween he hap ic and isual ende e s. F ic ion, iscosi y, magne ism and o he
su ace p ope ies may be easily added and sampled as addi ional en ies on he HRM s uc u e,
equi ing only he modi ica ion o he o ce- esponse acco dingly.
Hap ic esolu ion ge s scaled in sync wi h he cu en isual zoom s a e. Ge ing close o he
es ed objec esizes he hap ic space acco dingly, so al i udes ha pe haps we e no measu able
a lowe zoom le els (“blu ed”) become dis inc and pe cei able a highe esolu ions, and he
ouch pe cep ion o su ace change becomes mo e accu a e.
4.1 Blending hap ic mesos uc u e a he edges
The p esen ed Algo i hm 2 compu es so ansi ions a iangle edges wi h di e en mesos uc-
u es using a simple in e pola ion scheme. Fo each ace in he mesh, we keep ack o neighbo -
hood in o ma ion o all adjoining ace indices. Two aces a e adjoining i hey sha e a leas one
e ex in common. Neighbo hood in o ma ion is ac o ed in when loading he mesh. Since we
a e es ing low-densi y meshes made o simila -sized iangles, his means ha mos e ices a e
sha ed be ween h ee o six aces. When ollowing along he su ace o he mesh, he mesos uc-
u es in a neighbo ing aces may p oduce an ab up opog aphic change a he edge, ha i le o
8
Algo i hm 2 Hap ic mesos uc u e-blended ende ing
1: loop
2: Sample hap ic p obe posi ion PH= (xH,yH,zH);
3: De ec po en ial collision wi h a iangle in he oc ee;
4: i (∃collision wi h some iangle p ism T) hen
5: {The hap ic p obe is inside he p ism}
6: P ojec PHagains Tob aining su ace poin P;
7: Compu e 2D ex u e coo ds (s, )o Po e T;
8: Ob ain α,β,and γba ycen ic coo dina es o Pin T;
9: i (∃α,β,o γ≥1−ρ) hen
10: {We a e wi hin ρdis ance o an edge}
11: AD ←AW ←0; −→
AN ←−→
0 ;
12: o all adjoining iangles io T(Tincluded) do
13: P ojec PHagains i o ob ain su ace poin Pi;
14: Compu e 2D x coo ds (ui, i)o Pio e i;
15: Sample HRM pai [−→
Ni(ui, i),Hi(ui, i)];
16: E alua e weigh unc ion ωi om P,Piand ρ;
17: AD ←AD+ωiHi(ui, i)·;
18: −→
AN ←−→
AN +ωi−→
Ni(ui, i);
19: AW ←AW +ωi;
20: end o
21: AD ←AD/AW;
22: −→
AN ←−→
AN/AW;
23: else {Collision agains a single ace}
24: Sample HRM pai [−→
N(s, ),H(s, )];
25: AD ←H(s, );
26: −→
AN ←−→
N(s, );
27: end i
28: i (pene a ion =AD−dis ance(PH,P)>0) hen
29: {Posi i e pene a ion, a eal collision}
30: Calcula e o ce magni ude F(pene a ion);
31: Apply Fin he no mal −→
AN a he de ice;
32: end i
33: end i
34: end loop
s and will p oduce a sudden o ce change (in magni ude and o ien a ion) in he hap ic de ice. To
elimina e hese ab up jumps, we ollow he ollowing s i ching p ocedu e o blend he ansi ion
among aces.
Heigh ield and no mals close o he edges a e sampled and in e pola ed using a mul i- ex u ing
app oach om he ugosi y mesos uc u e. In Figu e 4(a) we see a schema ic o his heigh ield
s i ching. A band o pa ame ic size ρex ends a bo h sides o each edge. In his a ea we use
an alpha-blending unc ion o combine o e lapping posi ions, heigh s and no mals. This unc-
ion may exp ess any linea o nonlinea blending. We ex end each pa ame ic dis ance o he
iangle’s ba ycen ic coo dina es in his quan i y ρ, say 0.05 (o 5%) o e each HRM. One o
he blending maps o Figu e 4(b) is used hen o compu e an a e aged mesos uc u e ha spans
pa ame ically his ρac oss each edge.
I he p ojec ed poin o he hap ic p obe is inside he ρband o iangle A (in Figu e 4(a)), i
9
s ep o e he su ace, he pe cep ion is clea ly di e en be ween he wo me hods. Wi h he
o ce shading me hod he use only pe cei es esis ance on he going up and a jump going
down, bu no heigh di e ences can be pe cei ed. Wi h he HRM me hod he use pe cep ion
is clea ly be e in his case, because he pa s o he c oss going up and down gi e he eeling
o going up and down wi h di e en heigh on he op o he c oss han on he base.
As a summa y o his compa ison, we can conclude ha he o ce shading me hod can be a good
app oxima ion o modeling an appa en oughness o ma e ial, bu is no su icien o i egula
no mal maps whe e he pe cep ion has o be igh o he ex u e shape we wan o simula e.
This p oblem is sol ed wi h ou HRM algo i hm, which gi es an accu a e sense o he su ace
cha ac e is ics.
(a) C oss (using o ce shading wi h N2) (b) C oss (using HRM H2,N2)
(c) O als (using o ce shading wi h N3) (d) O als (using HRM H3,N3)
(e) Wa s (using o ce shading wi h N4) ( ) Wa s (using HRM H4,N4)
Fig. 8. Hap ic pe cep ion: Fo ce Shading s. HRM
16

5.4 Tes III. Pe cep ion o mesos uc u e wi h simple epea ing pa e ns
This es was de ised o es he lowe and uppe limi s o hap ic modeling and pe cep ion using
he HRM app oach. We chose a simple epea ing ex u e in a egula se a ed pa e n, each idge
wi h a le e ical side and a sloping igh one. The es measu es se e al pe cep ion a iables
ela ed o hap ic esolu ion: How a a e hey spaced? Can he idges be coun ed? How does i
eel when going le - o- igh and back?. Each ial was pe o med on he base mesh Mbusing
HRMs H1,jwi h he same se a ed pa e n a di e en esolu ions (and co esponding p ecom-
pu ed no mals N1,j, see igu es 9(a) and 9(b)).
(a) Coa se se a ed HRM
H1,32,N1,32
(b) Fine se a ed HRM
H1,512,N1,512
(c) Conca e mesh M wi h
HRM H7,N7
Fig. 9. Pe cep ion scaling adjus men o mesos uc u e
Fo each ial, he maximum heigh ield alue ( ha is, he al i ude o he p ism) was modula ed
a 1%, 5%, 10%, 15% and 20% o he a e age leng h o he mesh’ edges, and un in ials
wi h se e al use s. I is o be no ed ha o ce shading ailed mise ably his es , de ec ing jus
undi ec ional ib a ion a highe equencies and shown o be un eliable a bes a lowe ones.
5.4.1 E alua ion o esul s
When we es ed he HRMs, anging he su ace equencies om ew idges o many, only he las
wo showed a pe o mance h eshold. F ec256 is a mesos uc u e ha has an asymme ic se a ed
peak- alleycombina ion epea ed256 imes,and F ec512 isco espondingly doubled.Each we e
es ed up o a a co esponding isual esolu ion o 1 pixel wide o each idge. The esul s hin
o a p ac ical limi on how much geome y a ia ion may be modeled and pe cei ed by he
17
mesos uc u e app oach. Highe han ha is an indica ion ha mo e iangles and ecalcula ed
mesos uc u es a e needed o be e ep esen su ace hap ic de ails. The esul s a e summa ized
in igu e 10(a) and igu e 10(b):
(a) Tes esul s o F ec256 HRM H1,256,N1,256 (b) Tes esul s o F ec512 HRM H1,512,N1,512
Fig. 10. Hap ic pe cep ion o heigh ield ex u es
The solid line in each g aph ep esen he es sample mean and he su ounding shaded a eas ep-
esen an ampli ude o wo s anda d de ia ions a ound each alue. Th ee impo an expe imen al
ac s ha can be ex ac ed om his esul s:
•The e exis s a de ini e egion o he bes pe cep ion o hap ic ea u es, which si s be ween
maximum peaks and alleys o 5%-15% o a iangle’s edge size, wi h a “swee spo ” wi h
op imum pe cep ion a p ism al i ude 10%. The 5%-15% egion holds also o dynamic cha -
ac e is ics, such as sense o di ec ion in he g oo es, and sensing he di e ence be ween going
le - o- igh o igh - o-le as ab up o sloping. Howe e , a he highes ex u e esolu ion,
all es subjec s only el ib a ion wi hou disce ning any sense di ec ion o damping. This is
e lec ed on he s anda d de ia ion in e als a ound each plo . The de ia ion d ops o ze o in
he 5%-15% egion (All es e s epo ed accu a e pe cep ion o su ace ea u es), bu esul s
di e ge a he ends o he scale.
•A small heigh di e ences, he a iabili y in he pe cep ion o idges and di ec ion by es e s
is o be expec ed, since ain ea u es a e no pe cei ed by e e yone.
•Heigh modula ions g ea e han 20% esul ed in g owing ins abili ies in he hap ic de ice,
due o wild and as changes in he no mal di ec ion because o con inuing exe ed o ces in
high e ical walls, and o e shoo ing o ea u es due o eedback kick. This also caused he
a iabili y a he o he end o he plo .
These esul s hin a a p ac ical h eshold on how much geome ic mesos uc u e may be modeled
by his app oach. In one end o he a ia ion scale, mesh zones whe e su ace a ia ion exceeds
15% o a e age edge size a e hus candida es o ine emeshing. In he o he end o he scale,
i a iangle is pe cei ed as less de ailed as he hap ic ex u e dic a es, he hap ic sensa ion may
be enhanced by using he same ex u e sampled a a lowe a e.
On he o he hand, high esolu ion mesos uc u es wi h below- he- h eshold heigh s become pe -
cep ible when zooming on he scene (see igu e 11). The scaling e ec is kep in sync be ween
18
(a) High esolu ion mesos uc u e om
a a (b) High esolu ion mesos uc u e up
close
Fig. 11. Pe cep ion scaling adjus men o mesos uc u e
he isual and hap ic ield o iew, so sensa ion becomes inc easingly de ined when going om
a a ( igu e 11(a)) o nea ( igu e 11(b)). The size o he hap ic p obe is co espondingly educed
so i ollows much mo e accu a ely he alleys and idges in he ex u e. The e e se is also ue,
when going he o he way, ea u es become less pe cep ible.
5.5 Tes IV. Pe cep ion o non-mono onous mesos uc u e
In his es we measu e he abili y o pe cei e de ini e shapes in he hap ic ex u es: A ha d-
edged c oss; a so ex u e o sloping ings, peaks and dep essions; small- o-big wa s o bumps;
a p o uding ea u e in he shape o a coin; a g oo e in he shape o he le e S (see igu e 12).
The objec o his es is he mul i-modal quali y o pe cep ion: how much i co esponds wi h
he isual ep esen a ion and whe he can be “ ollowed along”.
5.5.1 E alua ion o esul s
As can be ex ac ed om he able, e en small sc a ches a e el and ollowed, un il hey become
oo deep and na ow o a p ope ende ing o he hap ic o ces gene a ed. All es e s we e able
o accu a ely de ec he a ge ea u es e en a low esolu ions, so he e is no a iance wo h
epo ing, excep when eaching he 20% h eshold le el, a which poin ins abili y se s in and
pe cep ion deg ades quickly.
5.6 Tes V. Pe cep ion o isual-hap ic dispa i y in a g ada ed mesh
He e we measu ed esolu ion changes in pe cep ion. We map he same hap ic ex u e in o a mesh
(M ) made o ec angula iangles o dec easing size, in o de o es he limi s o pe cep ion,
aliasing e ec s and a ising ins abili ies. We also measu e how hese quali ies change as we zoom
(bo h hap ically and isually) in he mesh.
19
Table 4
Hap ic pe cep ion o ine ea u es in non-mono onous mesos uc u e
% Heigh 1% 5% 10% 15% 20%
S aigh walls 100% 100% 100% 100% uns
yes yes yes yes
Round con ou s 100% 100% 100% 100% uns
yes yes yes yes
G oo es 100% 100% 100% 100% uns .
yes yes yes yes
So slopes 100% 100% 100% 100% 100%
yes yes yes yes yes
Small bumps 100% 100% 100% 100% 83% yes
yes yes yes yes 17% no
5.6.1 E alua ion o esul s
In ial mesh M (Figu e 9(c)), neighbo ing iangles p og essi ely educe hei a ea in hal om
le o igh (heigh is educed by sq (2)/2). Since mesos uc u e emains a he same esolu ion,
he esul ing mapped a eas ac ually double hei densi y om le o igh , and sampling alias-
ing occu s. Sha p ea u es pe cep ible a big iangles become smoo hed a smalle iangles. I
he scene is zoomed in (o ou ) hey become sha pe (o smoo he ) again. A ea u e becomes
unde ec able when he heigh di e ence becomes less han a co esponding isual pixel, jus as
expec ed by he Nyquis limi . In o he wo ds, i a isual di e ence is seen, hen i can be el .
6 Conclusions
We ha e de eloped a as and accu a e me hod o ende ing local hap ic ex u e in iangle
meshes, which allows he use o pe cei e co ec su ace de ails a se e al esolu ions. This
ex ends he use o heigh ield hap ics beyond he usual ield o gigan ic e ain ex u es and
allows pe cei ing highe su ace de ail wi hou modeling hem geome ically. This app oach can
be used o locally mapping elie ex u es in iangula meshes and hap ically ende hem in
eal ime. The me hod e en allows managing LoD in he isual and hap ic esolu ions o close
app oxima ions, and we ha e he added bene i o ha ing a eposi o y o asso ed HRMs. Gi en
ha all HRMs a e unc ions, a p ocedu al HRM i s wi hou any change in ou scheme.
In o de o apply ou me hod o pe cei ing o e layed sc a ches on he su ace [38], we ha e
ex ended i o accep HRMs ha ing pu e nega i e alues, o ep esen in e se heigh ields (see
igu e 12). In hese cases, o ce shading is no able o gi e he co ec pe cep ion because he e
a e neighbo poin s wi h e y di e en no mals which ac ually pushes he hap ic p obe away
om he sc a ch. Ou HRM- ende ing algo i hm allows a co ec pe cep ion o his so o cha -
ac e is ics as well, e en c uising along he g oo es o he sc a ches.
The app oach shows ample sui abili y o modeling and pe cei ing in eal ime e y complex
20
su ace ex u es o a ying equency ou o simple geome ic models such as bones, majo
body o gans, machine assembly pieces and o he s uc u es.
We a e ex ending u he his esea ch by explo ing a supe posi ion o mul iple esolu ion hap ic
ex u es app oaches. This would allow a be e pe cep ion o heigh ield displacemen s whe e
mo e hap ic de ail is needed by simula ing u he ela i ely s eep slopes o zooming in a high
equency ange. Using he esul s ob ained in his esea ch, we a e de ising a p ocedu e o scan
an objec ’s ine geome y om la ge meshed models o small iangles, and eplace i wi h a less
dense mesh o la ge iangles ha cap u es all he pe cep ible su ace equency de ails o he
o iginal model, as a blending con inui y o global mesos uc u e a lases (heigh displacemen s,
su ace no mals and o he p ope ies such as di ec ed ic ion and s ickiness).
Ou model allows adding ma e ial ic ion as a cons an global coe icien , o expanding he HRM
wi h a second 2D ex u e ield whose alue ep esen s a a iable ic ion coe icien a each ian-
gle poin . This will allow o include he added esis ance o ine mic os uc u e su ace p ope ies
in o he model. We expec o measu e pe o mance di e ences be ween using an HRM-based
app oach agains a pu e geome ic model when ende ing hap ic collisions, and ob aining a o-
bus answe o ha ques ion. The app oach uses hap ic impos o s o eplace he nea es objec
geome y, and in some ways is simila o he isualiza ion algo i hm ha Polica po [39] and
Baboud [40] desc ibe o as shading o geome ic objec s using displaced-mapped impos o s,
ei he as an assembly o a wo-sided (back/ on ) map o a six-sided (cube map).
Fig. 12. Example o a nega i e heigh ield
Acknowledgmen s
We wan o hank Iban Lozano o his aluable help on he implemen a ion o shade s. We ex-
end ou hanks o E a Monclús, San iago Mu illo, An oni Xica, and Ma cos Balsa o hei
obse a ions and pa ience du ing he measu ing phase.
This wo k has been pa ially co- inanced by p ojec TIN2004-08065-C02-01 o he Spanish Go -
e nmen (MEC) and FEDER unding, and by p ojec II-021-FA o he UE ALFA Co dial-2 ne -
wo k.
21

Re e ences
[1] C. Zilles, J. Salisbu y, A cons ain -based god-objec me hod o hap ic display, in: P oc. IEEE In ’l
Con . In elligen Robo s and Sys ems, IEEE Compu e Socie y, Washing on, DC, USA, 1995, pp.
31–46.
[2] A. Lécuye , J.-M. Bu kha d , L. E ienne, Feeling Bumps and Holes wi hou a Hap ic In e ace: he
Pe cep ion o Pseudo-Hap ic Tex u es, in: P oceedings o CHI 2004, 2004.
[3] C. Basdogan, M. A. S ini asan, Hap ic ende ing in i ual en i onmen s, in: K. S anney (Ed.),
Handbook o Vi ual En i onmen s, London, Law ence Ea lbaum, Inc., 2002, Ch. 6, pp. 117–1343,
2212–2218.
[4] FCS Con ol Sys ems, The Ne he lands, Hap icMASTER Ins alla ion Manual (Decembe 2002).
[5] K. MacLean, M. En iquez, Pe cep ual design o hap ic icons, in: P oceedings o he Eu ohap ics
2003, Dublin, I eland, 2003.
[6] L. Dominjon, A. Lécuye , J.-M. Bu kha d , G. And ade-Ba oso, S. Richi , The “Bubble” Technique:
In e ac ing wi h La ge Vi ual En i onmen s Using Hap ic De ices wi h Limi ed Wo kspace, in:
P oceedings o he Fi s Join Eu ohap ics Con e ence and Symposium on Hap ic In e aces o
Vi ual En i onmen and Teleope a o Sys ems, 2005.
[7] F. Con i, O. Kha ib, Spanning la ge wo kspaces using small hap ic de ices, in: P oceedings o he
Fi s Join Eu ohap ics Con e ence and Symposium on Hap ic In e aces o Vi ual En i onmen and
Teleope a o Sys ems, 2005.
[8] D. Xiao, R. Hubbold, Na iga ion guided by a i icial o ce ields, in: P oceedings o ACM CHI 98
Con e ence on Human Fac o s in Compu ing Sys ems, Vol. 1, 1998, pp. 179–186.
[9] R. Hubbold, M. Kea es, Real- ime simula ion o a s e che e acua ion in a la ge-scale i ual
en i onmen , Compu e G aphics Fo um 19 (2).
[10] M. A. O aduy, M. C. Lin, S able and Responsi e Six-Deg ee-o -F eedom Hap ic Manipula ion Using
Implici In eg a ion, in: P oceedings o he Fi s Join Eu ohap ics Con e ence and Symposium on
Hap ic In e aces o Vi ual En i onmen and Teleope a o Sys ems, 2005.
[11] J. Ma ín, J. Sa all, Mechanisms o Hap ic To que Feedback, in: P oceedings o he Fi s
Join Eu ohap ics Con e ence and Symposium on Hap ic In e aces o Vi ual En i onmen and
Teleope a o Sys ems, 2005.
[12] M. A. O aduy, M. C. Lin, Sensa ion p ese ing simpli ica ion o hap ic ende ing, ACM SIGGRAPH
2003 / ACM T ansac ions on G aphics 22 (2003) 543–553.
[13] C. Richa d, M. R. Cu kosky, F ic ion Iden i ica ion o Hap ic Display, in: P oceedings o he 1999
ASME IMECE, 1999.
[14] H. Iwa a, H. Yano, R. Kawamu a, A ay Fo ce Display o Ha dness Dis ibu ion, in: P oceedings o
he 10 h In e na ional Symposium on Hap ic In e aces o Vi ual En i onmen s and Teleope a o
Sys ems (HAPTICS’02), 2002.
[15] J. Zhang, S. Payandeh, J. Dill, Hap ic Subdi ision: an App oach o De ining Le el-o -de ail in Hap ic
Rende ing, in: P oceedings o he 10 h In e na ional Symposium on Hap ic In e aces o Vi ual
En i onmen s and Teleope a o Sys ems (HAPTICS’02), 2002.
[16] M. Mah ash, V. Haywa d, High-Fideli y Hap ic Syn hesis o Con ac wi h De o mable Bodies, IEEE
Compu e G aphics and Applica ions (2004) 48–55.
22
[17] E. Vidholm, I. Nys öm, A hap ic in e ac ion echnique o olume images based on g adien
di usion, in: P oceedings o he Fi s Join Eu ohap ics Con e ence and Symposium on Hap ic
In e aces o Vi ual En i onmen and Teleope a o Sys ems, 2005.
[18] K. S. Hale, K. M. S anney, De i ing hap ic design guidelines om human physiological,
psychophysical, and neu ological ounda ions, IEEE Compu e G aphics and Applica ions (2004)
33–39.
[19] A. G ego y, M. Lin, S. Go schalk, R. Taylo ., H-collide: A amewo k o as and accu a e collision
de ec ion o hap ic in e ac ion, in: P oceedings o IEEE Vi ual Reali y Con e ence 1999, IEEE,
1999, p. 38Ð45.
[20] D. E. Johnson, P. Willemsen, Accele a ed Hap ic Rende ing o Polygonal Models h ough Local
Descen , in: P oceedings o he 12 h In e na ional Symposium on Hap ic In e aces o Vi ual
En i onmen and Teleope a o Sys ems (HAPTICS?04), 2004.
[21] H. B. Mo genbesse , M. A. S ini asan, Fo ce shading o hap ic shape pe cep ion, in: P oceedings o
he ASME In l Mechanical Eng. Cong ess and Exposi ion, Dynamic Sys ems and Con ol Di ision,
1996, pp. 407–412.
[22] M. Minsky, S. Lede man, Simula ed hap ic ex u es: Roughness, in: P oceedings o he ASME
Dynamic Sys ems and Con ol Di ision, Vol. 58, 1996.
[23] J. Sii a, D. K. Pai, Hap ic ex u ing: A s ochas ic app oach, in: IEEE In e na ional Con e ence on
Robo ics and Au oma ion, Vol. 1, 1996.
[24] M. A. Cos a, M. R. Cu kosky, Roughness pe cep ion o hap ically displayed ac al su aces, in:
P oceedings o he ASME Dynamic Sys ems and Con ol Di ision, Vol. 69, 2000.
[25] R. Kla zky, S. Lede man, Touch in Vi ual En i onmen s, P en ice-Hall PTR, 2002, Ch. Pe cei ing
Tex u e h ough a P obe (Chap e 10).
[26] A. Wa , M. Wa , Ad anced anima ion and ende ing echniques, ACM P ess, New Yo k, NY, USA,
1991.
[27] C.-H. Ho, C. Basdogan, M. A. S ini asan, E icien poin -based ende ing echniques o hap ic
display o i ual objec s, P esence 8 (5) (1999) 477–491.
[28] R. Jagnow, J. Do sey, Vi ual sculp ing wi h hap ic displacemen maps, in: G aphics In e ace’02),
2002, pp. 125–132.
[29] S. Choi, H. Z. Tan, Pe cei ed ins abili y o i ualhap ic ex u e. i. expe imen al s udies, P esence
13 (4).
[30] S. Choi, H. Z. Tan, Pe cei ed ins abili y o i ualhap ic ex u e. ii. e ec o collision-de ec ion
algo i hm, P esence 14 (4).
[31] L. Kim, G. S. Sukha me, M. Desb un, A hap ic- ende ing echnique based on hyb id su ace
ep esen a ion, IEEE Compu e G aphics and Applica ions (2004) 66–75.
[32] K. Po e , D. Johnson, E. Cohen, Heigh Field Hap ics, in: P oceedings o he 12 h In e na ional
Symposium on Hap ic In e aces o Vi ual En i onmen and Teleope a o Sys ems (HAPTICS?04),
2004.
[33] M. O aduy, N. Jain, A. Sud, M. Lin, Hap ic display o in e ac ion be ween ex u ed models, in: IEEE
Visualiza ion 2004, 2004, pp. 297–304.
23
[34] S. D. Laycock, A. M. Day, A su ey o hap ic ende ing echniques, COMPUTER GRAPHICS o um
26 (1) (2007) 50–65.
[35] J. F. Blinn, Simula ion o w inkled su aces, in: P oceedings o SIGGRAPH’78, ACM, 1978.
[36] V. Theok is o, M. Fai én, I. Na azo, E. Monclús, Rende ing de ailed hap ic ex u es, in: P oceedings
o Second Wo kshop in Vi ual Reali y In e ac ions and Physical Simula ions (VRIPHYS ’05), Pisa,
I aly, 2005.
[37] J. Shankel, Fas Heigh ield No mal Calcula ion, in: Game P og amming Gems 3, Cha les Ri e
Media, Inc., 2002, pp. 344–348.
[38] C. Bosch, X. Pueyo, S. Mé illau, D. Ghazan a pou , A physically-based model o ende ing ealis ic
sc a ches, Compu e G aphics Fo um 23 (3).
[39] F. Polica po, M. M. Oli ei a, J. L. D. Comba, Real- ime elie mapping on a bi a y polygonal
su aces, in: SI3D ’05: P oceedings o he 2005 symposium on In e ac i e 3D g aphics and games,
ACM P ess, New Yo k, NY, USA, 2005, pp. 155–162.
[40] L. Baboud, X. Déco e , Rende ing geome y wi h elie ex u es, in: G aphics In e ace ’06, Quebec,
Canada, 2006.
24