B eas cance su ge y wi h augmen ed eali y
Ped o F. Gou eia
a
,
b
,
*
, Joana Cos a
a
, Ped o Mo gado
c
, Ronald Ka es
a
, Da id Pin o
a
,
Ca los Ma ioso
a
,Jo
~
ao Anacle o
a
, Ma a Ma inho
a
, Daniel Sim~
oes Lopes
d
,
A lindo R. Fe ei a
a
,
b
, Vasileios Va ou akis
e
,
, My ian hi Hadjicha alambous
e
,
Ma co A. Sil a
g
, Nickolas Papanikolaou
a
, Celes e Al es
a
, Fa ima Ca doso
a
,
Ma ia Jo~
ao Ca doso
a
,
h
a
B eas Uni , Champalimaud Clinical Cen e/Champalimaud Founda ion,A enida B asilia, 1400-038, Lisboa, Po ugal
b
Facul y o Medicine, Lisbon Uni e si y,A enida P o esso Egas Moniz, 1649-028, Lisboa, Po ugal
c
AI4medimaging,Rua do Pa que Poen e, Lo e 35, 4705-002, B aga, Po ugal
d
INESC ID, Ins i u o Supe io T
ecnico, Lisbon Uni e si y,Rua Al es Redol 9, 1000-029, Lisboa, Po ugal
e
Depa men o Mechanical &Manu ac u ing Enginee ing, Uni e si y o Cyp us,Dep . o Mechanical &Manu ac u ing Enginee ing Uni e si y o Cyp us,
Cyp us
Depa men o Medical Physics &Biomedical Enginee ing, Uni e si y College London,Male Place Enginee ing Building, Uni e si y College London, Gowe
S ee , London, WC1E 6BT, Uni ed Kingdom
g
Mic oso Co po a ion (Po ugal),Rua do Fogo de San elmo, Lo e 2.07.02, Lisboa, Po ugal
h
NOVA Medical School, Campo dos M
a i es da P
a ia 130, 1169-056, Lisboa, Po ugal
a icle in o
A icle his o y:
Recei ed 26 No embe 2020
Recei ed in e ised o m
18 Janua y 2021
Accep ed 20 Janua y 2021
A ailable online 27 Janua y 2021
Keywo ds:
B eas cance
B eas conse a i e su ge y
3D b eas model
Augmen ed eali y
abs ac
In oduc ion: Inno a ions in 3D spa ial echnology and augmen ed eali y imaging d i en by digi al high-
ech indus ial science ha e accele a ed expe imen al ad ances in b eas cance imaging and he
de elopmen o medical p ocedu es aimed o educe in asi eness. P esen a ion o case: A 57-yea -old
pos -menopausal woman p esen ed wi h sc een-de ec ed le -sided b eas cance . A e unde going all
s aging and p e-ope a i e s udies he pa ien was p oposed o conse a i e b eas su ge y wi h umo
localiza ion. Du ing su ge y, an expe imen al digi al and non-in asi e in a-ope a i e localiza ion
me hod wi h augmen ed eali y was compa ed wi h he s anda d p e-ope a i e localiza ion wi h ca bon
a ooing (ins i u ional p o ocol). The b eas su geon wea ing an augmen ed eali y headse (Hololens)
was able o isualize he umo loca ion p ojec ion inside he pa ien ’s le b eas in he usual supine
posi ion. Discussion: This wo k desc ibes, o ou knowledge, he fi s expe imen al es wi h a digi al
non-in asi e me hod o in a-ope a i e b eas cance localiza ion using augmen ed eali y o guide
b eas conse a i e su ge y. In his case, a success ul o e lap o he p e ious s anda d p e-ope a i e
ma ks wi h ca bon a ooing and umo isualiza ion inside he pa ien ’s b eas wi h augmen ed e-
ali y was ob ained. Conclusion: B eas cance conse a i e guided su ge y wi h augmen ed eali y can
pa e he way o a digi al non-in asi e me hod o in a-ope a i e umo localiza ion.
©2021 The Au ho (s). Published by Else ie L d. This is an open access a icle unde he CC BY-NC-ND
license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/).
1. In oduc ion
B eas conse a i e su ge y (BCS) combined wi h adio he apy
has become he ea men o choice o he majo i y o women
p esen ing wi h ea ly b eas cance [1]. The aim o BCS is o achie e
op imal long- e m local con ol, wi h ee ma gins a e umo
*Co esponding au ho . B eas Uni , Champalimaud Clinical Cen e/Champalimaud Founda ion, A enida B asília, 1400-038, Lisboa, Po ugal.
E-mail add esses: [email p o ec ed] (P.F. Gou eia), [email p o ec ed] (J. Cos a), [email p o ec ed] (P. Mo gado),
[email p o ec ed] (R. Ka es), da id.pin o@ undacaochampalimaud.p (D. Pin o), ca los.ma[email p o ec ed] (C. Ma ioso), joao.anacle o@
undacaochampalimaud.p (J. Anacle o), [email p o ec ed] (M. Ma inho), [email p o ec ed] (D.S. Lopes), a lindo. e ei a@
undacaochampalimaud.p (A.R. Fe ei a), a ou akis. [email p o ec ed] (V. Va ou akis), hadjicha alambous.my ian [email protected] (M. Hadjicha alambous), madasi@
mic oso .com (M.A. Sil a), nickolas.papanikolaou@ esea ch. champalimaud.o g (N. Papanikolaou), [email p o ec ed] (C. Al es), a imaca doso@
undacaochampalimaud.p (F. Ca doso), ma iajoao.ca doso@ undacaochampalimaud.p (M.J. Ca doso).
Con en s lis s a ailable a ScienceDi ec
The B eas
jou nal homepage: www.else ie .com/b s
h ps://doi.o g/10.1016/j.b eas .2021.01.004
0960-9776/©2021 The Au ho (s). Published by Else ie L d. This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/
).
The B eas 56 (2021) 14e17
excision and a good cosme ic ou come. Due o widesp ead
sc eening an impo an pe cen age o ea ly b eas cance s a e non-
palpable. As a consequence, umo localiza ion is needed o assis
he su geon du ing su ge y. Cu en modali ies o p e-ope a i e
localiza ion o b eas cance lesions a e all in asi e p ocedu es
equi ing imaging guidance: wi e-guided, ca bon a ooing, o
mo e ecen ly, adioac i e seed localiza ion, adio-occul lesion
localiza ion and magne ic seeds [2].
Inno a ions in 3D spa ial echnology and augmen ed eali y
(AR) d i en by digi al high- ech indus ial science ha e accele a ed
explo a o y esea ch in b eas cance imaging. Al hough ex ensi e
li e a u e exis s on 3D fini e elemen s simula ion and b eas issue
modeling, ew pape s ha e add essed b eas magne ic esonance
imaging (MRI) o 3D scan usion o simula e a eal b eas [3e8].
Recen ly, a pa ien -specific 3D digi al b eas model in eg a ing he
b eas o so and umo loca ion was c ea ed and alida ed wi h a
MRI o 3D su ace scan usion algo i hm [9]. The spa ial compu ing
applied o a b eas cance pa ien , me ging digi al and physical
ana omic s uc u es o he b eas wi h umo included in a digi al
3D b eas model, can be isualized h ough AR in he ope a ing
hea e .
The objec i e o his wo k is o epo a clinical case o a b eas
cance conse a i e guided su ge y wi h AR.
2. P esen a ion o case
2.1. Case his o y
A 57-yea -old pos -menopausal woman p esen ed wi h a
sc een-de ec ed le sided b eas cance . In e io pe ia eola
mic ocalcifica ions we e p esen wi h an es ima ed mammog aphy
ex ension o 17 mm associa ed o a 15 mm dys ophic a ea on ul-
asound (US). A 29 mm b eas MRI nodula lesion was obse ed. A
guided acuum-assis ed biopsy wi h clip placemen was pe o med
showing an in asi e b eas ca cinoma o non-special ype (NST),
g ade 2, ER and PR o 100%, HER2 nega i e wi h a Ki67 o 20% (1
agmen in 10), associa ed wi h in e media e g ade duc al ca ci-
noma in si u (DCIS). Bila e al physical examina ion was nega i e o
palpable lesions and axilla y lymph nodes.
2.2. Expe imen al p o ocol desc ip ion o p oduce a pa ien -specific
3D digi al b eas model (phan om model)
Th ee b eas su ace ma ke s (BSM) we e applied on he pa-
ien ’s b eas s (in a-mamma y sulcus) along wi h a 10 cm fiducial
ma ke below he xiphoid p ocess (Fig. 1). These e e ence poin s
we e anno a ed wi h a black pe manen ma ke a he pa ien ’s
skin. A 3D su ace scan o he pa ien was pe o med in he supine
posi ion wi h a ms a 90

deg ees (Fig. 1), cap u ing he size and
shape o bo h b eas s and o so using a Go!Scan 3D handheld by
C ea o mTM. A e 3D su ace scan, fish-oil capsules we e fixed on
he h ee BSM ma ks o MRI acquisi ion. A con as -enhanced MRI
was pe o med wi h he pa ien in p one posi ion and ma ke s in
place. The ins i u ional b eas MRI acquisi ion p o ocol was modi-
fied o include a final 5-min Dixon sequence wi h he phased a ay
o so MRI coil and he pa ien in he supine posi ion wi h a ms
along he body (Fig. 2).
Anno a ion, segmen a ion and olume compu a ion o he MRI
issue po ions (wi h Dixon sequence in he supine posi ion) we e
pe o med and alida ed by wo adiologis s using Ho os R so -
wa e 2.4.0 (b eas con ou , b eas issue including malignan u-
mo and la issimus do si muscle an e io bo de ). B eas su ace
ma ke s we e manually anno a ed on he 3D su ace scan (black
pe manen ma ke ) and MRI da a (fish-oil capsules).
B eas MRI/3D scan usion was accomplished wi h simple
o e lap be ween bo h modali ies, including umo , using BSM o
pe o m spa ial alignmen be ween bo h modali ies (Fig. 3). A
pa ien -specific 3D digi al b eas model was p ocessed and con-
e ed o a poin cloud b eas 3D model, e e ed he ea e as
phan om model (Fig. 5), eady o be uploaded o an AR headse :
Hololens 1.
2.3. Su ge y wi h AR headse
S anda d p e-ope a i e localiza ion wi h ca bon a ooing unde
ul asound guidance was pe o med a ge ing he nodula lesion
and he clip (mic ocalcifica ions). P eope a i e ma kings we e
d awn on he pa ien skin in he s anding posi ion. Du ing su ge y,
an expe imen al digi al and non-in asi e in a-ope a i e localiza-
ion me hod wi h AR was compa ed wi h he s anda d p e-
ope a i e localiza ion wi h ca bon a ooing. The b eas su geon
wea ing an AR headse (Fig. 4), synch onized he pa ien -specific
3D digi al b eas model ( iewed h ough he Hololens) wi h he
eal pa ien lying down in he ope a ing hea e bed, p ojec ing he
umo loca ion inside he le b eas (Fig. 5).
A b eas conse a i e su ge y wi h sen inel lymph node biopsy
h ough an in e io pe i-a eola incision was pe o med, ollowed
by de ec epai wi h local glandula flaps.
Fig. 1. 3D su ace scan o he pa ien in he supine posi ion wi h a fiducial ma ke
(whi e a ow) as a acking e e ence o he Hololens sys em.
Fig. 2. Le b eas cance lesion wi h clip a i ac isible a MRI wi h he pa ien in he
supine posi ion.
P.F. Gou eia, J. Cos a, P. Mo gado e al. The B eas 56 (2021) 14e17
15
Pa hology: 17 mm umo was emo ed consis ing o in e me-
dia e g ade DCIS wi h ee ma gins and one nega i e sen inel
lymph node. No in asi e cance was ound. Final TNM s aging was
pTis pN0(sn)cM0.
3. Discussion
Su geons need o be able o co ela e 2D adiological images
wi h he eal loca ion o he umo wi hin he pa ien s b eas o
su gical planning, bo h p e and pe -ope a i ely. Despi e all p e-
ope a i e and in a-ope a i e localiza ion echniques, accu acy is
s ill no pe ec , and he p ocess equi es addi ional exams and
in asi e p ocedu es in he p e-ope a i e se ing o expe ise in US
echniques in case o an in a-ope a i e se ing.
The diagnos ic b eas cance wo kflow includes di e en im-
aging modali ies acqui ed in di e en posi ions and wi h di e en
b eas comp essions and consequen de o ma ions. As b eas can-
ce s a e diagnosed mo e equen ly a an ea ly s age and mos o
hem a e non-palpable, localiza ion echniques a e equen ly
needed: wi e-guided, ca bon a ooing, adioac i e seed localiza-
ion, adio-occul lesion localiza ion and magne ic seeds a e
conside ed as s anda d. Howe e , among hese echniques, none
has p o en supe io i y in e ms o educing posi i e ma gins [2].
Wo ldwide, e-excision a es ollowing b eas cance conse a i e
su ge y exhibi wide a iabili y. Re-ope a ion a es ange om 0%
o 70% (by indi idual su geons) in he Uni ed S a es [10], om 17%
o 56% in Canada [11], and om 12 o 30% [12] in England.
Se e al s a egies ha e been p oposed o imp o e b eas cance
su gical ou comes. P eope a i e con as -enhanced b eas MRI has
eme ged as a solu ion, bu nei he he COMICE [13] no he MONET
[14] ial ound an associa ed benefi . Addi ional s udies a e needed
o u he exploi addi ional MRI in o ma ion and echniques o be
applied in he su gical hea e [15]. In aope a i e US (as an addi-
ional ool o s anda d localiza ion echniques) has been used
success ully o some ex en o educe posi i e ma gins, he e o e
educing e-ope a ion a es [16,17]. Howe e , he use o in a-
ope a i e US equi es a le el o expe ise beyond he s anda d o
b eas su geons and could also be ime-consuming wi hin he
su gical hea e . These conside a ions s ongly sugges he need o
de elopmen o a no el me hodology o enhance he su geon’s
isualiza ion o umo localiza ion and consequen ly o imp o e
su gical clinical ou comes.
This wo k desc ibes a success ul expe imen al es wi h a digi al
non-in asi e me hod o in a-ope a i e b eas cance localiza ion.
A pa ien -specific 3D digi al b eas model was p ocessed and
iewed h ough AR o guide a b eas cance conse a i e su ge y. In
his case an o e lap o he p e ious s anda d p e-ope a i e ma ks
wi h ca bon a ooing and umo isualiza ion inside he pa ien ’s
b eas wi h AR was ob ained (Fig. 5).
Pe kins e al. [18] p oposed a simila app oach, bu wi h a
di e en me hodology o su gical planning and no sui able o
li e su ge y: hey p ojec ed a b eas MRI 3D holog am egis e ed
wi h he pa ien in he supine posi ion and applied i wi h six
fiducial ma ke s. Ou me hodology inco po a es a b eas MRI o 3D
su ace scan usion in o de o p oduce a comple e pa ien -specific
3D digi al b eas model (phan om model) wi h only one fiducial
ma ke and h ee in a-mamma y b eas su ace ma ke s, used
du ing bo h imaging modali ies egis a ion o op imize image
synch oniza ion. The diagnos ic b eas MRI in p one posi ion is
pe o med wi h a final 5-min Dixon image sequence in he supine
posi ion. The ins i u ional p o ocol needed o be adap ed wi h pa-
ien eposi ioning om p one o supine du ing he MRI egis a-
ion. B eas MRI o 3D su ace scan usion algo i hms wi h pose
ans o ma ion simula ion om p one o supine [9] can po en ially
le e age his echnology wi hou he need o change diagnos ic
Fig. 3. Pa ien specific 3D digi al b eas model a e b eas MRI o 3D su ace scan
image usion wi h umo highligh ed ed on he le b eas .
Fig. 4. In ac ion: su geon wea ing Hololens headse a he su gical hea e .
Fig. 5. - Su geon iew o he phan om model (wi h umo loca ion) synch onized wi h
he pa ien h ough augmen ed eali y. An o e lap is obse ed be ween he wo ca bon
a ooing ma ks (whi e a ows) and umo p ojec ion wi h augmen ed eali y in he
ed colou .
P.F. Gou eia, J. Cos a, P. Mo gado e al. The B eas 56 (2021) 14e17
16
ins i u ional b eas MRI egis a ion p o ocols. A wo s eps p o-
spec i e clinical ial (p e-clinical plus clinical s eps) was designed
and will s a ec ui ing du ing 2021 o add ess his limi a ion.
To imp o e AR use expe ience applied o b eas cance con-
se a i e su ge y, 3D space came a ancho s ins alled in he oo o
he ope a ing hea e could be used o pe o m eal- ime 3D
mapping o he pa ien in he ope a ing hea e , eplacing handheld
3D su ace scan de ices.
3.1. Conclusion
B eas cance conse a i e guided su ge y wi h an AR headse ,
using 3D digi al b eas models, could pa e he way o a digi al,
non-in asi e me hod o in a-ope a i e umo localiza ion. The
p oposed solu ion has he po en ial o imp o e he su geon’s
isualiza ion o he umo while imp o ing pa ien ’s quali y o li e
(no pain, no anxie y, no bleeding).
Funding
This esea ch did no ecei e any specific g an om unding
agencies in he public, comme cial, o no - o -p ofi sec o s.
Fa ima Ca doso epo s pe sonal financial in e es in o m o
consul ancy ole o : Amgen, As ellas/Medi a ion, As aZeneca,
Celgene, Daiichi-Sankyo, Eisai, GE Oncology, Genen ech, Glax-
oSmi hKline, Mac ogenics, Medscape, Me ck-Sha p, Me us BV,
Mylan, Mundipha ma, No a is, Pfize , Pie e-Fab e, p IME
Oncology, Roche, Samsung Bioepis, Sanofi, Seagen, Te a.
Decla a ion o compe ing in e es
No conflic o in e es and financial ela ionships ele an o he
con en o his a icle ha e been disclosed by any o he o he
au ho s.
Re e ences
[1] an Dongen JA, Voogd AC, Fen iman IS, Leg and C, Syl es e RJ, Tong D, e al.
Long- e m esul s o a andomized ial compa ing b eas -conse ing he apy
wi h mas ec omy: Eu opean O ganiza ion o Resea ch and T ea men o
Cance 10801 ial. J Na l Cance Ins 2000;92:1143e50.
[2] Co si F, So en ino L, Bossi D, Sa ani A, Foschi D. P eope a i e localiza ion and
su gical ma gins in conse a i e b eas su ge y. In J Su g Oncol 2013;2013:
793819.
[3] Lee A, Schnabel J, Rajagopal V, Nielsen P, Nash M. B eas image egis a ion by
combining fini e elemen s and ee- o m De o ma ions2010.
[4] Reece GP, Me chan F, Andon J, Kha am H, Ra i-Chanda K, Wes on J, e al. 3D
su ace imaging o he human emale o so in up igh o supine posi ions. Med
Eng Phys 2015;37:375e83.
[5] Bjoe n E, Vasileios V, John HH, S en K, C is ian L, Thomas B, e al. Su ace
d i en biomechanical b eas image egis a ion. P ocSPIE2016.
[6] Va ou akis V, Eiben B, Hipwell JH, Williams NR, Kesh ga M, Hawkes DJ.
Mul iscale mechano-biological fini e elemen modelling o oncoplas ic b eas
su ge y-nume ical s udy owa ds su gical planning and cosme ic ou come
p edic ion. PloS One 2016;11:e0159766.
[7] Salmon R, Nguyen T, Moo e L, Bass B, Ga bey M. Mul imodal imaging o he
b eas o e ie e he e e ence s a e in he absence o g a i y using fini e
elemen Modeling2017.
[8] Du aes M, C oche P, Pages E, G auby E, Lasch L, Rebel L, e al. Su ge y o
nonpalpable b eas cance : fi s s ep o a i ual pe -ope a i e localiza ion?
Fi s s ep o i ual b eas cance localiza ion. B eas J 2019;25:874e9.
[9] Bessa S, Gou eia PF, Ca alho PH, Rod igues C, Sil a NL, Ca doso F, e al. 3D
digi al b eas cance models wi h mul imodal usion algo i hms. B eas
2020;49:281e90.
[10] McCahill LE, Single RM, Aiello Bowles EJ, Feigelson HS, James TA, Ba ney T,
e al. Va iabili y in eexcision ollowing b eas conse a ion su ge y. J Am Med
Assoc 2012;307:467e75.
[11] Po e G, Waga B, B yan H, Hewi M, Wai E, Dabbs K, e al. Ra es o b eas
cance su ge y in Canada om 2007/08 o 2009/10: e ospec i e coho
s udy. CMAJ Open 2014;2:E102e8.
[12] Jee an R, C omwell DA, T i ella M, Law ence G, Kea ins O, Pe ei a J, e al.
Reope a ion a es a e b eas conse ing su ge y o b eas cance among
women in England: e ospec i e s udy o hospi al episode s a is ics. BMJ
2012;345:e4505.
[13] Tu nbull L, B own S, Ha ey I, Oli ie C, D ew P, Napp V, e al. Compa a i e
e ec i eness o MRI in b eas cance (COMICE) ial: a andomised con olled
ial. Lance 2010;375:563e71.
[14] Pe e s NH, an Esse S, an den Bosch MA, S o m RK, Plaisie PW, an Dalen T,
e al. P eope a i e MRI and su gical managemen in pa ien s wi h non-
palpable b eas cance : he MONET - andomised con olled ial. Eu J Canc
2011;47:879e86.
[15] Sa danelli F, T imboli RM, Houssami N, Gilbe FJ, Helbich TH, Al a ez
Beni o M, e al. Sol ing he p eope a i e b eas MRI conund um: design and
p o ocol o he MIPA s udy. Eu Radiol 2020.
[16] K ekel NM, Haloua MH, Lopes Ca dozo AM, de Wi RH, Bosch AM, de Wid -
Le e LM, e al. In aope a i e ul asound guidance o palpable b eas cance
excision (COBALT ial): a mul icen e, andomised con olled ial. Lance
Oncol 2013;14:48e54.
[17] Spick C, Bal ze PA. Diagnos ic u ili y o second-look US o b eas lesions
iden ified a MR imaging: sys ema ic e iew and me a-analysis. Radiology
2014;273:401e9.
[18] Pe kins S, Lin M, S ini asan S, Wheele A, Ha g ea es B, Daniel B. A mixed-
eali y sys em o b eas su gical Planning2017.
P.F. Gou eia, J. Cos a, P. Mo gado e al. The B eas 56 (2021) 14e17
17