Faculdade de Engenha ia da Uni e sidade do Po o
Compu a ional echniques o au oma ed analysis o
animal issue his ological images
F ede ico A. R. B. Junquei a
Mas e Thesis
In eg a ed Mas e s in Bioenginee ing
Supe iso :
P o . João Manuel R. S. Ta a es
Associa e P o esso o he Mechanical Enginee ing Depa men , FEUP
Co-Supe iso :
Augus o Manuel Rod igues Faus ino
Associa e P o esso o he Pa hology and Molecula Immunology Depa men ICBAS
June 2015
ii
iii
Ag adecimen os
Em p imei o luga que o ag adece ao P o esso João Ta a es pela disponibilidade e
aconselhamen o o necidos ao longo des es úl imos meses. Sem a sua o ien ação es e abalho
não se ia possí el. Que o ag adece ambém ao P o esso Augus o Faus ino pela sua
disponibilidade, simpa ia e on ade de ajuda .
Que o ag adece do undo do co ação à Lia, que me acompanhou a 100% nes a jo nada
longa e nunca me deixou pe de o No e. Sem o seu ca inho, mui a paciência e apoio
incondicional es e abalho não es a ia comple o.
À minha mãe, ao meu pai, à minha i mã, ao meu i mão e a minha a ó pelas ga galhadas,
ca inho e odo o apoio nes a e apa inal.
Ao Ped o pelos bons conselhos e omb o amigo semp e que p ecisei.
Ao companhei os da ese Mo gana, Jessica e Rica do po me ou i em nos momen os de
s ess.
i
“E e ybody has a plan un il he ge s punched in he ace.”
Myke Tyson

i
ii
Resumo
O es udo de ecidos celula es o nece uma incon es á el on e de conhecimen o e
comp eensão ace ca do co po humano e do ambien e que o odeia. Acede a es a in o mação
é, po an o, c ucial pa a de e mina e diagnos ica uma g ande a iedade de pa ologias,
de e á eis somen e ao ní el mic oscópico. A his ologia desempenha um papel impo an e na
obse ação de células e suas ca ac e ís icas ana ómicas, e igualmen e pa a o diagnós ico clínico
de pa ologias in ol endo uma ano mal con o mação celula . Nas imagens his ológicas,
algo i mos de segmen ação semi-au omá icos ou au omá icos são capazes de sepa a e
iden i ica es u u as celula es de aco do com as suas di e enças mo ológicas. Es es algo i mos
de segmen ação são a p imei a abo dagem a sis emas de isão compu acional e, no que espei a
à his opa ologia, o diagnós ico au omá ico de imagens his ológicas. Como as amos as
his ológicas êm uma espessu a eduzida, as ca ac e ís icas olumé icas são quase
impe cep í eis, co eespondendo a pe das de in o mação aliosas, p incipalmen e opog á icas
e olumé icas, c í icas pa a um co ec o diagnós ico.
Consequen emen e, a combinação de algo i mos de segmen ação e econs ução 3D
aplicados a da ase s de imagens his ológicas o necem uma maio in o mação ace ca da
pa ologia analisada e es u u as mic oscópicas, des acando egiões ano mais.
Tendo is o em conside ação, o p esen e abalho ocou-se em desen ol e algo i mo
compu acional au omá ico capaz de ealiza econs ução 3D de supe ícies de ecidos
ele an es em secções his ológicas 2D. Uma p imei a abo dagem oi desen ol ida ocada em
des aca as es u u as ele an es nas secções de ecido. Depois, um es udo ei o com base em
algo i mos de egis o de imagem oi le ado a cabo pa a descob i qual a me odologia mais
indicada pa a alinha as secções p o enien es de da ase s de imagens. Combinando os melho es
mé odos de p ocessamen o e egis o de imagem ( esul ado DICE pa a caso 1: ; esul ado DICE
pa a caso 2: ; esul ado DICE pa a caso 3: ), a aliados em abo dagens p é ias, em conjun o com
um algo i mo de econs ução 3D oi possí el uma ep esen ação olumé ica das es u u as de
ecidos pe inen es do da ase de imagens alinhadas.
iii
ix
Abs ac
The s udy o cellula issues p o ides an incon es able sou ce o in o ma ion and
comp ehension abou he human body and he su ounding en i onmen . Accessing his
in o ma ion is, he e o e, c ucial o de e mine and diagnose a wide a ie y o pa hologies
de ec able only a a mic oscopic scale. His ology plays an impo an ole in he obse a ion o
cells and hei ana omical ea u es, and so o clinical diagnosis o all he pa hologies in ol ing
abno mal cellula con o ma ion. In he his ological images, semi-au oma ed o au oma ed
segmen a ion algo i hms a e able o sepa a e and iden i y cellula s uc u es acco ding o
mo phological di e ences. These segmen a ion algo i hms a e he i s app oach o
compu a ional ision sys ems and, conce ning his opa hology, he au oma ed diagnose o
his ological images. Since he his ological samples a e hin, he olume ic ea u es a e almos
unno iceable, co esponding o losses o aluable in o ma ion, mainly opog aphical and
olume ic da a, c i ical o a co ec diagnos ic.
Hence, he combina ion o segmen a ion and 3D econs uc ion algo i hms applied o
his ological image da ase s p o ides mo e in o ma ion abou he analysed pa hology and
mic oscopic s uc u es, highligh ing abno mal a eas.
Taking his in o conside a ion, he p esen wo k ocussed on de eloping an au oma ic
compu a ional algo i hm capable o pe o ming he 3D su ace econs uc ion o ele an issue
s uc u es o 2D his ological slices. A i s app oach was de eloped ixed on highligh ing he
ele an s uc u es om he issue sec ions. A e ha , a s udy on image egis a ion algo i hms
was conduc ed o ind he mos sui ed me hodology o align he slices om his ological image
da ase s. Combining he op-pe o ming image p ocessing and egis a ion me hods (DICE sco e:
0.9267±0.0337 o Case 1; 0.9367±0.0356 o Case 2; 0.9683±0.0283 o Case 3), e alua ed in
he p e ious app oaches, wi h a 3D su ace econs uc ion algo i hm i was possible o calcula e
a olume ic ep esen a ion o he pe inen issue s uc u es om he aligned image da ase .
x i

x ii
Lis o Tables
Table I. Cha ac e iza ion o he h ee image da ase s s udied in his wo k. ..................... 38
Table II. Table con aining he mean DICE sco es and he elapsed imes o he in ensi y-
based egis a ion implemen a ion, wi h di e en ypes o ans o ma ion ( igid,
simila i y and a ine) and models ( e e ence slice, pai wise), on images om bo h
op-pe o ming p e-p ocessing me hods (YCbC colo ans o ma ion and CLAHE-
ed). ..................................................................................................... 66
Table III. Table con aining he mean DICE sco es and he elapsed imes o he ea u e-
based egis a ion implemen a ion wi h pai wise model on images wi h bo h
smoo hed and non-smoo hed image no maliza ion p e-p ocessing. ......................... 69
Table IV. Table con aining he mean DICE sco es and he elapsed imes o he in ensi y-
based non- igid egis a ion implemen a ion, wi h wo di e en algo i hms (B-spline
and Demon) and models ( e e ence slice, pai wise), on images ob ained om CLAHE-
ed p e-p ocessing me hod. ......................................................................... 71
Table V. Table con aining he mean DICE sco es and he elapsed imes o he in ensi y-
based igid egis a ion me hod implemen a ion, on wo di e en image da ase s
(Case 1 and 2), p e iously p e-p ocessed by CLAHE- ed algo i hm. ......................... 73
Table VI. Table con aining he mean DICE sco es and he elapsed imes o he in ensi y-
based igid egis a ion me hod implemen a ion, on all he image da ase s (Case 1, 2
and 3), conside ing wo di e en g ayscale images (hema oxylin and eosin images),
ob ained h ough p e-p ocessing (CLAHE- ed and YCbC ans o ma ion) and
segmen a ion (s ain decon olu ion algo i hm). ................................................. 77
x iii
xix
Abb e ia ions and Ac onyms
CNS
Cen al Ne ous Sys em
CT
Compu e Tomog aphy
ECM
Ex acellula Ma ix
H & E
Haema oxylin and Eosin
HSV
Hue – Sa u a ion - Value
ITK
Insigh Segmen a ion and Regis a ion Toolki
µm
Mic ome e
mm
Millime e
MRI
Magne ic Resonance Imaging
nm
Nanome e
PCA
P incipal Componen analysis
PET
Posi on Emission Tomog aphy
PNS
Pe iphe al Ne ous Sys em
3D
Th ee Dimensions
2D
Two Dimensions
RGB
Red – G een - Blue
CLAHE
Con as Limi ed Adap i e His og am Equaliza ion
OD
Op ical Densi y
xx
1
Chap e 1
In oduc ion
1.1 - Mo i a ion
His ological s udies p o ide an impo an help in he unde s anding o some complex
pa hophysiological p ocesses conce ning diseases a he cellula scale. These s udies a e
conside ed he gold s anda d o assessing he na u al esponse o a cellula issue in ace o a
pa hology o he apeu ic in e en ion (Chak a a y, Bedell e al. 2008). To p oduce
his opa hology slides, a a he complex p o ocol mus be execu ed in ol ing a subs an ial
amoun o human labo and in o ma ion p ocessing (Randell, Ruddle e al. 2012).
Al hough in- i o imaging echniques, such as he MRI and PET, assess ana omical and
pa hological in o ma ion wi hou in asi e p ocedu es, hey equi e ex ensi e alida ion when
compa ed o his ological ex- i o examina ions (Chak a a y, Bedell e al. 2008).
Visual in e p e a ion, he co e o mos medical diagnos ic p ocedu es and he inal
diagnos ic decision o cance and o he diseases, is based on issue examina ion. This me hod
equi es a long ime, in ensi e manual labo o p oduce iable esul s and p esen s a sampling
bias ha p omo es in a- and in e - e iewe disc epancies when analysing his ological issues
(Se el, Kong e al. 2009). Thus, i is clea he need o au oma ed p ocesses conce ning
mo phology diagnos ics in medicine, o imp o e he diagnos ic accu acy and p o ide a as and
eliable second opinion o his opa hologis s. Au oma ed sys ems can educe human ac o
mis akes and inc ease he speed o diagnos ic p ocesses (Nedz ed, Belo se ko sky e al. 2005).
The olume ic da a analysis om ele an issue s uc u es isible om 2D his ological
slices is no o en a s aigh o wa d p ocess, equi ing a g ea amoun o expe ience om
his opa hologis s (Koshi, Holla e al. 1997). The e o e, h ee dimensional econs uc ion o
issue samples a a mic oscopic esolu ion e eals signi ican po en ial o imp o e he s udy o

In oduc ion
2
disease p ocesses when s uc u al o spa ial modi ica ions a e in ol ed (D. 1978). The
combina ion o 3D image econs uc ion me hodologies wi h s aining echniques p o ides a
be e unde s anding on unc ional in o ma ion conce ning he cellula s uc u es (Robe s,
Magee e al. 2012).
Hence, he e is an u ge o de elop ully au oma ed app oaches o issue analysis in
his ological sec ion images, combining he bes compu a ional me hods o p oduce an accu a e
and eliable 3D econs uc ion algo i hm, enhancing his way he medical s udy and clinical
diagnos ic o a ious diseases. The implemen a ion o hese algo i hms could p o ide a be e
insigh in o he in ica e spa ial ela ions be ween he s udied cell issues and su ounding
issues.
Wi h his in mind, a compu a ional amewo k was de eloped in his s udy, composed by
se e al image p ocessing and egis a ion algo i hms and culmina ing in a 3D issue
econs uc ion me hod. To accomplish his, a p e ious li e a u e e iew was pe o med on he
mos sui ed me hods o pe o m each ask, as well as his ological image no ions, c ucial o
co ec ly analyse he image da ase s es ed o alida e he de eloped algo i hm, and also o
de ine he ele an issue s uc u e o be econs uc ed.
1.2 - Objec i es
The p esen s udy aimed, p ima ily, he e iew and e alua ion o cu en ly implemen ed
image p ocessing and egis a ion echniques in he li e a u e, anging om s anda d p e-
p ocessing me hods o complex image egis a ion amewo ks, and ecen ly de eloped
algo i hms o his ological image analysis.
Wi h all he concep s and in o ma ion ga he ed om he li e a u e e iew, i was possible
o pu sue he main goal o his s udy, which consis ed in he de elopmen o a comple ely
au oma ic compu a ional amewo k, capable o pe o ming accu a e his ological image
alignmen and 3D econs uc ion o issues. The e o e, p o iding de ailed olume ic
in o ma ion conce ning ele an issues ea u es, unob ainable h ough 2D con en ional slice
analysis. The objec i es behind each s ep de eloped o he inal amewo k, a e explained
below.
Image p e-p ocessing – i s s ep, de eloped o accomplish he highes colo
di e en ia ion be ween issues wi h dis inc s ains, wi h se e al con as enhancemen
me hods being es ed.
Image segmen a ion – aiming o p o ide he mos accu a e disc imina ion be ween
di e en issue s ains and/o o he in e es ing s uc u es in he his ological images,
p e iously p e-p ocessed.
Image egis a ion – o he implemen a ion o his s ep, he g ea ocus was o achie e
he mos co ec slice alignmen , hus, mimicking he o iginal disposi ion o his ological
issues in na u al condi ions, be o e he issue p epa a ion p ocedu e.
3
3D su ace econs uc ion – his inal p ocedu e enables he in e pola ion and su ace
econs uc ion o he p e iously p ocessed and aligned s ack o slices. The main objec i e
wi h his s ep is o p o ide de ailed h ee dimensional insigh o e commonly s udied 2D issue
s uc u es, such as neoplasic issues.
1.3 - Documen S uc u e
The p esen wo k is di ided in 4 chap e s, besides he in oduc ion. A b ie desc ip ion
on he con en s and subjec s add essed in each one is p o ided below.
Chap e 2 - Li e a u e Re iew: On his chap e , an in oduc ion o his ology and i s
impo ance as a way o assess cellula esponses o pa hogens o ea men s a a mic oscopic
le el is add essed, as well as he s anda d his ological issue sample p epa a ion, o
mic oscopic obse a ion. This las opic is also discussed, including s anda d and ecen
echniques o pe o m mic oscopic obse a ion on his ological issue sec ions. In his chap e ,
i is also p esen ed a e iew on bo h s anda d and ecen ly de eloped image p ocessing
me hods, including p e-p ocessing echniques, segmen a ion me hods and egis a ion
algo i hms, implemen ed on s ained his ological images.
Chap e 3 – Me hodology: This chap e desc ibes he implemen ed me hodology in his
p ojec , app oaching i s he p e-p ocessing echniques es ed, ollowed by image egis a ion
amewo ks and culmina ing in he inal wo k low. The his ological image da ase used o es
he de eloped compu a ional amewo k is also desc ibed in his chap e .
Chap e 4 – Resul s and Discussion: In his chap e , all he esul s ob ained ei he om
p e-p ocessing me hods, segmen a ion o egis a ion me hodologies a e p esen ed and
sys ema ically discussed.
Chap e 5 – Conclusion: This las chap e , comp ehends he inal conclusions abou he
esul s, ob ained h ough he pe o med s udy, in addi ion o u u e de elopmen pe spec i es.
1.4 - P incipal Con ibu ions
The p incipal con ibu ions p o ided by he p esen wo k can be subdi ided in o wo
domains: he li e a u e e iew and he de eloped compu a ional amewo k.
The li e a u e e iew p esen ed ep esen s an in oduc ion o esea che s o de elope s
in e es ed and un amilia wi h his ological issue image p ocessing me hods, o undamen al
In oduc ion
4
his ology concep s as well as compu a ional echniques, anging om image p e-p ocessing o
image egis a ion algo i hms, sui ed o his ological issue samples. Fu he mo e, i also
p esen s cu en s a e o he a me hodologies o image egis a ion, implemen ed in
his ological image da ase s, which ob ained success ul slice alignmen esul s.
The p e-p ocessing p elimina y e alua ion on colo space ans o ma ion me hods, bes
sui ed o enhance colo con as in s ained issue images, is comple e and he esul an image
examples p esen ed in his documen p o ide a g ea insigh on hese simple algo i hms o
inc ease RGB colo con as , no only applicable o his ological slices. The CLAHE- ed echnique
was concei ed o he p esen wo k, and i was p o en o be he bes p e-p ocessing me hod
o enhance s ain con as in H & E his ological images, o igina ing accu a e issue segmen a ion
esul s, wi h simple clus e ing me hods (kmeans).
The s ain decon olu ion algo i hm, despi e being based on p e ious wo ks on he
li e a u e, he compu a ional amewo k ha enabled au oma ic s ain disc imina ion was
concei ed and de eloped in his p ojec . This was accomplished using simple echniques and
he esul s ob ained we e consis en o mos o he es ed his ology slices.
The en i e compu a ional amewo k p oposed in his p ojec enables he econs uc ion
o a h ee dimensional olume based on eal his ological issue s uc u es. I was achie ed
h ough he implemen a ion o compu a ionally cheap algo i hms, and we e ob ained highly
de ailed olume ic ep esen a ion, no only o di e en ly s ained issues bu also, in some
cases, he accu a e econs uc ion o neoplasic issues, p esen in he conside ed his ological
image da ase s.
5
Chap e 2
Li e a u e Re iew
This chap e p esen s he essen ial concep s equi ed o unde s and he opics unde
s udy as well as all he esea ch done so a on he subjec .
Fi s ly, i will be p esen ed an o e iew on he his ological concep s conce ning he
cu en labo a o y app oach o acquisi ion o samples, as well as he ele an ea u es o he
di e en ypes o cellula issues, since hese images om he issue samples ep esen he
case s udy o his p ojec .
A s udy on he mos sui able compu a ional me hods o p ocess and ex ac in o ma ion
om he his ological images is e iewed and analyzed u he in his chap e . This li e a u e
e iew culmina es in he p esen a ion and analysis o he mos accu a e 3D econs uc ion
algo i hms o biological images, ega ding he u u e econs uc ion o ce ain ele an
po ions o cellula issues in he his ological samples.
2.1. His ology
His ology is he science ha is de o ed o s udy he de ailed mo phology o cells and
issues conce ning he way in which hese cons i u e he di e en o gans in he body, a a
mic oscopic le el. The me hods implemen ed by his ologis s equi e he s udy o li ing cells
ou side he condi ions in which hei de elopmen is na u al, imposing a con olled
en i onmen (Junquei a and Ca nei o 1987).
His ological s udies p o ide an impo an help in he unde s anding o some complex
pa hophysiological p ocesses conce ning diseases a he cellula scale. Since hese s udies a e
also undamen al o e alua e he pe o mance o new he apies and d ug agen s, hey a e
conside ed he gold s anda d o assessing he na u al esponse o a cellula issue in ace o a
Li e a u e Re iew
12
highligh ing mos o he signi ican cellula s uc u es in he issues. This echnique
s ains he cell nucleus and o he acidic s uc u es in blue o iole (haema oxylin) and
in pink he cy oplasm and collagen (eosin). O he echniques enable he con as o
mo e speci ic issue s uc u es o o ganisms by ecu ing o his ochemical eac ions,
and also, in he case o immunohis ochemical s ains, o assess he p esence o absence
o a ce ain p o ein. These me hods a e pe o med only when he H & E s aining ails
o accen ua e he con as o he s udied s uc u e in he issue (Junquei a and
Ca nei o 1987, Randell, Ruddle e al. 2012). Rep esen a i e images o he isual
appea ance unde he mic oscope o he e e ed s aining me hods applied on cellula
issues a e p esen ed in Figu e 7.
Figu e 7. His ology pic u e o a se o cells lining a duc s ained wi h H & E (on
he le ), and a his ology image s ained wi h immunohis ochemical echniques
o enhance, in ed, he p esence o he p o ein ac in in he cells (on he igh ).
Adap ed om Leeds Uni e si y His ology Guide (Michelle Peckham 2003).
In he ozen ixa ion, he issues o be analysed a e apidly ozen ( eplacing he chemical
ixa ion and inclusion s eps in he p e ious p o ocol), and hen s ained wi h H & E echnique.
Despi e p oducing lowe quali y slides his me hod acqui es slices in a sho e ime, ideal when
is equi ed a as examina ion o he issue (Randell, Ruddle e al. 2012).
2.1.3. Mic oscopy and His ological Sample Obse a ion
A e he p epa a ion o his ological samples (mo e de ails in sec ion 2.1.2) he
mic oscopic cellula s uc u es p esen in hem a e obse ed unde he mic oscope. In his
sec ion, he p e alen ypes o mic oscopy implemen ed o isualize and analyze hose issue
slices as well as inno a i e me hods o pe o m he obse a ion and diagnos ic o his ological
images a e add essed. The e a e wo majo ypes o mic oscopy de ices, he ligh o op ical
mic oscopes and he elec onic mic oscopes. The mos c i ical ac o conce ning a mic oscope
is i s esolu ion powe o esolu ion limi , which is measu ed by he minimum dis ance be ween
wo pa icles in he image (Junquei a and Ca nei o 1987, Randell, Ruddle e al. 2012).
Func ional and ope a ional de ails conce ning di e en mic oscopes o bo h g oups a e
explained u he .

2.1. His ology
13
The con en ional ligh o op ical mic oscope exhibi s images o he s ained issues
h ough illumina ion, which ans e ses he sample, gene a ed by a ligh sou ce. I is composed
by bo h mechanical and op ical pa s and has a limi esolu ion o 0.2 µm. The op ical pa
comp ises h ee se s o lenses, namely he condense , he objec i es and ocula lenses. The
i s condenses he ligh om he sou ce o he his ological sample, he objec i es collec he
ligh ha c ossed he sample and p ojec s an augmen ed e sion o he ecei ed image, anging
he magni ica ion om 2.5x o 40x, in o he ocula lens also con ibu ing o he inal
magni ica ion in a ac o o 10. The inal magni ica ion is hen, he p oduc o bo h objec i e
and ocula magni ica ion. Howe e , by con en ion he ocula magni ica ion ac o is no
included in image desc ip ions. Besides he no mal ligh mic oscope, op ical mic oscopy also
comp ises o he wo majo ypes o mic oscopes, he con ocal and he luo escence
mic oscopes (Junquei a and Ca nei o 1987, Randell, Ruddle e al. 2012).
Con ocal mic oscopes allow he ocusing o hinne sec ions in he image, a oiding he
obse a ion o o e lapping planes o he issue, ac ha deg ades and educes he image’s
de ini ion. In o de o pe o m his speci ic ocus, he ligh beam ha c osses he his ological
sample is na ow and he issue’s image mus ans e se a small o i ice. Consequen ly, his
se up only allows he ocussed plane o he o iginal image o each he de ec o , blocking all
o he consecu i e planes. Since only a hin sec ion is ocussed a a ime i is possible he h ee
dimensional (3D) econs uc ion by ga he ing all he planes o he analyzed issue, h ough a
compu a ional algo i hm (applica ion la e explo ed in he ollowing sec ions) (Junquei a and
Ca nei o 1987).
In luo escence mic oscopy, he analyzed samples a e ligh ed by a me cu y ligh sou ce
and, by ecu ing o ce ain il e s he wa e-leng h o he p ojec ed ligh can be egula ed.
Ce ain biological s uc u es p esen in he issue sample ha e a ini y o luo escen
subs ances ha when exci ed by he p ojec ed ligh hey answe by emi ing ligh in speci ic
wa e-leng h. Th ough he applica ion o his echnique ce ain biological componen s exhibi
b igh colo s in he obse ed image, being highligh ed om he su oundings (Junquei a and
Ca nei o 1987).
Elec onic mic oscopy is based on he in e ac ion be ween elec ons and he issues
p esen in he sample o be analyzed. Conside ing ha ligh mic oscopes ha e a limi esolu ion
o 0.2 µm, elec on mic oscopy ep esen s a mo e accu a e solu ion, o e ing a mo e de ailed
image o smalle componen s in he s udied issue wi h a limi esolu ion o app oxima ely 3
nm. Nowadays, exis wo ypes o elec on mic oscopes, ansmission and scanning elec on
mic oscopes (Junquei a and Ca nei o 1987).
T ansmission mic oscopes possess a esol ing powe o app oxima ely 3 nm, hus allowing
he de ailed obse a ion o isola ed biomolecules o pa icles 400 housand imes magni ied.
Fo en i e issue samples, he magni ying powe is educed o 120 housand imes, s ill a high
esolu ion when compa ed wi h he op ical mic oscope. The ope a ing mode o his mic oscope
is based on he de ou o elec ons when in con ac wi h magne ic ields analogous o lens’
Li e a u e Re iew
14
ligh e lec ion in he op ical mic oscope. The elec on beam is p oduced upon hea ing a
ungs en ca hode, and, due o a ol age po en ial be ween he la e and he anode, he
elec ons a e accele a ed and ans e se in high speed he mic oscope ube. In he ube, he
beam is condensed h ough an elec omagne ic lens (coils) and in e ac s wi h he issue sample,
a e sing i and being consecu i ely ampli ied by a sequence o magni ying lenses. In he end,
he elec ons each a de ec o ( luo escence pla e) and imp in a black and whi e image o he
analyzed sample. The p in ed g ayscale is done acco ding o he amoun o elec ons ha
c ossed he mic oscope’s column and so, he issue sample. Da ke spo s a e elec on-dense
a eas, meaning ha mo e elec ons a e sed he issue unal e ed, no encoun e ing any
s uc u e (Junquei a and Ca nei o 1987).
Scanning mic oscopes acqui e almos 3D images om he su ace o issues and cells in
he analyzed sample. To pe o m his, he issue is co e ed wi h a me allic coa ing, and a
na ow elec on beam is di ec ed o he sample going h ough he en i e su ace o he issue,
wi hou a e sing i , in opposi ion o he ansmission mic oscopes. The emi ed elec ons
e lec on he su ace and a e collec ed by a de ec o , ampli ying hem and, wi h he
in e en ion o o he elec onic componen s, a signal is p oduced in he o m o a black and
whi e image, simila o he ansmission mic oscope.
The images p oduced by his elec on mic oscopy equipmen can be consul ed in a moni o
o s o ed (Junquei a and Ca nei o 1987).
Examples o biological images collec ed om some o he p e iously e e ed ypes o
mic oscopes a e depic ed in Figu e 8.
Figu e 8. Compa a i e images acqui ed om op ical mic oscopy (a) and elec on mic oscopy (b).
Image (a) is a ligh mic og aph o Ha de ian gland om a neona e Alliga o mississipiensis s ained wi h
Me hyl G een-Py onin Y (b -blood essel; hg-Ha de ian gland and ln-lympha ic node). Adap ed om
(Reho ek and Smi h 2007). Image (b) is a mic og aph o a sec ion o mouse li e s ained in a sa u a ed
solu ion o u anyl ace a e (m-mi ochond ia and p-highly dense RNA pa icles). Adap ed om (Wa son
1958).
2.1. His ology
15
Recen al e na i es o he mic oscopic cu en app oaches ha e been de eloped. In o de
o coun e he ex ensi e ime dispensed in lea ning and accus oming o he mic oscope usage
and conside ing he dec ease in use o hese de ices in medical schools (Randell, Ruddle e al.
2013), hese ins i u ions ha e been using i ual slides (His opa hology slides scanned and
s o ed as digi al images), o eaching pu poses. These slides allow a g ea e in e ac ion
be ween he s uden s and he ele an mo phological ea u es p esen in he isualized issue
(Kuma , Velan e al. 2004).
In Leeds Uni e si y, Randell R. and collabo a o s ha e de eloped a i ual eali y
mic oscope ha consis s o a wall-sized high-de ini ion display (Powe wall) capable o
ende ing gigapixel i ual slides in eal ime (Figu e 9). This sys em p o ides a i e imes
g ea e slide a ea han con en ional mic oscopes wi h equi alen magni ica ion, and since i
has a wall-size is be e sui ed o g oup in e p e a ion. This no el app oach enables s uden s
o coope a i ely in e p e he displayed images, showing a mo e in e ac i e app en iceship. A
complemen a y s udy, pe o med by T eano D. and co-wo ke s, aimed o e i y his new
solu ion o i ual slides analysis as a iable eplacemen o con en ional mic oscopy in he
his opa hologis s’ in es iga ion and diagnos ic ou ine. In ac , he diagnos ic made by
consul ing i ual slices akes 60% longe , mainly due o he conside able amoun o ime spen
o na iga e ac oss he en i e image in he small display size, p o ided by common compu e
moni o s and inadequa e use in e aces. Wi h his in mind, he aim o his s udy was o assess
i by inc easing he display size, using he Powe wall, he diagnos ic would each simila speed
when compa ing o con en ional mic oscopy. The pe o med es in his s udy in ol ed a simple
diagnosis, inding small objec s in he image, a decision abou a lymph node and sco e a issue
mic oa ay. By using he i ual mic oscope, his opa hologis s pe o med clinical diagnos ics
and all he o he assigned asks in simila imes as when using a con en ional mic oscope
(T eano , Jo dan‐Owe s e al. 2009, Randell, Hu chins e al. 2012, Randell, Ruddle e al. 2013).
Figu e 9. Leeds Uni e si y wall-sized i ual
mic oscope. Adap ed om (Randell, Hu chins e al.
2012).
In he Uni e si y o Sou h Ca olina School o Medicine, acco ding o (Blake, La oie e al.
2003), he ansi ion and implemen a ion o i ual slides and i ual mic oscopes o eaching
Li e a u e Re iew
16
pu poses was pe o med. The his ological slides we e scanned and iewed up o a 400x
magni ica ion ecu ing o he M SID iewe (wa ele -based mul i esolu ion seamless image
da abase, p ope y o Liza dTech (Ho anes, Deal e al. 1999)) and he compu e as a i ual
mic oscope. The s a ed app oach possesses use ul ea u es, including e ec i e mic oscope and
elescope unc ions p o iding g ea e e sa ili y o issue sample s udy and inc eased speed in
localizing he s uc u es o in e es , when compa ed o he con en ional mic oscope.
In ligh o he s a ed, digi al pa hology p omises in e es ing ad an ages, bo h in e ms o
e iciency and sa e y conside ing con en ional mic oscopy p ocedu es (Randell, Ruddle e al.
2012). Po en ial ad an ages associa ed o a digi al sys em eside in he possibili y o ale
his opa hologis s abou he p esence o new slides o cases o be analyzed (simila o he
wo k low in adiology diagnos ics) and p o ide an easie coope a ion be ween echnicians when
in es iga ing a pa icula case. The la e is ex emely impo an in he wo k low o specialis s,
since his digi al me hod allows a as e and sa e way o sha e mic oscopic isualiza ions o
issue samples wi h o he specialis s, om o he labs and also coun ies, o ob ain second
opinions, an ex emely impo an p ocedu e o ensu e a lawless diagnos ic. Wi h he digi al
p ocedu e, slides can be simul aneously sen o se e al his opa hologis s and, since he physical
anspo a ion o hose slides is inexis en , he e is a educed isk o losing o mixing hem,
hus a oiding an e oneous diagnos ic (Della Mea, Demichelis e al. 2006, Gilbe son, Ho e al.
2006, Nakhleh 2008). Also wi h he pu pose o p o iding a as and accu a e second opinion
o doc o s and his opa hologis s, se e al compu a ional me hods a e being de eloped o
p ocess and analyze digi al issue images. Some o hose me hods a e in oduced in ollowing
sec ions.
2.2. Image P ocessing
17
2.2. Image P ocessing
The i s app oach in o de o acqui e isual ea u es and in o ma ion om images, in he
pa icula case o his s udy, om his ological images, in ol es some compu a ional s a egies
cons i u ing he image p ocessing p ocedu e.
The beginnings o image p ocessing ace back o he middle o he 20 h cen u y, when i
s a ed o be applied o imp o e mic oscope image’s quali y, basically h ough equency
il e ing (signal- o-noise a io, con as and image es o a ion me hods). Real de elopmen s
we e made since hen, and he analog image p ocessing was eplaced by digi al image
p ocessing wi h he ad en o powe ul compu e s capable o applying sophis ica ed algo i hms
o la ge images in an accep able amoun o ime (Bonne 2004).
Since he isual in e p e a ion is he co e o mos medical diagnos ic p ocedu es and he
inal diagnos ic decision, o cance and o he diseases, is based on issue examina ion,
medicine ep esen s a la ge applica ion ield o image p ocessing and analysis algo i hms
(Beng sson 2003). Howe e , isual in e p e a ion and e alua ion p esen se e al weaknesses.
Fo pa hologis s, i is a ime-consuming, cumbe some and edious p ocess o analyze a la ge
numbe o issue samples in p ac ice, hus equi ing a long ime and in ensi e manual labo o
p oduce iable esul s. Besides om his p oblem, isual e alua ions can, in many cases, be
subjec o unaccep able in e and in a- e iewe disc epancies (20% disc epancy be ween
cen al and ins i u ional e iewe s, as epo ed by Teo L.A. e al. in (Teo , Spos o e al.
2007)), due o he sampling bias, con i ming ha i ep esen s an e o -p one me hod
(Beng sson 2003, Kong, Se el e al. 2009, Se el, Kong e al. 2009).
To o e come he s a ed weaknesses, es ablished in he cu en ly used isual e alua ion
p ocess, allied o he ac ha digi al images a e g owing in popula i y, compu a ional me hods
eso ing o au oma ed image p ocessing and analysis algo i hms a e being de eloped
(Beng sson 2003, Kong, Se el e al. 2009). The au oma ic p ocessing and analysis o issue
images p o ides eliable da a, accele a es da a acquisi ion p ocess and by allowing digi al
image managemen i can eplace o he e alua ion me hods, mo e expensi e and impossible
o execu e (Cisne os, Co de o e al. 2011).
Au oma ed sys ems can exclude human ac o mis akes and inc ease he speed o
diagnos ic p ocesses. These sys ems ep esen an impo an asse conside ing ha he amoun
o expe ienced specialis s ha conduc a co ec his ological analysis is educed o
concen a ed in big medical cen e s. The e o e, his leads o an accumula ion o cases poo ly
o misdiagnosed, conduc ing o inco ec un imely ea men s and ul ima ely esul ing in
disablemen o dea h. Conside ing he abo emen ioned i is clea he need o au oma ed
p ocesses conce ning mo phology diagnos ics in medicine, o imp o e he diagnos ic accu acy
and compensa e he sca ce numbe o specialis s (Nedz ed, Belo se ko sky e al. 2005).

Li e a u e Re iew
18
The key challenges in his ological image compu a ional analysis a e au oma ed cellula
segmen a ion and classi ica ion in issue images. Ne e heless, due o he complex na u e and
a ie y o his ological images, i is di icul o de elop au oma ic segmen a ion me hods
applicable o any ype o hose images (Nedz ed, Belo se ko sky e al. 2005, Chomphuwise ,
Magee e al. 2011).
The gene al p ocedu e o au oma ic image p ocessing and analysis can be di ided in
se e al s eps, s a ing wi h he acquisi ion o digi al his ology images, which can ange om
di e se esolu ions depending on he applica ion and he size o he biological s uc u e in
s udy, on he his ological sample. The ollowing s ep is he image p ocessing o iden i y he
a ge issues o biological s uc u es in ques ion, comp ising, as a s anda d amewo k, image
enhancemen , image segmen a ion, ea u e ex ac ion and implemen a ion o machine
lea ning algo i hms (Caicedo 2009). To pe o m each o he p e iously e e ed s ages a wide
a ie y o compu a ional me hods can be implemen ed, acco ding o di e en pu poses ( o
example, au oma ion o mass sc eening o his ological specimens o quan i a i e analysis o a
signi ican s uc u e in he issue) (Beng sson 2003).
The image p ocessing and analysis pipeline ha is going o be p oduced in his wo k
consis s o h ee pa icula s eps, he image p e-p ocessing, image segmen a ion and 3D
econs uc ion (in sec ion 2.3). A s a e o he a on me hods o bo h hese s eps is p esen ed
in mo e de ail in u he sec ions.
2.2.1. Image P e-P ocessing
Al hough segmen a ion is he mos impo an s ep in image p ocessing and analysis, i is
unusual o achie e a consis en and use ul segmen a ion using only a single p ocedu e. In o de
o ob ain a success ul segmen a ion, algo i hms ypically apply a cons uc ed combina ion o
me hods, including a wide a ie y o p ep ocessing s eps (Bea e and Lehmann 2006).
To p ocess his ological images, an ini ial p ep ocessing s ep mus be applied o educe he
compu a ional cos s h ough mul i-scale image decomposi ion (Gonzalez and Woods 2008). This
ini ial p ocess p oduces low esolu ion images ha can be analyzed o loca e in e es ing
s uc u es and allow he implemen a ion o o he image p ocessing s eps only on hose
s uc u es’ pixels. The p ep ocessing s ep is mean also o es o e he images, by educing
image noise, low in ensi y con as and in ensi y inhomogenei ies p esen in he his ological
da a. To pe o m his, me hods such as image smoo hing, denoise and enhancemen can be
applied (He, Long e al. 2010). Image smoo hing is commonly pe o med ecu ing o spa ial
il e ing me hods used o emo e image high equency noise. Image denoising me hods a e
implemen ed o emo e image noise p oduced in image acquisi ion and comp ession p ocesses
(Aube and Ko np obs 2006) and image enhancemen echniques a ou an inc ease in
con as be ween he egions o in e es and he backg ound, being he adap i e il e s he
mos commonly employed me hods (Gonzalez and Woods 2008).
2.2. Image P ocessing
19
Conside ing he amoun o manual labo in ol ed in issue samples p epa a ion (sec ion
2.1.2), his p ocess ends o in oduce ce ain ypes o a i ac s ha equi e p ope image
p ep ocessing echniques o be coun e ed. The majo i y o a i ac s ound in his ological
images a e based on o ien a ion di e ences ound in he sec ions moun ed in glass slides,
a iable luminance g adien (depending on he slide egion whe e he issue obse ed), non-
issue noise p oduced by dus o bubbles and s aining a ia ions ( a iable issue hickness and
s ain concen a ions o igina e colo a ia ions in he his ological s ained s uc u es).
The e o e, image p e-p ocessing echniques a e applied o deal wi h acquisi ion a i ac s and
de ec i e his ology sec ions (Mosaligan i, Pan e al. 2006).
Some echniques a e specially designed o deal wi h his ological a i ac s p esen in he
digi al images, namely he de ec i e sec ion exclusion and p incipal componen analysis (PCA)
alignmen .
In o de o imp o e he 3D econs uc ion obus ness de ec i e sec ions ha e o be
iden i ied and emo ed om he egis a ion p ocess. Since all images a e acqui ed wi h he
same magni ica ion, issue sizes in consecu i e images should no su e signi ican a ia ions.
Thus, when a la ge a ia ion is e i ied i is p obably due o b oken o de ec i e sec ions. To
elimina e hem he ele an s uc u e a eas we e compu ed o each image and plo ed agains
sec ion loca ion, using bina y masks (masks con aining in o ma ion abou he issue pixel
loca ion. Tissue pixels iden i ied and s o ed as bina y masks). Spikes in his plo a e po en ial
de ec i e sec ions (Mosaligan i, Pan e al. 2006).
P incipal componen analysis alignmen is used o es ima e issue o ien a ion, acco ding
o p io knowledge o ypical s uc u e a angemen , conce ning he s udied issue. Since issue
o ien a ions a e used o ini ialize egis a ion me hods (sec ion 2.3), by using his echnique,
he likelihood o con e ging o a mo e eliable global solu ion is inc eased (Mosaligan i, Pan e
al. 2006).
The s aining condi ions e i ied in di e en his ological slices su e conside able
a ia ions (Figu e 10). The e o e, a e he being digi ized, images p esen conside able colo
anges. To no malize colo dis ibu ions p esen ac oss he slices, his og am equaliza ion
ep esen s a iable solu ion (Se el, Ca alyu ek e al. 2009). His og am equaliza ion is a well-
known and widely used image enhancemen echnique, due o i s simplici y, high pe o mance
in almos all ypes o images. This echnique is pe o med by emapping o g ey-le els in an
image based on a p obabili y dis ibu ion o he inpu g ey-le els, s e ching he dynamic ange
o he image his og am. Thus, esul ing in o e all image con as enhancemen . The d awbacks
o his me hod a e no iceable in images wi h high and low mean b igh ness. The esul is a
signi ican change in he image ou look, whe eas he pu pose was only o enhance he con as .
His og am equaliza ion is bes sui ed o enhance he edges be ween di e en s uc u es, bu ,
in e u n, educes local de ails wi hin hose s uc u es, p oducing o e enhancemen and
sa u a ion a e ac s (Kau , Kau e al. 2011). The his og am equaliza ion p ocedu e is based on
he assump ion ha he p ocessed image p esen s uni o m image quali y in all egions, and
he e o e, one single g ayscale mapping p o ides simila con as enhancemen h oughou all
Li e a u e Re iew
20
hese egions. Bu , when dis ibu ions o g ayscale in ensi ies a e a iable acco ding o
di e en egions in he image, he p e ious assump ion is in alid. Facing his, an adap i e
his og am equaliza ion echnique, capable o de e mining he mapping o each pixel based on
i s local g ayscale dis ibu ion (su ounding pixels), could signi ican ly ou pe o m he s anda d
me hod. Howe e , when g ayscale dis ibu ion is highly localized, ull his og am equaliza ion
migh no be desi able o ans o m e y low con as images. By limi ing he con as allowed
h ough his og am equaliza ion, he possibili y o wo e y close g ayscales being mapped o
signi ican ly di e en g ayscales, i.e. high slope segmen s p esen in he g ayscale mapping
cu e, can be a oided. Combining he con as limi ing app oach wi h he p e iously men ioned
adap i e his og am equaliza ion me hod, he esul is e e ed o as Con as Limi ed Adap i e
His og am Equaliza ion (CLAHE) (Reza 2004).
Figu e 10. Sequence o his ological images om MKI (Mi osis-ka yo hexis index) cells, a ying in colo
in o ma ion due o s aining di e ences. Adap ed om (Se el, Ca alyu ek e al. 2009).
Fo medical images, colo enhancemen ep esen s a aluable ool o aid in isualiza ion,
de ec ion and segmen a ion o speci ic issue s uc u es. An e ec i e app oach o inc ease
colo con as on an image and main ain i s hue ( he pu e colo ) is o ans o m he RGB (R -
Red, G - G een, B - Blue) colo in o HSV (Hue, Sa u a ion, Value) colo space, modi ying only
he sa u a ion (luminance) alue in he image’s pixels. The ans o ma ion is gi en by:
𝐻=𝑐𝑜𝑠−1×( 1
2[(𝑅−𝐺)+(𝑅−𝐵)]
√(𝑅−𝐺)2+(𝑅−𝐵)(𝐺−𝐵)) (1)
𝑆=1− 3
𝑅+𝐺+𝐵[min⁡(𝑅,𝐺,𝐵)] (2)
𝑉=1
3(𝑅+𝐺+𝐵) (3)
The luminance manipula ion, h ough g ey-le el enhancemen p ocesses, wi hou
a ec ing he o he wo componen s in HSV colo is possible due o he lack o co ela ion
be ween hese componen s. The usual p ocess s a s by pe o ming he colo ans o m o
2.2. Image P ocessing
21
con e he image in HSV colo whe e he luminance o colo sa u a ion can be modi ied,
dis ega ding he o he componen s. Then, he e e se ans o m back o RGB colo is applied
in o de o asce ain he e ec s o he p oduced modi ica ion (Bau is a and Yagi 2010).
Hukkanen J. and co-.wo ke s implemen ed a p e-p ocessing me hod o imp o e he
e iciency o nuclei segmen a ion (mo e in o ma ion in sec ion 2.2.2.) in his ological images.
The p e-p ocessing me hod pe o ms he con e sion o H & E s ained his ological images
o iginally in RGB colo space in o CIE L*a*b colo space. The L componen , he luminosi y
componen , is hen deno ed as a g ey-le el image, which is u he p ocessed o ob ain he
segmen a ion. The L*a*b colo space consis s o a luminosi y laye “L*”, a ch oma ici y laye
“a*” (indica ing he colo loca ion in he ed-g een axis) and a ch oma ici y laye “b*”
(indica ing he colo loca ion in he blue-yellow axis)(Hukkanen, Ha egan e al. 2010).
In (Tabesh, Te e o skiy e al. 2007) is p esen ed a s udy conce ning image ea u es o
cance diagnosis and his ological g ading o p os a e images. The ea u es ep esen ing colo ,
ex u e and mo phological de ails we e combined in a supe ised lea ning amewo k. The i s
s age in his amewo k in ol ed p e-p ocessing echniques, including backg ound emo al and
image his og am ma ching o a e e ence image. The backg ound was iden i ied and hen
emo ed om he analysis h ough colo issue image ans o ma ion, om RGB colo space
in o YCbC colo space (Gonzalez and Woods 2008), and pos e io h esholding (sec ion 2.2.2.)
o he luminance (Y) componen wi h a global empi ically de e mined h eshold. A e his, he
bina y mask con aining he issues o in e es is e ined ia closing and opening ope a ions o
ill gaps be ween issue s uc u es and emo e small a e ac s om he image. A con ex hull
ope a ion (Gonzalez and Woods 2008) is hen, applied o ensu e he in eg a ion o lumens as
issue o in e es , a oiding i s exclusion om he bina y mask. The second p e-p ocessing s ep,
implemen ed in his s udy, consis ed o an his og am ma ching be ween he analysed
his ological image and a e e ence image, h ough he ans o ma ion⁡𝐹𝑟−1[𝐹𝑖(𝑥)], whe e 𝑥 is
he pixel alue in each o he ed, blue and g een channels. 𝐹𝑟 and 𝐹𝑖 a e, he cumula i e
dis ibu ion unc ion o pixel alues o he inpu and e e ence images, espec i ely.
His og am ma ching is pe o med o mi iga e colo a ia ions p oduced by s aining and
illumina ion condi ions, which can a ec segmen a ion e iciency.
The S ain Decon olu ion echnique is a p e-p ocessing me hod, based on colo
decon olu ion, which aims o decon ol e he applied s ains on a ce ain RGB colo image, o
gene a e sepa a e images, whe e each g ayscale image shows he dis ibu ion o a single s ain.
This algo i hm assumes ha he chemical s ains implemen ed o dye he issue slides ollows
he Bee -Lambe Law o abso p ion, which p o ides a loga i hmic ela ionship be ween he
o iginal RGB colo channels and a s ain ma ix. This complex me hodology is p esen ed by bo h
Li e a u e Re iew
28
based one, and i was capable o econs uc ing selec ed umo in asion on s om a
conside able ex end o his ological se ial sec ions (90-500 slices). Th ough his echnique i was
achie ed an ex emely de ailed 3D econs uc ion o in asion o solid umo s, as isualized in
Figu e 13.
Figu e 13. Two iews o 3D econs uc ion o u e ine ce ix ca cinoma umo in asion on s, om
di e en his ological specimens. Adap ed om (B aumann, Kuska e al. 2005).
An ex ension o he p e ious s udy was ca ied ou by Ryk J. and co-wo ke s, whe e he La ge
Image Mic oscope A ay (LIMA), a ib a ome capable o sec ioning issues o 40 mm hickness
slices, is used o sec ion en i e o gans, pa icula ly lungs, in o slice images in o de o es ablish
di ec co ela ion be ween lung pa hology and Compu e Tomog aphy (CT) images. A came a
and a s e eomic oscope moun ed on he ib a ome scan he en i e su ace a ea o he issue,
con olled by a cus om so wa e, esponsible o he en i e p ocess au oma ion. The alignmen
is accomplished h ough he combina ion o a cus om code and he Insigh Segmen a ion and
Regis a ion Toolki (ITK), esul ing in he egis a ion o high magni ica ion and esolu ion
pa hology images wi h he co esponding CT images (de Ryk, Nama i e al. 2004).
Ano he echnique de eloped by (Reho ek and Smi h 2007) aims o he gene a ion o 3D
images om speci ic mic oana omical s uc u es con ained in issue sec ions. The p esen ed
3D econs uc ion echnique no only allows he concu en isualiza ion o mul iple s uc u es
o issues bu also enables he analysis o spa ial opog aphy om he his ological sec ions.
This echnique in ol es, o emos , he isual iden i ica ion and manual delimi a ion o he
egion o in e es ’s posi ion in a 2D sec ion digi al image. All he ma ked digi al images we e
hen aligned o p e ious slices based on iducial landma ks ( e e ence s uc u es in he image),
accomplished by ende ing he second o wo adjacen issue sec ions h ough manual o a ion
o he image on op o he o he slice.
A semiau oma ic me hod was de eloped by (Ku ien, Boyce e al. 2005) o p oduce h ee-
dimensional econs uc ions o in asi e b eas ca cinoma, ecu ing o common labo a o y
equipmen o e alua e he spa ial a angemen o pa enchymal cells. The umou s udied in
his wo k was s ained immunohis ochemically o econs uc wo 3D images, one o no mal
pa enchymal cells and he o he o malignan . The digi al his ological sec ions we e acqui ed
using a mic oscope, a scanne and a came a connec ed o a compu e , and he alignmen was

2.3. Image Regis a ion and 3D Recons uc ion
29
pe o med using a semiau oma ic me hod, allowing manual in e ac ion h ough a g aphical
in e ace. The e e ed me hod uses c oss co ela ion coe icien as an in eg i y slice i
measu e and an au oma ic p ocess based on he Fibonacci sea ch algo i hm (Ramap abha,
Balaji e al. 2012) o achie e an au oma ic alignmen . Finally, he esul an econs uc ed
olume was ob ained using maximum, minimum poin p ojec ion and back o on opaci y
blending. The esul s ob ained h ough his me hod we e dis inc and accu a e 3D
econs uc ed models, con ibu ing o a mo e comp ehensi e and explici isualiza ion (high
esolu ion) o spa ial a angemen o no mal and malignan pa enchymal issues and hei
ela ion o he su ounding issues (Figu e 14). This app oach p o ided an insigh on in asi e
b eas ca cinoma p oli e a ion, unob ainable h ough con en ional isualiza ion o 2D
his ological sec ions.
Figu e 14. 3D econs uc ion o in asi e b eas ca cinoma
immunohis ochemically s ained, illus a ing he spa ial
a angemen o he di e en pa enchymal issues. Adap ed
om (Ku ien, Boyce e al. 2005).
In (Ou selin, Roche e al. 2001) i is emphasized he need o a obus me hod o pe o m
he alignmen o his ological sec ions o 3D econs uc ion. In his wo k, i is p oposed an
in ensi y-based me hod o egis e he image slices, i s by using a block ma ching s a egy
(Jain 1981), allowing he compu a ion o local displacemen s be ween image slices, and hen,
a igid ans o ma ion, es ima ed by hose local measu es. The en i e p ocess is ully
au oma ed and in eg a ed in o a mul i-scale amewo k in o de o imp o e accu acy and
compu a ion imes. The esul s ob ained expe imen ally highligh he capaci y o he p oposed
algo i hm o each sub-pixel accu acy while being able o compensa e la ge displacemen s
be ween slices.
Con en ional 3D his opa hology is limi ed by low esolu ion, ime and di icul y wi h
acqui ing a la ge numbe o images wi h a mic oscope, he absence o a ully in eg a ed sys em
o 3D econs uc ion (Nama i, De Ryk e al. 2007) and, in he case o manually guided 3D
econs uc ion, he ime equi ed o he whole p ocess (Pe ie, Flynn e al. 2002). Conce ning
hose limi a ions, Robe s N. and collabo a o s de eloped a 3D his opa hology so wa e using
Li e a u e Re iew
30
au oma ed i ual slide scanne s o p oduce high- esolu ion digi al images and 3D
econs uc ions o he cellula issue. The so wa e is applicable o any ype o issue, equi es
minimal manual in e en ion, once he slides a e p epa ed, and e eals easonably obus ness
o e a wide a ie y o da a o ype o applica ion. The sys em digi ises au oma ically he i ual
slides, which subsequen ly communica es wi h he image se ing so wa e o align he image
and p oduce he isualiza ion. To pe o m he alignmen i uses a high- esolu ion egis a ion
ollowed by a mul i-le el egis a ion me hod, whe eby he use is able o manually selec ,
zoom and e- egis e he a ea o in e es (Robe s, Magee e al. 2012). The egis a ion
algo i hm p oposed by he au ho s a s by pe o ming a sequen ial slice- o-slice image based
egis a ion, a mul i-s age me hod based on ex ension o phase co ela ion (De Cas o and
Mo andi 1987), which consis s o a i s igid alignmen igno ing scale ha se es as inpu o a
non- igid egis a ion me hod ha di ides he inpu image in o equally spaced squa e pa ches,
indi idually aligned h ough he phase co ela ion. A non- igid B-spline based ans o m is
es ima ed h ough a leas squa es e o minimising me hod and applied o he whole image o
app oxima e a se o poin s om each pa ch. The B-spline is applied a mul iple inc easing
esolu ions and B-spline g id sizes. This egis a ion me hod uses as e e ence a ep esen a i e
cen al i ual slide (gene ally he slice con aining he la ges po ion o he issue o be
s udied). The subsequen images a e aligned o hei neighbou s and hen conca ena ed o o m
a 3D olume ic da ase o he use o isualize, a e he comple ion o igid and non- igid
ans o ms. As p e iously e e ed in his s udy, i was also de eloped a use -in e ace o
p omo e he in e ac ion and use selec ion o sub-a eas o in e es in he image o e- egis e
a highe esolu ion. This was pe o med using he explained non- igid mul iple-le el esolu ion
me hod, p o iding his way sub-cellula accu acy econs uc ions. A e he 3D olume
econs uc ion he de eloped so wa e enables in e ac i e segmen a ion o he olume ic
s uc u es by implemen ing se e al echniques, including a manual me hod called “colo
example h esholding” in which he use de ines a h eshold and selec s a ce ain colo in he
image and he so wa e anno a es all he pixels sha ing RGB colo wi h he selec ed one and
he conside ed h eshold. Ano he app oach implemen s egion g owing algo i hms (E o d
2000) o segmen spa ially connec ed simila pixels (mo e in o ma ion on sec ion 2.2.2), using
he use selec ed poin s as seeds and he h eshold as colo simila i y c i e ia. In he end, i
was implemen ed an iso-su acing using ma ching cubes (Lo ensen and Cline 1987) and mesh
decima ion (code om VTK (Sch oede , Ma in e al. 2003)) o ende he segmen ed olume
and allow he isualiza ion o dis inc pa s o he olume sepa a ely (Magee, T eano e al.
2008, Robe s, Magee e al. 2012).
In (Chak a a y, Bedell e al. 2008) is desc ibed a me hod o 3D econs uc ion o wo
dimensional his ological sec ions om mouse b ain o c ea e olume ic da a, a undamen al
s ep in he analysis o ex- i o da a o alida e in- i o imaging echniques. To achie e he
la e , he econs uc ed olume is di ec ly mapped in o in- i o ana omical MRI olumes, hus
enabling accu a e alida ion o ecen imaging echnologies, as well as he in eg a ion o
his ological cellula o molecula da a in in- i o s uc u al and unc ional da a. The p ocedu e
2.3. Image Regis a ion and 3D Recons uc ion
31
desc ibed in his wo k is an ex ension o a p e ious s udy by Chak a a y e al. (Chak a a y,
Be and e al. 2006) pe o med o he econs uc ion o human basal ganglia and halamus,
whe e he his ological sec ions su e ed i s manual alignmen and colo adjus men , and hen
non-linea co ec ion echniques o enhance he image alignmen . In (Chak a a y, Bedell e
al. 2008) i was used a digi al block ace o pe o m image egis a ion and a olume de i ed
om he a e age o en MRI olumes o mice (Lau, Le ch e al. 2008) o se e as olume ic
e e ence. The me hod s a ed by he au ho s a s wi h he alignmen o his ological sec ions
o he co esponding block ace image. Fi s , a slice- o-slice alignmen o he block ace da a
was execu ed and a ans o ma ion mapping each slice o he nex , con e ging his p ocess o
he cen e slice ( e e ence), was es ima ed. Then, o he his ological da ase i was applied
and es ima ed a linea ans o ma ion o map all he slices in o he co esponding slice in he
block ace econs uc ed olume. Bo h wo dimensional ans o ma ions we e adap ed om he
linea egis a ion echnique p oposed in (Collins, Neelin e al. 1994). Pos e io ly a nonlinea
mo phological co ec ion was implemen ed, consis ing o a slice- o-slice wa ping o co ec
mo phological inconsis encies be ween slices. To pe o m he es ima ion o hese
ans o ma ions he ANIMAL algo i hm (an i e a i e algo i hm ha es ima es a 3D de o ma ion
ield on a la ice o nodes when a sou ce olume is ma ched o a a ge olume) was used
(Collins and E ans 1997). Then, in o de o coun e in ensi y inhomogenei ies be ween
egis e ed slices, possibly p oduced by s aining densi ies and slice hickness i egula i ies ha
can challenge he co ec isualiza ion o he images, a nonlinea in ensi y co ec ion was
pe o med. This p ocedu e i s applies he nonlinea ans o ma ions es ima ed in he
mo phological co ec ion o consecu i e se ies o ou slices, o g an ha all o hem p esen
mo phological consis ency. A e his, each slice was po ioned in o equally sized squa e
pa ches, assuming ha su icien ly small a eas would enable he es ima ion o a i s o de
polynomial scaling ac o o ma ch he join his og ams o each squa e by using a leas - immed
squa es polynomial es ima ion (P ima, Ayache e al. 2001). Once pe o med his p ocess o
each squa e pa ch, a g id con aining he scaling ac o s p e iously es ima ed was in e pola ed
o ma ch he esolu ion o he his ological image, and hen, he p oduced in e pola ion ield
was mul iplied by all he slices o enhance he g ey-le el consis ency. Finally, he his ological
econs uc ed olume was di ec ly wa ped o he s a ed MRI empla e olume e e ence. The
esul s ob ained h ough his ou ine demons a e an e ec i e alignmen o he his ological
olume wi h he MRI empla e. These esul s enhance he ac ha he combina ion o issue
p epa a ion echniques and au oma ed image p ocessing allows o a mo e comp ehensi e,
mul i-modal e alua ion o pa hology o he apeu ic in e en ion e ec i eness in oden models
o CNS disease. The esul s ob ained h ough he s a ed echnique a e p esen ed in Figu e 15
(Chak a a y, Bedell e al. 2008).
Li e a u e Re iew
32
Figu e 15. Volume ic esul s om he econs uc ion o se ial his ological slices acqui ed by Chak a a y
M. and collabo a o s. F om le o he igh : Block ace econs uc ed, o iginal da a s acked wi hou any
co ec ion, econs uc ion model a e slice- o-block ace linea alignmen , esul ing econs uc ion om
nonlinea mo phological co ec ion and econs uc ion a e nonlinea in ensi y co ec ion. Adap ed om
(Chak a a y, Bedell e al. 2008).
Johnson J. and co-wo ke s p esen ed a hyb id landma k/in ensi y-based de o mable
egis a ion algo i hm. This algo i hm applies an i e a i e p ocess by p oducing accu a e
co espondences be ween image s uc u es nea landma k loca ions and elemen s sepa a e
om hem by ma ching co esponding landma ks and image in ensi ies, espec i ely. Fi s a e
egis e ed landma ks dis ega ding in ensi y and hen in ensi y di e ences be ween slices a e
minimized (Johnson and Ch is ensen 2002). Despi e he accu a e esul s, his echnique was
no applied o econs uc 3D images om 2D slices and also e ealed ha when applied o
unclassi ied da a based on landma ks does no lead o eliable egis a ions.
Ano he me hod o au oma ic egis a ion o his ology sec ions was p esen ed by A ganda
C. and collabo a o s, consis ing in a echnique o non-de o mable egis a ion ecu ing o
Sobel ans o ms and segmen ed con ou s. The goal o his p ojec was o accu a ely align
issue sec ions in olume ic da a and also o de ec and ende ele an s uc u es in 3D. The
algo i hm p oposed accomplished his by inding he bes sui ed igid body ans o ma ion
( ansla ion and o a ion) o he images being egis e ed (applied globally), h ough
maximiza ion o a ma ching unc ion based on image co ela ion (applied locally on speci ic
loca ions e ealed by segmen a ion me hods). A mul i esolu ion py amidal app oach was hen
implemen ed, eaching he bes egis a ion ans o ma ion in inc easing esolu ion s ages
(A ganda-Ca e as, Fe nandez-Gonzalez e al. 2004).
In (Mosaligan i, Pan e al. 2006) was p esen ed a mu ual in o ma ion based egis a ion
app oach, ha ing as basis he maximiza ion o mu ual in o ma ion (MI) (Maes, Collignon e al.
1997), an e ec i e simila i y measu e o egis e mul i-modal images when he image
in ensi ies a e no linea ly co ela ed. The p oposed echnique includes ou s ages, including
he ans o m, me ic, op imize and in e pola o phase, in o de o egis e a consecu i e slice
in o a s a iona y image. The ans o m s age is modelled, as p e iously e e ed app oaches,
2.3. Image Regis a ion and 3D Recons uc ion
33
o a igid 2D ans o m, which allows o a ion and ansla ion. To pe o m he op imize and
in e pola ion s age we e inco po a ed a egula s ep g adien op imize (Maes, Vande meulen
e al. 1999) and a bi-linea in e pola ion (Maes, Collignon e al. 1997), espec i ely. The
egis a ion was acqui ed h ough MI op imiza ion mul i esolu ion s a egies, as p o ed in
p e ious s udies ha can p o ide simila obus ness o di ec egis a ion combined wi h an
inc eased compu a ional speed when compa ed o o he mul i esolu ion echniques
(S udholme, Hill e al. 1996, Maes, Vande meulen e al. 1999). This app oach was pe o med
applying 3-le el image py amids (Figu e 16), wi h image magni ica ions o 10x, 20x and 50x.
S a ing wi h he ans o m ob ained o he lowe magni ica ion images a e scaled and
employed o ini ialize he nex highe magni ica ion, epea ing his p ocess un il he highes
esolu ion ( he imp o emen in accu acy o esolu ions abo e 50x do no compensa e he
compu a ional cos s). A e his, a wo-le el op imiza ion (Figu e 17) was execu ed o achie e
a highe MI in he econs uc ed model. This p ocess in oduces a es ic ed ansla ion and
o a ion a ound he PCA ini ializa ion (desc ibed in 2.2.1), al e ing he con e ged solu ions.
This s ep is epea ed un il no imp o emen is e i ied in he mu ual in o ma ion model. The
p esen ed s udy de eloped a mul i- esolu ion MI based egis a ion algo i hm combined wi h a
no el op imiza ion s a egy ha allows no only a educ ion o manual in e en ion in he
egis a ion p ocess bu also a highe chance o ob aining a con e ged global solu ion
(Mosaligan i, Pan e al. 2006).
Figu e 16. Schema ic ep esen a ion o he ITK-based
egis a ion amewo k. The ans o ms pass om a lowe o a
highe esolu ion based on he 3-le el image py amids shown.
Adap ed om (Mosaligan i, Pan e al. 2006).
Figu e 17. Rep esen a ion o he egula s ep g adien descen (A,C) and he wo-le el op imize (B,D)
on 2 slides om placen a image da ase (PCA was applied ini ially as p e-p ocessing), used in (Mosaligan i,
Pan e al. 2006).

Li e a u e Re iew
34
2.4. Key Issues
35
2.4. Key Issues
The p esen sec ion summa izes he con en s add essed in he li e a u e e iew,
highligh ing he undamen al concep s o be e ained.
 His ological echniques a e conside ed he gold s anda d o assessing he na u al
esponse o a cellula issue in ace o a pa hology o he apeu ic in e en ion.
 The exis en undamen al ypes o issue a e epi helial (co e s body su aces),
connec i e (p o ides suppo and connec ion o all o he issues in he body), muscula
(designed o con ac ion) and ne ous (ensu es he communica ion be ween senso ial
cells, CNS and muscles).
 In o de o isualize his ological samples and hei issue s uc u es unde he
mic oscope, a p e ious p epa a ion o he issue comp ising ixa ion, inclusion and
s aining echniques mus be pe o med.
 Digi al his ological slices p o ide nume ous ad an ages when compa ed o con en ional
mic oscopy, namely in he possibili y o ale his opa hologis s abou he p esence o
new slides o analysis and p o ide easie coope a ion be ween echnicians when
in es iga ing a pa icula case (i.e. a second opinion).
 Au oma ed compu a ional sys ems, comp ising segmen a ion and 3D econs uc ion
algo i hms can p o iding a as and accu a e second opinion o doc o s and
his opa hologis s.
 Compu a ional sys ems can also educe he wo kload o his opa hologis s, who pe o m
a a he ime consuming and labo ious ask, subjec ed o high in e and in a- eade
a iabili y.
 Image p e-p ocessing me hods a e applied o he his ological images o educe he
isual noise and enhance he con as be ween he in e es s uc u es and he
emainde issue.
 Segmen a ion is he mos ele an p ocess in image p ocessing and analysis sys ems,
enabling he dis inc ion and iden i ica ion o some pa icula biological issue om he
emainde componen s in he inpu image.
Li e a u e Re iew
36
 Th ee-dimensional econs uc ion o issue samples a a mic oscopic esolu ion e eals
signi ican po en ial o imp o e he s udy o disease p ocesses when s uc u al o
spa ial modi ica ions a e in ol ed.
 Image egis a ion is conside ed an op imiza ion p ocess ha aligns wo o mo e
images, aiming o ind he image ans o m esponsible o he bes alignmen o
closes simila i y o s uc u es o in e es be ween consecu i e images.
37
Chap e 3
Me hodology
In his chap e , he me hodology de eloped and all he algo i hms cons i u ing i will be
ho oughly explo ed, as well as all he image da ase s con aining animal cell issue, es ed o
p o ide he necessa y alida ion o he p oposed amewo k.
Fi s ly, he expe imen al da abase analyzed in his s udy, co dially supplied by he
Pa hology Labo a o y o he Ins i u e o Biomedical Sciences Abel Salaza , is going o be
add essed, including he desc ip ion o all h ee di e en issue da ase s, he p ope ies o he
digi al image acquisi ion de ice and he ele an issue egions o be p ocessed and highligh ed
by he p oposed me hodology.
The second sec ion o he me hodology p esen s an exhaus i e desc ip ion and explana ion
o all he compu a ional me hods de eloped and employed o pe o m image p ocessing and
egis a ion on he da ase , culmina ing wi h he inal amewo k implemen ed in eg a ing he
mos sui ed p e-p ocessing, segmen a ion, egis a ion and 3D econs uc ion o he cell
issues. The p esen ed compu a ional amewo k was implemen ed in Ma lab R2014a® (Inc.,
Na ick, Massachuse s, Uni ed S a es).
3.1. Da ase
The h ee image da ase s s udied in his p ojec we e acqui ed and p epa ed, using he
s anda d me hod de ailed in sec ion 2.1.2., in he Pa hology labo a o y in Ins i u e o
Biomedical Sciences Abel Salaza . In o de o ob ain he digi al images an Olympus scanne
p og am was used, c ea ing an image o each slice in he h ee s udied cases, desc ibed u he
in his sec ion. Some ea u es ound in images om he da ase s ha can unde mine he image
p ocessing and analysis p oposed by he algo i hm de elop in his wo k, a e also add essed in
he p esen sec ion.
Me hodology
44
image h eshold (sec ion 2.2.2.) o he Cb g ayscale image is de e mined using he O su
algo i hm (O su 1975), and hen applied o he image o ob ain a bina y mask o he ele an
s uc u e in he image ( he backg ound in ensi y becomes 0). The inal mask is ob ained a e
small adjus men s wi h mo phological ope a o s, namely image closing and opening (in his
o de ).
The e ec i eness o all he p esen ed image p e-p ocessing me hods was es ed wi h a
kmeans algo i hm (sec ion 2.2.2.), based on (Ma hwo ks), upg aded o pe o m colo ed image
segmen a ion. This clus e ing algo i hm was implemen ed in his s age due o i s simplici y and
low compu a ional cos , enabling he image segmen a ion in mul iple classes, o , in his case,
cell issues, o de e mine he op-pe o ming p e-p ocessing me hod in s ain colo
disc imina ion. The colo upg ade was achie ed by eshaping he incoming RGB image ma ix
o a ow × column × 3 ma ix, hus ga he ing on each column he pixel in ensi y in o ma ion o
each colo channel image. These pixel in ensi y iple s we e used in he segmen a ion
p ocedu e o disc imina e classes. The kmeans was applied o 3 and hen o 4 classes, in o de
o sepa a e he image backg ound om wo o h ee di e en cell issues.
The gene al scheme o he i s wo k low implemen ed is p esen ed in Figu e 22.
P e-P ocessing
Colo space
ans o ma ion
and manipula ion
(All he colo
space we e
es ed)
CLAHE Image
No maliza ion
+
Con as
enhancemen
+
Mask
Segmen a ion
Kmeans: o 3/4
classes
Figu e 22. Schema ic model ep esen ing he i s wo k low implemen ed in he s udy. Se e al con as
enhancemen p e-p ocessing echniques we e applied o he da ase , and hei e iciency was es ed
wi h he kmeans algo i hm, o 3 and 4 classes.

3.2. Wo k low implemen a ion
45
3.2.2. Second App oach – Based on Image Regis a ion
The knowledge acqui ed om he p e-p ocessing s udy, conce ning he bes me hods o
p o ide an accu a e segmen a ion, led o he egis a ion es , whe e se e al egis a ion
echniques we e app oached. To de e mine he mos app op ia e, wo c i e ia we e aken in o
accoun : he DICE sco e (Sø ensen 1948, Cheung 2012) and he compu a ional cos .
Fou egis a ion algo i hms we e es ed in wo di e en egis a ion models, pai wise and
e e ence slice (mo e de ails in sec ion 2.3.).
The i s algo i hm implemen ed is a ea u e-based egis a ion me hod and he second an
in ensi y-based algo i hm. The ea u e egis a ion me hod only enables o a ion and
ansla ion o he mo ing images, since he only ype o ans o ma ion implemen ed is
simila i y. On he o he hand, he in ensi y–based model allows o he wo ypes o
ans o ma ion, he igid and he a ine (sec ion 2.3.). The o he wo algo i hms es ed, he
Demon algo i hm and he B-spline algo i hm, which a e bo h capable o pe o ming non- igid
egis a ion. All hese me hods a e explained u he in his sec ion.
The au oma ic ea u e-based egis a ion s a s by de ec ing image ea u es in bo h
images, mo ing and ixed, mainly h ough he implemen a ion o he Speed-Up Robus Fea u es
(SURF) algo i hm (Bay, Ess e al. 2008), an algo i hm ha sea ches blob ea u es, egions o
he image whe e se e al p ope ies ( o example, b igh ness and in ensi y) emain cons an .
The de ec ed ea u es a e hen ex ac ed including hei loca ion in he image, h ough he
pixels su ounding he in e es poin o ea u e blob (Bay, Ess e al. 2008). The de ec ed and
ex ac ed ea u e egions in he mo ing and ixed images a e ma ched using pa allel
hie a chical clus e ing ees (Muja and Lowe 2012), esul ing in a pai o indexes om he
ma ched ea u es. The loca ions o hose ma ching poin s a e also e ie ed and he
ans o ma ion is pe o med based on he ma ched poin s o bo h ixed and mo ing image,
eso ing o M-es ima o Sample Consensus (MSAC) algo i hm (To and Zisse man 2000) o
exclude ou lie s. The inlie s o bo h se s o ma ching poin s a e mapped and o igina e a 2D
geome ic ans o m objec (Ha ley and Zisse man 2003). In he las s ep, he geome ic
ans o ma ion objec gene a ed is applied o he mo ing image, c ea ing his way he
egis e ed image. The scale and angle applied (conside ing ha i is a simila i y ans o m)
wi h he ans o m objec o c ea e he inal image a e eco e ed o se e as quali y con ol
o he whole p ocedu e, allowing his way he emo al o comple ely dis o ed images, possible
ou comes o he egis a ion p ocess. The s a ed amewo k was pe o med based on
(Ma hWo ks).
Conce ning he in ensi y-based egis a ion h ee key componen s ha e o be p e iously
de ined in o de o con igu e he whole egis a ion p ocess, he op imize , he me ic and he
Me hodology
46
ans o ma ion ype (sec ion 2.3.). The me ic is, in ac , he image simila i y me ic
esponsible o e alua ing he egis a ion’s accu acy and he op imize (Regula S ep G adien
Op imize , whose algo i hm is desc ibed in (Pennec, Cachie e al. 1999)) de ines he p ocedu e
o minimiza ion o maximiza ion o his simila i y me ic (Mean Squa es me ic
implemen a ion in (Ma hWo ks)). This egis a ion me hod is an i e a i e p ocess ha can be
pe o med in h ee di e en ans o ma ion ypes ( igid, simila i y and a ine) and always
equi es wo images, a ixed and a mo ing image. The whole egis a ion p ocess s a s wi h
an in e nally de e mined ans o ma ion ma ix combined wi h he ans o ma ion ype
speci ied, de e mining he image ans o ma ion ha is going o be applied o he mo ing
image wi h bilinea in e pola ion (Ma hWo ks). A e he in e pola ion he me ic compa es
bo h ans o med mo ing image and ixed image, compu ing he me ic alue and hen he
op imize checks o ending condi ions o s op he egis a ion p ocess. These condi ions can
be he maximum numbe o i e a ions (de ined by he op imize ) o a ce ain me ic alue
h eshold. I he s op condi ion does no e i y he op imize adjus s he ans o ma ion ma ix
o ini ia e a new egis a ion cycle. The maximum numbe o i e a ions and he op imize s ep
size he egis a ion can be al e ed o imp o e he egis a ion, bu always wi h a g ea e
compu a ional cos (Ma hWo ks). The explained wo k low is p esen ed in Figu e 23Figu e 23.
Figu e 23. Rep esen a ion o he in ensi y-based egis a ion amewo k implemen ed.
Adap ed om (Ma hWo ks).
The i s non- igid egis a ion me hodology es ed was he B-Spline G id, Image and Poin
Regis a ion de eloped by (K oon 2008), based on he algo i hm de eloped in (Ruecke ,
Sonoda e al. 1999). This is an in ensi y based egis a ion echnique whose algo i hm
implemen s a g id o B-spline con ol poin s (sec ion 2.3.) ha con ol he ans o ma ion o
3.2. Wo k low implemen a ion
47
he mo ing image o e he ixed one. I measu es he egis a ion e o , h ough squa ed pixel
dis ance (Ve cau e en, Pennec e al. 2009), a simila i y c i e ion based on he in o ma ion
heo y and calcula es he amoun o in o ma ion exis en in he egis e ed mo ing image abou
he ixed one (conside ing a egis a ion p ocedu e). The B-Spline me hod applies he Fas
Limi ed Memo y Op imize (K oon 2009), a Quasi-New on op imize , o mo e he con ol poin s,
in o de o achie e he op imal egis a ion be ween bo h images wi h minimal simila i y e o .
The implemen ed B-Spline me hod can also pe o med igid and a ine ans o ma ions (sec ion
2.3.).
The Demon algo i hm is a non- igid egis a ion echnique as e and a he simple han
he B-Spline. This algo i hm was i s desc ibed by (Thi ion 1998) and ollowed by (Wang, Dong
e al. 2005), and he me hodology ollowed in he p esen s udy was de eloped by (K oon 2008).
Fo each pixel a eloci y, o mo emen , is de ined by his me hod, using he in ensi y
di e ences and g adien in o ma ion. The eloci y ma ix is smoo hed by a Gaussian il e and
i e a i ely applied o ans o m he mo ing image and egis e i on o he ixed image. The
ans o ma ion is op imized by a limi memo y BFGS op imize (Liu and Nocedal 1989) in an
i e a i e and mul i- esolu ion way. The Demon algo i hm also pe o ms a ine egis a ion
(sec ion 2.3.).
In he second app oach we e also es ed combina ions o he p e iously desc ibed
algo i hms, s a ing wi h a igid egis a ion echnique ollowed by a mo e accu a e and
compu a ional demanding non- igid egis a ion algo i hm, as pe o med in p e ious s udies
(Robe s, Magee e al. 2012).
In o de o align all he images om he da ase s a ailable, wo di e en egis a ion
models, he e e ence slice model and he pai wise model, we e implemen ed o all he
abo emen ioned egis a ion me hods, s a ing om he middle slice (in he da ase s) since,
gene ally, i is he sec ion wi h mos issue (Robe s, Magee e al. 2012). Th ough he e e ence
model he egis a ion p ocedu e is pe o med conside ing only as ixed image he middle slice
om he image da ase , hus, being all he slices egis e ed o e e ence sec ion. The pai wise
model is pe o med in a cascade p ocess s a ing om he cen e slice and pe o ming
egis a ion in pai s o slices (mo ing image becomes ixed image in he nex alignmen ) in wo
di ec ions - un il he op slice in he i s un and he i s slice in he second un.
The wo k low o he egis a ion app oach combined wi h he bes sui ed p e-p ocessing
me hod is p esen ed in Figu e 24.
Me hodology
48
3.2.3. Final App oach – Based on he comple e wo k low wi h
he 3D econs uc ion
In he hi d and inal app oach, he mos success ul p e-p ocessing and egis a ion
me hod, o combined me hods, o he es ed image da ase s we e connec ed and he 3D
econs uc ion inal s ep was added o he wo k low.
In he p e-p ocessing s age he p e iously es ed me hods we e subjec ed o an e iciency
es , bu o a di e en segmen a ion echnique, he s ain decon olu ion me hod, explained in
mo e de ail u he in his sec ion. The me hod implemen ed o his i s s ep was he CLAHE
bu wi h a di e ence in he algo i hm p esen ed in he sec ion 3.2.1., since only he ed
channel was subjec ed o he his og am equaliza ion. This was pe o med due o he high
in luence o he ed channel in bo h Hema oxylin and Eosin s ain, p o ing, h ough he
segmen a ion, o be he bes s ain disc iminan me hod. The image was also subjec ed o a FIR
il e ( il e s wi h ini e impulse esponse) con olu ion o emo e line a i ac s o de o ma ions
on he analyzed slice (sec ion 3.1.).
Image Regis a ion
In ensi y-
based
egis a ion
Type: Rigid
and A ine
Fea u e-
based
egis a ion
Type:
Simila i y
Demon
algo i hm
Type: non-
igid
B-spline
egis a ion
Type: non-
igid
Combina ion
o wo
egis a ion
models
P e-P ocessing
1.
YCbC ans o ma ion
(wi h no maliza ion)
2. CLAHE- ed
Figu e 24. Schema ic model ep esen ing he second wo k low implemen ed in he s udy. In he i s s ep,
he mos success ul p e-p ocessing me hod was applied o he image da ase , ollowed by he image
egis a ion s ep, wi h se e al me hods being es ed.
3.2. Wo k low implemen a ion
49
The s ain decon olu ion echnique de eloped o his s udy was based on he wo k ca ied
ou by (Rui ok and Johns on 2001, Chan 2013) and also om (Unpublished 2015). The s ain
decon olu ion is a segmen a ion me hod ha is capable o decon ol e a s ained image ( o
example wi h H&E) and gene a e sepa a e images, each one wi h he dis ibu ion o a single
s ain (Hema oxylin, Eosin and Backg ound, in he case o he es ed da ase s). This me hod
assumes ha he chemicals employed o s ain he issues slides ollow he Bee -Lambe Law
o abso p ion (11):
𝐼=⁡𝐼𝑜𝑒−𝑆𝐷 (11)
whe e 𝐼 is he RGB image, 𝐼𝑜 is he in ensi y o he sou ce, 𝐷 he s ain densi y map and 𝑆 he
s ain abso p ion ma ix. This equa ion g an s a loga i hmic ela ion be ween he o iginal RGB
colo channels and he s ain in ensi ies. The implemen ed algo i hm s a s by con e ing he
RGB image 𝐼 in o he op ical densi y place (OD) – 𝐼𝑂𝐷 - h ough he p e ious equa ion in he
o m (12):
𝐼𝑂𝐷 =⁡−log⁡(𝐼
𝐼𝑜) (12)
A e he OD con e sion, he image pixels a e p ojec ed in o he Maxwellian ch oma ici y
plane. In he Maxwellian plane he dis ance be ween wo poin s is p opo ional o hei
ch oma ic dissemblance (Maxwell and Zaidi 1993), o wi hin his s udy, he pixels ha belong
o he same s ain a e p ojec ed close han he pixels belonging o di e en s ains. To calcula e
he pai o co esponding Maxwellian coo dina es (𝑚1,𝑚2) o each pixel 𝑝, he ollowing
equa ion (13) was implemen ed:
[𝑚1
𝑚2]=⁡[0.7071 −0.7071 0
−0.4082 −0.4082 0.8165]𝑝
‖𝑝‖1 (13)
The desc ibed Maxwellian plane p ope y and he pai o Maxwellian coo dina es allows
he implemen a ion o an unsupe ised classi ie , o example kmeans, o assign a label o
each pixel acco ding o i s s ain. Wi h he classi ica ion pe o med, he ollowing s ep is he
S ain Ma ix es ima ion. Fo his s udy, i is conside ed he exis ence o h ee s ains o classes,
he Hema oxylin (class 1), Eosin (class 2) and he Backg ound (class 3). The Maxwellian
coo dina es o he mean alue in each o hese classes - 𝜇𝑐=(𝜇𝑚1𝑐,𝜇𝑚2𝑐), ob ained h ough
he kmeans classi ica ion, is con e ed back o he OD space h ough he equa ions (14)
exposed below:
𝑆𝑐,𝑟 =0.7071𝜇𝑚1𝑐 −0.4082𝜇𝑚2𝑐 +0.33
𝑆𝑐,𝑔 =−0.7071𝜇𝑚1𝑐 −0.4082𝜇𝑚2𝑐 +0.33 (14)
𝑆𝑐,𝑏 =1−𝑆𝑐,𝑟 −𝑆𝑐,𝑔
whe e 𝑆𝑐,𝑟 is he ed channel alue o he s ain ec o o he class 𝑐, 𝑆𝑐,𝑔 he g een channel
alue and 𝑆𝑐,𝑏 he blue channel coun e pa . The s ain ec o is gene a ed o each class 𝑐 by
𝑆𝑐=[𝑆𝑐,𝑟,𝑆𝑐,𝑔,𝑆𝑐,𝑏]𝑇, and, in u n, he s ain ma ix by 𝑆=⁡[𝑆1,𝑆2,𝑆3], being 𝑐=1,2,3 he H, E

Me hodology
50
and B s ains, espec i ely. The s ain densi y map 𝐷 is ob ained h ough he in e se o he s ain
ma ix as demons a ed in equa ion (15):
𝐷=𝑆−1𝐼𝑂𝐷 (15)
The c ea ed s ain ma ix is hen applied o he OD image o each pixel o ob ain a inal
h ee channel image, composed by images om he h ee di e en s ains, Hema oxylin, Eosin
and Backg ound. The s ain images a e sepa a ed and he i s wo, he ele an s ains, su e
a con as enhancemen p ocess. The con as enhancemen me hod embedded in he s ain
decon olu ion amewo k pe o ms a mapping o in ensi y alues in g ayscale images, so ha
he ou pu da a (image) p esen s a sa u a ion o 1 % in high and low in ensi ies, hus an
inc eased con as in in ensi ies (Ma hWo ks).
The egis a ion me hod implemen ed was he au oma ic in ensi y-based egis a ion
algo i hm (sec ion 3.2.2.) wi h ans o m ype igid and pe o med aking as cons an ixed
image he e e ence slice o he image da ase ( he cen al slice), and as mo ing images all
he slices successi ely, om he i s o he las in he da ase , aligning hem o he e e ence
and s acking hem o o m olume ic da a.
To execu e he 3D econs uc ion and display we e implemen ed using wo di e en
amewo ks. The algo i hms es ed we e he Ma ching Cubes algo i hm (Lo ensen and Cline
1987) implemen ed by (Hamme 2011) and he Isosu ace amewo k om (Ma hWo ks),
adap ed o he image da ase s es ed in his s udy.
The Ma ching Cubes algo i hm c ea es polygonal su ace ep esen a ions o isosu aces
(Lo ensen and Cline 1987) o 3D scala ields, such as he slice s ack o med a e he
egis a ion comple e p ocess o e all he da ase images. The implemen ed me hod consis s
in a ec o ized e sion o he algo i hm, which compu es a iangula ed mesh o he isosu ace
wi hin a gi en 3D ma ix o scala alues, acco ding o he isosu ace alue ( he cons an alue
o le el se om he 3D image ep esen ed in he isosu ace), and speci ied in e ms o a ace
and e ex lis . The o ien a ion o he iangles gene a ed is selec ed acco ding o he no mal
poin om highe o lowe alues (Hamme 2011). This me hod enables he isualiza ion o he
gene a ed 3D su ace, wi h cus omizable colo s and iew.
The Isosu ace amewo k om (Ma hWo ks) also pe o ms he ex ac ion o he
isosu ace da a om olume ic acco ding o an isosu ace alue, al hough wi h a di e en
p ocess. This me hod pe o ms i s a olume ic in e pola ion using linea in e pola ion wi h
(Ma hWo ks), and hen connec s he poin s wi h equal isosu ace alue, o ming a h ee
dimensional su ace. This me hod is combined wi h a 3D da a smoo hing echnique (Ma hWo ks)
o smoo h he edges o he econs uc ed su ace and a 2D polygon illing me hod (Ma hWo ks)
o enable i s colo ing and isualiza ion.
The inal wo k low schema ic ep esen a ion is shown in Figu e 25.
3.2. Wo k low implemen a ion
51
3D Recons uc ion
Ma ching
Cubes
Isosu ace
P e-P ocessing
CLAHE in ed channel
Regis a ion
In ensi y-based igid
egis a ion wi h
e e ence slice
Segmen a ion
S ain decon olu ion
Re e ence Slice
New image
Regis e ed Image
is sa ed
Figu e 25. Schema ic model ep esen ing he hi d and inal wo k low implemen ed in he s udy. The
app oach s a s by compu ing a loop o each slice in he da ase (minus he e e ence middle slice)
in which he image su e s p e-p ocessing, segmen a ion h ough s ain decon olu ion and egis a ion
wi h he e e ence slice as ixed image. All he egis e ed images a e s o ed and s acked o o m he
3D da a da ase , which will, pos e io ly, se e as inpu o bo h 3D econs uc ion amewo ks.
52
53
Chap e 4
Resul s and Discussion
In his chap e , all he expe imen al esul s ob ained wi h he h ee app oaches desc ibed
in he me hodology (Chap e 3) and hei espec i e discussion a e p esen ed. The p e-
p ocessing esul s will be he i s o be analyzed, based on colo con as enhancemen and
segmen a ion esul s, ollowed by egis a ion amewo ks compa ison and e alua ion (second
app oach). Finally, he inal wo k low will be discussed, inishing wi h he 3D econs uc ion
esul s. All he selec ed pa ame e s and decisions pe o med a e p ope ly indica ed and
jus i ied, h oughou he en i e chap e . The wo k lows in his s udy we e es ed on a Windows
64bi In el Co e i7-4700HQ CPU a 2.40 GHz, wi h 8 GB o RAM compu e sys em.
4.1. P e-p ocessing s age
In his sec ion, esul s om he applica ion o p e-p ocessing me hods, add essed in he
i s app oach (sec ion 3.2.1.) will be p esen ed and discussed u he in his sec ion. In he
end, he bes me hods o wo k lows will be de e mined based on isual in e p e a ion and a
segmen a ion es , pe o med wi h kmeans algo i hm.
Obse ing all he lesions and hei desc ip ion, in sec ion 3.1.2., ound in images om he
h ee da ase s, i can be concluded ha hey p esen e y dis inc na u es and occu s ained
by bo h hema oxylin and eosin s ains, in a wide ange o colo in ensi ies. In ligh o he
p e ious, he p e-p ocessing echniques we e de eloped ocusing he colo con as
enhancemen and ul ima ely he dis inc ion be ween issues s ained wi h each one o hese
s ains.
The esul an images a e implemen a ion o he p e-p ocessing echniques, p esen ed
h oughou his sec ion, applied o a single slice om he second da ase , se ing as an
illus a i e example. The o iginal image om he selec ed slice (nº 35) is p esen ed in Figu e
26.
Resul s and Discussion
60
Figu e 34. Resul an images om YCbC colo space ans o ma ion applied o he no malized RGB
image (Figu e 30), wi h cons an Cb ac o (1/16) and dec easing luminance mul iplying ac o s -
1/20 (a), 1/40 (b) and 1/80 (c).
Analyzing he Figu e 34, showing he esul ob ained when he YCbC colo space
ans o ma ion is applied o he no malized image, ou come o he no maliza ion me hod
(Figu e 30), i is e iden he imp o emen in e ms o image quali y and sha pness. A gene al
compa ison be ween he h ee images shows ha hey a e simila in e ms o issue colo s
among ele an s uc u es, and he eby he diminu ion ac o s smalle han 1/20 do no
inc ease he colo con as . The only di e ence be ween hese images is he g een colo noise
no iced among pixels om he backg ound, which is highe o lowe alues. Conside ing his,
he bes sui ed diminu ion alue o hese images is 1/20x, since i p o ides a clea e
disc imina ion be ween di e en s ained issues. This alue, as well as he blue ch ominance
ac o (1/16) we e de e mined h ough expe imen a ion, i.e. colo con as enhancemen
op imiza ion. As no iceable in Figu e 34 he diminu ion o bo h ac o s esul s in an o e all
image da kening (luminance diminu ion), pa icula ly in issues s ained wi h hema oxylin, due
o highe blue colo in luence (blue ch ominance diminu ion). The YCbC colo space
ans o ma ion p eceded by an image no maliza ion was he mos accomplished me hod, o
he p esen ed in his sec ion, in inc easing he colo con as be ween Hema oxylin s ained, in
deep blue shades, and Eosin s ained issues, in magen a.
The masking me hod de eloped o his s udy (sec ion 3.2.1.) in ol ed a combina ion o
wo colo space ans o ma ion echniques, HSV and YCbC , wi h he p ede e mined
pa ame e s (enhancemen ac o s, 2x o HSV and 1/20x o YCbC ). An au oma ic h eshold
ope a ion is ollowed, in o de o c ea e a bina y image wi h he lymph node de ached om
he backg ound. The inal s eps o his masking me hod includes wo mo phological
ope a ions, s a ing wi h an image closing, wi h a 4 pixel adius disk shaped s uc u al elemen ,
a
b
c

4.1. P e-p ocessing s age
61
ollowed by an image opening pe o med wi h he same elemen , bu wi h 6 pixel adius. These
s eps we e conduc ed o achie e a unique inal mask s uc u e, as he one p esen ed in Figu e
35, o he o iginal image (Figu e 26). The masking p ocess is impo an o inc ease he
in luence o each di e en issue o be segmen ed in he segmen a ion p ocedu e.
Figu e 35. Mask s uc u e, c ea ed om he
o iginal image o emo e he backg ound (a).
To co obo a e he isual in e p e a ion and selec ion o he mos adequa e p e-p ocessing
me hod o enhance colo con as based on he esul an images, in he da ase s p epa ed o
his s udy, a simple segmen a ion p ocedu e was pe o med, using kmeans (sec ion 2.2.2.) o
each me hod and conside ing h ee di e en classes (hema oxylin s ained issues, eosin s ained
issues and backg ound), wi h 15 epe i ions and using squa e Euclidean dis ance o assign he
clus e s. The segmen a ion esul s a e shown in Figu e 36 as well as he p ocessed images used
o acqui e hem, espec i ely, masked wi h he s uc u e on Figu e 35. Compa ing he
masked images o igina ed om he di e en p e-p ocessing me hods and he espec i e h ee
class segmen a ion esul , ep esen ed in Figu e 36, i is clea ly no iceable ha CLAHE and
YCbC ans o ma ion we e he bes pe o ming me hods in dis inguishing he eosin and
hema oxylin s ained issues. Al hough CLAHE was he op me hod o dis inguish bo h s ains in
he segmen a ion (Figu e 36.b), he YCbC echnique p o ed o be he bes me hod o enhance
he con as be ween he lesion (neoplasic issue) and heal hy issues, isible in he espec i e
segmen a ion esul (Figu e 36.g). The h ee class segmen a ion esul co esponden o he
HSV enhancemen p esen s mos ly o e segmen ed issues, no being able o accu a ely ind
nei he eosin no hema oxylin s ained issues. On he con a y, he L*a*b ans o ma ion esul
p esen s unde segmen a ion o bo h s ains and an inconsis en pixel label h oughou he
en i e lymph node.
Facing he p e ious esul s, and in an a emp o imp o e he segmen a ion accu acy
ano he es was conduc ed bu conside ing ou classes, dealing his way wi h issues
p esen ing lowe concen a ion on eosin s ain colo , and possibly segmen addi ional issues,
such as he blood essels. The segmen a ion esul s a e p esen ed on Figu e 37.
Th ough compa ison be ween he esul s o he ou class segmen a ion and he 3 class
coun e pa in each image, i can be concluded ha was an o e all imp o emen in he issue
disc imina ion, mo e e iden in he wo me hods wi h he wo s ou come in he p e ious es
(HSV and L*a*b colo space ans o ma ion).
Resul s and Discussion
62
The CLAHE- ed segmen ed image p esen s a highe di e en ia ion in he issue
su ounding blood essels, main aining almos unal e ed he o he pixel’s assigned labels, in
ela ion o Figu e 36.b. A simila compa ison can be made o bo h YCbC segmen a ion esul s,
we e he ou h label was assigned o pixels su ounding he blood essels (Figu e 37.d),
al hough less e iden han he CLAHE pixel labelling. Despi e a conside able imp o emen in
HSV ans o med image segmen a ion, i does no p esen a sa is ac o y ou come, being a la ge
amoun o eosin s ained issue s ill wi h miss assigned labels. The CIE L*a*b esul s we e clea ly
Figu e 36. Masked images, esul an om he con as enhancemen echniques applied o he
o iginal image (le column) – CLAHE- ed (a), HSV ans o ma ion (c), L*a*b ans o ma ion (e) and
YCbC ans o ma ion (g). Kmeans esul s o he espec i e le image ( igh column) conside ing 3
classes – CLAHE- ed (b), HSV ans o ma ion (d), L*a*b ans o ma ion ( ) and YCbC ans o ma ion
(h).
a
b
c
d
e
g
h
4.1. P e-p ocessing s age
63
be e han he p e ious segmen a ion, bu also inconsis en , wi h a conside able amoun o
pixels belonging o hema oxylin and eosin s ains being assigned o a hi d label.
All hings conside ed, and o assess he e iciency o op-pe o ming p e-p ocessing wo k lows,
o s ained issues con as enhancemen , when applied o he o he wo da ase s, he
implemen a ion was made and he esul an images a e p esen ed in Figu e 40, ega ding case
1, and Figu e 39 o case 3. The selec ed slices, o se e as example (slice nº35), om case 1
and case 3 a e shown in Figu e 38.
Figu e 37. Segmen a ion esul s conside ing ou classes, o CLAHE image (a), HSV enhanced image
(b), L*a*b ans o med image (c) and YCbC image (d).
a
b
c
d
Figu e 38. O iginal image (slice nº 35) om case 1 (on he le ). O iginal image (slice nº35)
om he hi d da ase (on he igh ).
Resul s and Discussion
64
Analyzing he ou comes om he segmen a ion p ocess o bo h slices, om case 1 (Figu e
39Figu e 40.b and Figu e 39Figu e 40.d) and case 3 (Figu e 40.b and Figu e 40.d), i can be
concluded ha bo h p e-p ocessing wo k lows p oduced a sa is ac o y segmen a ion, hus
achie ing an accu a e disc imina ion o bo h di e en s ains, hema oxylin and eosin, in he
issues. The ac ha bo h au oma ic p e-p ocessing echniques demons a ed consis ency in
Figu e 39. Masked images, esul an om he con as enhancemen echniques applied o he o iginal
image om case 1 – CLAHE- ed (a), YCbC (c). Resul s om kmeans segmen a ion pe o med in he
espec i e le image (b, d).
Figu e 40. Masked images, esul an om he con as enhancemen echniques applied o he o iginal image
om case 3 – CLAHE- ed (a) and YCbC (c). Resul s om kmeans segmen a ion pe o med in he espec i e p e-
p ocessed image, o CLAHE- ed is image (b) and o YCbC is (d).
a
b
c
a
d
c
d
b
4.1. P e-p ocessing s age
65
da ase s wi h la ge mo phological and colo di e ences (case 1, case 2 and 3), ein o ces hei
quali y, and hus hei au oma ic colo con as enhancing capabili y. Compa ing bo h
me hods, i is no iceable a highe di e en ia ion be ween dis inc s ains (Hema oxylin and
Eosin) wi h he CLAHE- ed app oach, being he co esponden issues accu a ely sepa a ed,
e en in he collision umo whe e bo h cell lines we e clea ly iden i ied. On he o he hand,
he YCbC wo k low p oduced a issue di e en ia ion in he collision umo in case 1 and
di ided di e en issues nea he inne neoplasic issue in he case 3 image.
Du ing he p e-p ocessing echnique’s de elopmen and es ing, hey p esen ed se e al
in a an in e -case inconsis encies in e ms o con as enhancemen e iciency ( isible when
compa ing Figu e 36, Figu e 40 and Figu e 39) and, he e o e, in he inal segmen a ion accu acy.
These e o s a e mainly due o s aining incong ui ies in slices om di e en da ase s and e en
om slices o he same case s udy, a ac al eady men ioned in sec ion 3.1.2., p esen ing, in
he las scena io, iden ical s uc u es wi h sligh ly di e en colo s. Since hese p e-p ocessing
echniques ely on pixel in ensi y and colo p ope ies and ha e o be au oma ic, he
a o emen ioned ac is he mos p obable cause o some e o s in he segmen a ion.
Despi e some echniques, and espec i e esul s, being explo ed and desc ibed o mask
blood essels om he images, namely, CIE L*a*b and no maliza ion echniques, a e se e al
ials, in ense es ing and combina ion o echniques, he de elopmen o an au oma ic me hod
capable o pe o ming accu a e dis inc ion o blood essels, in all images con ained in he h ee
da ase s was p o en impossible. In some cases, only he ex e io blood essels we e emo ed,
in o he cases occu ed a pa ial emo al on he same. Facing his ac , he egions con aining
blood essels in he images we e no emo ed wi h he de eloped mask echnique,
implemen ed in he inal p e-p ocessing wo k low.
CLAHE- ed his og am equaliza ion wi h backg ound emo al (masking p ocess) and he
YCbC colo space, also wi h backg ound emo al, p eceded by a no maliza ion s ep, bo h op
pe o ming me hods in he segmen a ion p ocess, we e implemen ed o all slices om he
h ee image da ase s, in o de o imp o e image egis a ion, a p ocess exhaus i ely explo ed
in he nex sec ion.

Resul s and Discussion
66
4.2. Regis a ion s age
In he p esen sec ion, he egis a ion me hods, o combina ion o me hods, a e going o
be explo ed and hei e iciency in slice alignmen will be compa ed, based on he DICE sco e
and he compu a ional cos equi ed o compu e he en i e da ase image egis a ion. The
compu a ional cos was measu ed as he ime equi ed o comple e he egis a ion o all he
slices. The da ase selec ed o es he egis a ion me hods was case 3, con aining 100 slices,
since i is he comple e da ase wi h he lowes numbe o images. All he expe imen s, using
di e en egis a ion algo i hms we e conduc ed o e e ence slice and pai wise egis a ion
models (sec ion 3.2.2.). All he images submi ed o a egis a ion me hod su e ed a p e ious
g ayscale con e sion ollowed by a con as adjus men wi h (Ma hWo ks).
The i s me hod explo ed was he au oma ic - in ensi y based echnique, due o i s
simplici y and i s h ee ypes o ans o ma ion ( igid, simila i y and a ine). The bes sui ed
p e-p ocessed image o pe o m egis a ion, among he esul an om he p e-p ocessing
me hods, desc ibed in he p e ious sec ion, was de e mined o his i s es and used o he
o he in ensi y-based me hods (Demon and B-spline algo i hms). The mean DICE sco e, a e age
DICE sco e om he 100 egis e ed slices, measu ed o assess he egis a ion quali y, and he
ime elapsed du ing he egis a ion o he en i e da ase a e p esen ed in Table II, o all he
expe imen s conduc ed.
Table II. Table con aining he mean DICE sco es and he elapsed imes o he in ensi y-based egis a ion
implemen a ion, wi h di e en ypes o ans o ma ion ( igid, simila i y and a ine) and models
( e e ence slice, pai wise), on images om bo h op-pe o ming p e-p ocessing me hods (YCbC colo
ans o ma ion and CLAHE- ed).
Fi s ly, he images esul ing om he YCbC colo space ans o ma ion ( he me hod was
applied o all slices in he da ase ) masked and unmasked we e es ed wi h igid
P e-p ocessing
me hod
T ans o ma ion
In ensi y- based egis a ion
Re e ence Slice
Pai wise
YCbC colo space
ans o ma ion
Wi h Mask
Rigid
DICE: 0.6632±0.1739
Time: 54.808968 s
DICE: 0.6425±0.2612
Time: 53.710495 s
Wi hou Mask
Rigid
DICE: 0.9114±0.0288
Time: 66.556130 s
DICE: 0.8897±0.0411
Time: 66.166884 s
Simila i y
DICE: 0.9033±0.0642
Time: 81.728404 s
DICE: 0.8906±0.0422
Time: 72.233815 s
A ine
DICE: 0.9054±0.0626
Time: 74.174649 s
DICE: 0.8897±0.0432
Time: 67.782040 s
CLAHE- ed
Wi hou Mask
Rigid
DICE: 0.9683±0.0283
Time: 55.990489 s
DICE: 0.9419±0.0385
Time: 47.265646 s
Regis a ion s age
67
ans o ma ion, o assess i he backg ound emo al would imp o e he egis a ion p ocess, as
i did wi h he segmen a ion (sec ion 4.1.). Compa ing he DICE sco es i is clea ha wi hou
masking p ocess he egis a ion is conside ably mo e accu a e (Masked DICE = 0.6632±0.1739;
Wi hou mask DICE = 0.9114±0.0288; o e e ence slice model), al hough wi h a highe
compu a ional cos . These esul s p o e no only ha he masking me hod implemen ed is
i egula in e ms o emo ed s uc u es, bu also ha a masked image possesses a lesse
in o ma ion, o an in ensi y-based egis a ion me hod, han he same image wi h backg ound.
Conside ing he disc epancy be ween he ob ained DICE sco es, he ollowing expe imen s we e
all conduc ed wi hou masked images, o comp ehend which one o he egis a ion ypes
pe o ms he bes egis a ion conside ing he a ailable da ase s. Compa ing he highe igid
sco e wi h bo h simila i y and a ine DICE sco es i is possible o deduce ha when he
e e ence model is pe o med he igid ans o ma ion sligh ly ou pe o ms he o he wo ypes
when he model is he e e ence slice ( igid DICE = 0.9114±0.0288; simila i y DICE =
0.9033±0.0642; a ine DICE = 0.9054±0.0626), and, when implemen ing he pai wise amewo k
he h ee sco es a e simila ( igid DICE = 0.8897±0.0411; simila i y DICE = 0.8906±0.0422; a ine
DICE = 0.8897±0.0432), wi h sligh ly highe DICE esul o he simila i y ype. A e his es ,
he se o YCbC images was subs i u ed by he esul ing images om he CLAHE- ed me hod
o e i y which p e-p ocessing echnique p oduces he bes images o in ensi y-based
egis a ion. This es was conduc ed wi hou backg ound emo al and o he op sco ing
egis a ion ype – igid egis a ion. The compa ison be ween he DICE sco es indica es ha
he bes p e-p ocessing me hod, o he conside ed da ase in ensi y-based image egis a ion
is he CLAHE- ed ( o e e ence slice model CLAHE igid DICE = 0.9683±0.0283, YCbC igid DICE
= 0.9114±0.0288; o pai wise model CLAHE igid DICE = 0.9419±0.0385, YCbC igid DICE =
0.8897±0.0411). All he es s pe o med wi h e e ence slice model consis en ly p esen a
highe DICE sco e when con on ed wi h hei pai wise model coun e pa s, sugges ing ha ,
o his case 3 he bes model o pe o m in ensi y-based egis a ion, and he e o e, he en i e
image da ase alignmen is he egis a ion o each image o he e e ence slice. In he p esen
wo k he de ined e e ence slice is he middle one (slice nº50 in he conside ed da ase – case
3), because i is he mos ep esen a i e in e ms o issues, acco ding o (Chak a a y, Bedell
e al. 2008)).
Analyzing he compu ing ime o all he di e en egis a ion ypes he esul s (Table II),
i can be obse ed ha he igid ans o ma ion is he less compu a ionally expensi e, ollowed
by he a ine and hen he simila i y ans o m, wi h he highes ime. These esul s go
acco ding o expec ed, because he igid ans o m consis s only in ansla ion and o a ion,
being simple han he o he wo (simila i y – ansla ion, o a ion and scale; a ine –
ansla ion, o a ion, scale and shee ). Compa ing, on he o he hand, he imes ob ained by
bo h egis a ion models, i is isible a consis en inc eased compu a ional cos in he
egis a ion me hods pe o med using e e ence slice.
Resul s and Discussion
68
Alignmen images and pai o ixed and mo ing image o he mos ele an egis a ion
expe imen s a e shown in Figu e 42, Figu e 42 and Figu e 43. The chosen mo ing image is slice
nº35, he same as in he p e-p ocessing sec ion (4.1.).
Figu e 42. Images ep esen ing in ensi y-based image egis a ion
pai wise model and igid ans o ma ion ype, om YCbC p e-
p ocessing. Image (a) ep esen s he ixed image (slice nº36), (c) he
mo ing image (slice nº35) and (b) he igid egis a ion o e lapping
image (Fixed and egis e ed images).
b
a
d
c
a
b
c
Figu e 41. Images ep esen ing in ensi y-based image egis a ion
pe o med wi h e e ence slice model and wo ypes o ans o ma ion,
om YCbC p e-p ocessing. Image (a) ep esen s he ixed image (slice
nº50) and (c) he mo ing image (slice nº35) o bo h igid (b) and a ine
(d) egis a ion ype o e lapping image (Fixed and egis e ed images).
Regis a ion s age
69
A e expe imen ing he au oma ic in ensi y-based egis a ion algo i hm, explained in
sec ion 3.2.2., an au oma ic ea u e-based simila i y egis a ion app oach was implemen ed
h ough an algo i hm also e e ed in his sec ion. Based on he op-sco ing esul s ob ained
wi h he i s me hod, he ea u e-based echnique was applied o he CLAHE- ed p ocessed
images, bu wi hou any iable esul s. This was due o he lack o ma ching poin s e i ied in
he egis a ion p ocess o mos o he slices con ained in he da ase , ei he ecu ing o
e e ence slice o pai wise model. Taking his in o accoun and ha ea u e-based me hods
pe o m image alignmen based on ea u es co espondence (be ween mo ing and ixed
images), such as poin s, lines and con ou s, ano he p e-p ocessing echnique was applied o
he da ase images, he image no maliza ion (sec ion3.2.1.). This me hod was implemen ed
due o i s capabili y o de ach he lymph node (in his case) om he backg ound, p omo ing
i s con ou ex ac ion. Aiming also o imp o e he con ou ex ac ion o he ea u e
egis a ion me hod a gaussian smoo hing il e , wi h ke nel dimensions 4x4, was applied o
he no malized image. The egis a ion esul s, DICE mean sco e and compu a ional ime, o
smoo hed and non-smoo hed images a e p esen ed in Table III. The e e ence slice egis a ion
model was es ed o his se o images, bu i was incapable o p o iding a ull da ase
egis a ion, and hus, he DICE sco e was no measu ed.
Table III. Table con aining he mean DICE sco es and he elapsed imes o he ea u e-
based egis a ion implemen a ion wi h pai wise model on images wi h bo h smoo hed
and non-smoo hed image no maliza ion p e-p ocessing.
P e-p ocessing me hod
Fea u e-based simila i y egis a ion
(Pai wise model)
Image
No maliza ion
Wi h smoo hing
DICE sco e: 0.9218±0.0481
Time: 30.537524 s
Wi hou
smoo hing
DICE sco e: 0.8412± 0.0999
Time: 27.915660 s
a
b
c
Figu e 43. Images ep esen ing in ensi y-based image egis a ion
pe o med wi h e e ence slice model and igid ans o ma ion ype,
om CLAHE- ed p e-p ocessing. Image (a) ep esen s he ixed image
(slice nº50), (c) he mo ing image (slice nº35) and (b) he igid
egis a ion o e lapping image (Fixed and egis e ed images).
Resul s and Discussion
76
Analyzing Table VI i was possible o conclude which o he s ain decon olu ion ou come
images is he mos sui ed o pe o m image egis a ion. The highes mean DICE sco es e i ied
belong o egis a ions pe o med wi h hema oxylin images, in all h ee da ase s, jus i ied by
i s g ea e ep esen a i eness in e ms o high in ensi y s uc u es, as i can be isualized in
Figu e 49. In case 2 bo h images, hema oxylin and eosin, sha e simila s uc u es, wi h high
pixel in ensi y, a ac , co obo a ed by he closes p oximi y in he DICE sco es measu ed. In
e ms o compu a ional ime, he less demanding a e he da ase egis a ions pe o med wi h
he eosin images, because o he same eason explained o he DICE sco e, less amoun o
pixels o be co ela ed in he p ocedu e. I is also no iceable ha , o case 1 he egis a ion
was no only compu a ionally expensi e bu also inaccu a e and inconsis en (high s anda d
de ia ion alues), wi h mean DICE sco es below hose e i ied in o CLAHE- ed (Table V),
sugges ing ha he s ain decon olu ion p o ed i sel incapable o dis inguish bo h issue s ains
in a subs an ial po ion o slices, om his da ase .
Figu e 49. Segmen a ion esul s ob ained wi h he s ain decon olu ion algo i hm ( i s column): case 1-(a), 2-(d)
and 3 (g). Hema oxylin image ex ac ed wi h his algo i hm (second column): case 1-(b), 2-(e) and 3-(h). Eosin
image ex ac ed ( hi d column): case 1-(c), 2-( ) and 3-(i).
h
g
e
d
c
b
a
i

Final Resul s
77
Table VI. Table con aining he mean DICE sco es and he elapsed imes
o he in ensi y-based igid egis a ion me hod implemen a ion, on all
he image da ase s (Case 1, 2 and 3), conside ing wo di e en g ayscale
images (hema oxylin and eosin images), ob ained h ough p e-p ocessing
(CLAHE- ed and YCbC ans o ma ion) and segmen a ion (s ain
decon olu ion algo i hm).
Case s udy
Regis a ion
Hema oxylin image
Eosin image
1
DICE: 0.8387±0.2785
Time: 71 min (app ox.)
DICE: 9.4781e-04±0.0102
Time: 70 min (app ox.)
2
DICE: 0.9345±0.0368
Time: 204.695529 s
DICE: 0.9179±0.0610
Time: 155.079913 s
3
DICE: 0.9676±0.0296
Time: 183.110440 s
DICE: 0.6152±0.0894
Time: 158.491751 s
Conside ing he p e ious esul s he selec ed se o g ayscale images chosen o ind he
egis a ion ans o ma ion we e he hema oxylin s ain images. The ans o ma ion ound on
each slice egis a ion wi h he e e ence was applied o he co esponden eosin image, being
aligned wi h he same ansla ion and o a ion han he hema oxylin image. He eupon, bo h
g ayscale image da ase s (hema oxylin and eosin) we e equally aligned, and his amewo k
was pe o med o each case s udy.
A e egis e ing he comple e image da ase o he e e ence slice, all images we e
s acked c ea ing olume da a, in which he z coo dina e ep esen s he slice numbe . All he
image da ase s (case 1, 2 and 3) we e s acked in wo ec angula p isms each, one o he
hema oxylin and he o he o eosin egis e ed images, wi h leng h and wid h de ined by he
leng h and wid h o he e e ence slice (because he egis e ed image acqui es he size o he
ixed image), and wi h heigh equal o he numbe o slices con ained in he da ase . These
inal olumes we e smoo hed wi h a box il e (a spa ial domain low-pass il e in which each
poin in he ou pu is he a e age o he su ounding poin s in he inpu s uc u e), wi h 5x5
Figu e 50. Images ep esen ing in ensi y-based image egis a ion pe o med wi h e e ence
slice model and igid ans o ma ion ype, om hema oxylin images ob ained h ough s ain
decon olu ion (Case 3). Image (a) ep esen s he ixed image (slice nº50), (c) he mo ing image
(slice nº35), (b) he egis a ion o e lapping image and (d) he ans o med eosin s ained slice
(nº35), wi h he egis a ion ans o ma ion de e mined o he hema oxylin images.
d
c
b
a
Resul s and Discussion
78
ke nel, and se ed as inpu o bo h 3D su ace econs uc ion me hods, he Ma ching Cubes
algo i hm and he isosu ace amewo k om (Ma hWo ks) desc ibed in sec ion 3.2.3. Using he
ma ching cubes algo i hm i was only possible he econs uc ion o one s ained su ace a he
ime, being ep esen ed in Figu e 51 he hema oxylin and eosin econs uc ions om case 3
image da ase , wi h iso alue equal o 0.5 o hema oxylin s ack and 0.65 o eosin. Bo h
iso alues we e de e mined h ough expe imen a ion, aiming o econs uc a comple e 3D
su ace o he conside ed issue (s ained wi h hema oxylin o eosin) and a he same ime
educe he noise p oduced by backg ound a i ac s wi h in ensi ies simila o he econs uc ed
s uc u e, hus, educing i s in elligibili y o he obse e .
Figu e 51. 3D su ace econs uc ion om case 3 egis e ed image da ase , using Ma ching cubes
algo i hm. Images (a) and (b) a e om hema oxylin image da ase su ace econs uc ion wi h di e en
iews. Images (c) and (d) a e eosin s acked images’ su ace econs uc ion. S uc u e (1) ep esen s he
neoplasic issue ound in Figu e 20 – A; s uc u e (2) ep esen s he neoplasic issue ma ked in Figu e 20
– B and s uc u e (3) is he blood essel in Figu e 20 – C.
a
b
c
d
1
1
2
2
3
3
Final Resul s
79
Implemen ing he isosu ace amewo k i was possible o combine bo h issue s ains
s acks’ su ace econs uc ion in one image, using he pa ch unc ion (Ma hWo ks), and he
esul ob ained o case 2 and 3 can be consul ed in Figu e 52. To each issue s ain
econs uc ion was assigned a colo , based on he eal colo p esen ed in he o iginal image
da ase . This was accomplished using an auxilia y algo i hm, which, i s , sepa a es bo h s ains
om he CLAHE- ed image based on s ain decon olu ion segmen a ion, pe o ms an a e age
pixel in ensi y measu emen in he h ee colo channels o bo h s ain images om he slice and
hen, a e he da ase comple e analysis, pe o ms ano he a e age calcula ion o all he
slices, ob aining wo inal RGB iple s o be applied o he espec i e s ained issue 3D
econs uc ion. Fo his me hod we e applied wo di e en iso alues, 0.5 and 0.65 (same as in
he p e ious me hod) o hema oxylin and eosin s acks, espec i ely.
Obse ing Figu e 51 and Figu e 52, i is possible o conclude ha bo h cases 2 and 3 p esen
a consis en slice alignmen , isible in he su ace uni o mi y along he Z-axis. This is mo e
e iden in he ma ching cubes algo i hm esul s, sugges ing ha he 3D in e pola ion pe o med
by his me hod p o ides mo e accu a e esul s. The success o he en i e amewo k is p o en
no only by his ea u e, bu also by he simila i y, in e ms o mo phology be ween wha is
seen in he o iginal 2D image slices, om bo h da ase s, and he esul o he 3D su ace
econs uc ion. Focusing on Figu e 51 and in case 3 issue analysis (sec ion 3.1.2.) he
Figu e 52. 3D su ace econs uc ion om case 2 (second line) and 3 ( i s line) egis e ed image da ase ,
using isosu aces amewo k. Image (a)- op iew, (b)-bo om iew and (c)-side iew, ep esen he case 3
olume ic da a combining bo h hema oxylin and eosin isosu aces in h ee di e en iews. Image (d)- op iew,
(e)-bo om iew and ( )-side iew, ep esen he case 2 olume ic da a combining bo h hema oxylin and eosin
isosu aces in h ee di e en iews.
a
e
d
c
b
Resul s and Discussion
80
esemblances a e ob ious, he e o e, enabling he co ela ion be ween some ele an
s uc u es iden i ied in Figu e 20. The mos e iden s uc u es econs uc ed a e he neoplasic
issue s ained wi h hema oxylin (Figu e 51.a – 1) and he o he wo s ained wi h eosin (Figu e
51.d – 2), and also he blood essel (Figu e 51.c – 3 and Figu e 20 – c). These 3D su ace
econs uc ions a e consis en and a e able o p o ide impo an olume ic in o ma ion abou
his ological slices con ained in he da ase s, con e ing a plana obse a ion in a 3D
isualiza ion o he issues s ained wi h hema oxylin and eosin in an ei he sepa a e o me ged
e sion. Analyzing he su ace econs uc ion pe o med o case 2 (Figu e 52– d,e, ), i is
e i ied a educed eosin issue de ini ion, maybe due o he spa se na u e o he issue on he
o iginal da ase ha dec eases he in elligibili y o he olume ic da a. The colo s exp essed
in he inal su ace econs uc ions p esen high a ini y wi h he s ain colo s e i ied in he
CLAHE- ed images in Figu e 29, poin ing he e iciency o he p e iously desc ibed colo
algo i hm in combina ion wi h he isosu ace amewo k.
The 3D su ace econs uc ion amewo k applied o case 1 p esen ed some compu a ional
issues, which comp omised he esul demons a ion. The p ocessing powe used o es all he
amewo ks in his s udy (In el Co e i7-4700HQ CPU a 2.40 GHz, wi h 8 GB o RAM on a Windows
64bi sys em) was no su icien o load and isualize he olume ic da a, mainly due o he
bad pe o mance o he s ain decon olu ion me hod in disc imina ing issue s ains in mos slices
con ained in he da ase and he la ge image dimensions.
Taking in o accoun all he p e ious 3D su ace econs uc ions, esul ing om he inal
amewo k de eloped o his s udy, i can be concluded ha he h ee-dimensional
ep esen a ion o he da ase s was accomplished o he majo i y o he a ailable da ase s,
and wi h in e es ing and eliable esul s. Especially o case 3, he econs uc ion was able o
p o ide accu a e 3D olume ic ea u es om neoplasic issues in addi ion o o e all
mo phological cha ac e is ics o he lymph node. The mos ele an a e he h ee dimensional
p og ession on he Z-axis o he lesion o e o he issues (Figu e 51), di icul o isually
es ima e om a se o 2D slices.
4.4. Summa y
In summa y, he op pe o ming p e-p ocessing wo k lows o hema oxylin and eosin s ain
con as enhancemen we e he CLAHE- ed his og am equaliza ion ollowed by a backg ound
emo al (masking p ocess) and he YCbC colo space, also wi h backg ound emo al, p eceded
by a no maliza ion s ep.
A e es ing se e al me hods, i was concluded ha he mos e icien amewo k o
pe o m egis a ion in he conside ed his ological image da ase s is he au oma ic in ensi y-
based egis a ion wi h igid ans o ma ion ype and e e ence slice model on CLAHE- ed p e-
p ocessed slices. This was deduced based on bo h DICE sco e esul s and compu a ional cos ,
being he me hod wi h lowe imes o execu ion and he second highes DICE sco es. The
Summa y
81
highes sco e was ob ained wi h B-spline non- igid algo i hm, bu , since his me hod o igina ed
dis o ion a i ac s o p o ide accu a e egis a ions, co up ing aluable issue in o ma ion,
and he imes equi ed o compu e en i e da ase s could each app oxima ely one hou , he
igid me hod was conside ed he mos sui ed o he ask.
Combining he bes me hods om bo h p e-p ocessing and egis a ion sec ions, he inal
wo k low was comple ed wi h he 3D su ace econs uc ion o bo h hema oxylin and eosin
s ained issues egis e ed s ack. The eosin and hema oxylin sepa a e images we e acqui ed
om an in e media e segmen a ion s ep, he s ain decon olu ion algo i hm, a specially
de eloped me hod o s ained issues di e en ia ion om his ological images. Th ough he
combined implemen a ion o hese ou main s eps an accu a e and in e es ing olume ic
ep esen a ion o bo h hema oxylin and eosin s ained issues was de e mined. Bo h 3D su aces
can be shown oge he o in sepa a e images.

82
83
Chap e 5
Conclusion
Au oma ed echniques o image p ocessing and analysis applied o his ological s udies
cons i u e an impo an asse in he comp ehension and assessmen o ele an aspec s,
olume ic da a and models om nume ous diseases, unob ainable h ough 2D issue slice
analysis. Impelled by he ele ance o his s udies and he u ge o assis he his opa hologis s
in pe o ming medical diagnos ic, mo e accu a e and eliable algo i hms ha e been de eloped,
p o iding a as and us wo hy second opinion.
The p esen s udy explo es his ological concep s, including issue ypes and sample
p epa a ion as well as success ul echniques o image p ocessing and analysis, highligh ing
cu en eliable app oaches in he ield o 3D his ological issue econs uc ion.
Wi h he p e ious esea ch in mind and conside ing he need o p oduce a 3D
econs uc ion app oach o cellula issue analysis, in his wo k i was de eloped an au oma ed
compu a ional amewo k combining he bes algo i hms and echniques implemen ed by he
e iewed app oaches, ocused no only on econs uc ing he ele an issues, bu also on hei
spa ial ela ion wi h he su ounding s uc u es.
A e conduc ing he p esen s udy, i is possible o conclude ha se e al a i ac s p esen
in his ological issue sec ions, such as di e ences in s ain colo s be ween slices and/o image
da ase s, can educe he consis ency o bo h image p ocessing and egis a ion me hods when
applied o hem. On he o he hand, h ough he implemen a ion o accu a e egis a ion
echniques on he image da ase s, he p esence o issue o ien a ion dissimila i ies, due o he
manual na u e inhe en o he sample p epa a ion p ocess, was almos mi iga ed.
The colo con as enhancing p e-p ocessing me hods de eloped and applied o he
o iginal digi al images e ec i ely p o ided an imp o emen in disc imina ing di e en s ained
issues. This can be in e ed ei he om isual analysis o esul ing images o om he s ain
decon olu ion segmen a ion esul ’s accu acy, and, consequen ly high pe o mance in he
egis a ion p ocess, wi h DICE sco es o 0.9345±0.0368 o case 2 and 0.9676±0.0296 o case
Conclusion
84
3. Case 1 achie ed poo e esul s in gene al, wi h a inal DICE sco e o 0.8387±0.2785 in he
egis a ion p ocess, jus i ied by an inconsis en s ain decon olu ion p ocedu e, e ealing i sel
insu icien o accu a ely sepa a e bo h s ains, in a g ea po ion o images om his da ase .
A consis en neoplasic issue disc imina ion was no ob ained h ough he implemen ed
algo i hm, bu , by obse ing he 3D su ace econs uc ion o bo h eosin and hema oxylin
(sec ion 4.3.) he p esence o his issue is e iden . Despi e being speci ied he lesions in
sec ion 3.1.2. as issues o be a ge ed by he algo i hm, aiming o p omo e an isola ed 3D
econs uc ion o hese neoplasic issues, he s ained issue sepa a ion and pos e io 3D
econs uc ion e eals he in e ac ion be ween he neoplasia and he su ounding heal hy
issues. The e o e, he consis en econs uc ion o bo h s ained issues including he lesion,
wi h accu a e olume ic shape, cons i u es ele an in o ma ion o diagnos ic pu pose.
Conside ing all his, i is concluded ha he algo i hm de eloped in his s udy p o ides a
as and accu a e 3D su ace olume, comp ising bo h dis inc s ains ep esen ed independen ly
and showing he h ee dimensional ela ion be ween lesions and he su ounding issue. The
p oposed amewo k ob ained a sa is ac o y pe o mance o wo o he h ee a ailable
da ase s.
5.1. Fu u e Wo k Pe spec i es
One u u e wo k imp o emen is he p epa a ion and es ing o mo e his ological image
da ase s o u he alida e he de eloped algo i hm, and also o assess he algo i hm’s
accu acy when analyzing issues wi hou H & E s ain. This would also ins iga e a e inemen in
he compu a ional echniques, adap ing he au oma ic amewo k o mo e se s o images and
de elop a mo e uni e sal algo i hm, capable o pe o ming de ailed issue 3D econs uc ion
o any his ological sec ion.
Ano he u u e imp o emen would be he inco po a ion o a unc ional and gene al lesion
de ec o , able o dis inguish special issues om he o he s ained heal hy ones. The
imp o emen o he s ain decon olu ion me hod, implemen ing supe ised classi ica ion o he
s ained issues, using as aining se a po ion o he en i e image da ase , could lead o mo e
accu a e s aining disc imina ion and possibly o de ine o he in e es ing issues, such as
neoplasic issues.
An in e es ing u u e de elopmen would be he implemen a ion o he p oduced
compu a ional amewo k in openc (Open Sou ce Compu e Vision), enabling a as e image
p ocessing, o la ge image da ase s, such as case 1, and he c ea ion o an in ui i e use
in e ace o inpu image da ase manipula ion and 3D econs uc ed olume obse a ion.
85
Re e ences
A ganda-Ca e as, I., R. Fe nandez-Gonzalez and C. O iz-de-Solo zano (2004). Au oma ic
egis a ion o se ial mamma y gland sec ions. Enginee ing in Medicine and Biology Socie y,
2004. IEMBS'04. 26 h Annual In e na ional Con e ence o he IEEE, IEEE.
Aube , G. and P. Ko np obs (2006). Ma hema ical p oblems in image p ocessing: pa ial
di e en ial equa ions and he calculus o a ia ions, Sp inge Science & Business Media.
Bau is a, P. A. and Y. Yagi (2010). "Imp o ing he isualiza ion and de ec ion o issue olds in
whole slide images h ough colo enhancemen ." Jou nal o pa hology in o ma ics 1: 1-25.
Bay, H., A. Ess, T. Tuy elaa s and L. Van Gool (2008). "Speeded-up obus ea u es (SURF)."
Compu e ision and image unde s anding 110(3): 346-359.
Bea e, R. and G. Lehmann (2006). "The wa e shed ans o m in ITK-discussion and new
de elopmen s." Insigh J: 1-24.
Beng sson, E. (2003). Compu e ized cell image analysis: Pas , p esen , and u u e. Image
Analysis, Sp inge : 395-407.
Blake, C. A., H. A. La oie and C. F. Mille e (2003). "Teaching medical his ology a he
Uni e si y o Sou h Ca olina School o Medicine: ansi ion o i ual slides and i ual
mic oscopes." The Ana omical Reco d Pa B: The New Ana omis 275(1): 196-206.
Bonne , N. (2004). "Some ends in mic oscope image p ocessing." Mic on 35(8): 635-653.
Books ein, F. L. (1980). "The Measu emen o Biological Shape and Shape Change." Sys . Zool
29(1): 102-104.
B aumann, U.-D., J.-P. Kuska, J. Einenkel, L.-C. Ho n, M. Lo le and M. Hockel (2005). "Th ee-
dimensional econs uc ion and quan i ica ion o ce ical ca cinoma in asion on s om
his ological se ial sec ions." Medical Imaging, IEEE T ansac ions on 24(10): 1286-1307.
Caicedo, J. (2009, 06 No embe ). "Fea u es o His ology Images." Re ie ed 03 Feb ua y,
2015, om h p://www.in o med.unal.edu.co/jccaicedo/docs / e iew.pd .
Chak a a y, M. M., B. J. Bedell, S. P. Zehn ne , A. C. E ans and D. L. Collins (2008). Th ee-
dimensional econs uc ion o se ial his ological mouse b ain sec ions. Biomedical Imaging:
F om Nano o Mac o, 2008. ISBI 2008. 5 h IEEE In e na ional Symposium on, IEEE.
Chak a a y, M. M., G. Be and, C. P. Hodge, A. F. Sadiko and D. L. Collins (2006). "The
c ea ion o a b ain a las o image guided neu osu ge y using se ial his ological da a."
Neu oimage 30(2): 359-376.
Chan, A. C. S. (2013, 21 July 2013). "Tumo cell ex ac ion by colo decon olu ion." Re ie ed
15 June, 2015, om h p://web.hku.hk/~ccsigma/colo -decon /colo -decon .h ml.
Re e ences
92
Teo , L. A., R. Spos o, A. Khaya , S. Qualman, G. Reaman and D. Pa ham (2007). "The p oblems
and p omise o cen al pa hology e iew: de elopmen o a s anda dized p ocedu e o he
child en's oncology g oup." Pedia ic and De elopmen al Pa hology 10(3): 199-207.
Thi ion, J.-P. (1998). "Image ma ching as a di usion p ocess: an analogy wi h Maxwell's
demons." Medical image analysis 2(3): 243-260.
To , P. H. and A. Zisse man (2000). "MLESAC: A new obus es ima o wi h applica ion o
es ima ing image geome y." Compu e Vision and Image Unde s anding 78(1): 138-156.
T eano , D., N. Jo dan‐Owe s, J. Hod ien, J. Wood, P. Qui ke and R. A. Ruddle (2009). "Vi ual
eali y Powe wall e sus con en ional mic oscope o iewing pa hology slides: an expe imen al
compa ison." His opa hology 55(3): 294-300.
Unpublished, W. (2015). A Disc imina i e F amewo k o S ain Decon olu ion o His opa hology
Images in he Maxwellian Space.
Ve cau e en, T., X. Pennec, A. Pe chan and N. Ayache (2009). "Di eomo phic demons:
E icien non-pa ame ic image egis a ion." Neu oImage 45(1): S61-S72.
Ve non, D. (1991). Machine ision-Au oma ed isual inspec ion and obo ision, P en ice Hall.
Wang, H., L. Dong, J. O'Daniel, R. Mohan, A. S. Ga den, K. K. Ang, D. A. Kuban, M. Bonnen, J.
Y. Chang and R. Cheung (2005). "Valida ion o an accele a ed'demons' algo i hm o de o mable
image egis a ion in adia ion he apy." Physics in medicine and biology 50(12): 2887.
Wa son, M. L. (1958). "S aining o issue sec ions o elec on mic oscopy wi h hea y me als."
The Jou nal o Biophysical and Biochemical Cy ology 4(4): 475-478.
Weszka, J. S. (1978). "A su ey o h eshold selec ion echniques." Compu e G aphics and
Image P ocessing 7(2): 259-265.
Xu, R. and D. Wunsch (2005). "Su ey o clus e ing algo i hms." Neu al Ne wo ks, IEEE
T ansac ions on 16(3): 645-678.
Zi o a, B. and J. Flusse (2003). "Image egis a ion me hods: a su ey." Image and ision
compu ing 21(11): 977-1000.