A Low Cos 3D Vision Sys em o Posi ioning Welding Mobile Robo s
using a FPGA p o oyping sys em
I. Ibáñez, M.A. Agui e, A. To alba and L.G. F anquelo
Escuela Supe io de Ingenie os. Uni e sidad de Se illa
A da. Camino de los Descub imien os s/n,
41092 Se illa (SPAIN)
[email p o ec ed], o[email p o ec ed], leopold[email p o ec ed]
Abs ac – The aim o his wo k is o explo e some solu ions o
a i icial ision sys ems applied o welding au onomous obo s.
We ake ad an age om he UNSHADES-1 sys em, de eloped
in Filia ion (blind o e alua ion). This sys em can be used o
building a powe ul 3D ision sys em. The sys em concen a es
and p ocess he h ee images in pa allel, p oducing an accu a e
alue o he posi ion, ha can be imp o ed i he ela i e
posi ion o he came as a e well de ined.
I. INTRODUCTION
Au onomous Robo s a e comple e sys ems ha ope a e
inside complex en i onmen s wi hou being guided and
con olled by human ope a o s. They can es ablish
ela ionships, ake decisions in un- ime and adap hei
ac ion plans o di e en ex e nal ci cums ances. The ecei ed
in o ma ion make hem au o-p og ammables [1-2], al e ing
hei p og am in o de o sol e he ac ual si ua ion. Those
sys em ha e some AI in hei beha iou . Mos o cases, he
equi ed in o ma ions a e ela ed o posi ion, speed,
accele a ion, s eng h, o que, sizes and shapes om objec s
and empe a u e. Quan i ica ion o hose measu es a e made
by means o mechanical, op ical, e mical, elec ical,
ul asounds, ... senso s. Howe e , ad anced esea ch
specially akes in o accoun a i icial ision ansduce s. This
esea ch can ake ad an age om he newes mic oelec onic
sys ems, whe e design schedule ha e been sho ened using
he ad anced FPGA’s. These de ices can p ocess images in
“ eal ime”, p ocessing ame by ame on he ly.
Be ween he a i icial ision based solu ions, he mos
a ac i e ones o Robo ics applica ions a e objec
ecogni ion and classi ica ion, ensambling, welding, s ee ing
and synch onisa ion wi h o he obo s.
One o he mos complex applica ion is welding sys ems
based on au onomous obo s. An eme ging echnology ha
concen a es s a egies based on me alu gy, obo ics,
elec ical and elec onic enginee ing and mic oelec onics.
Welding obo s a e widesp ead demanded sys ems, specially
whe e he indus y has a high le el o au oma ion.
This pape ela es a new cus om sys em o de e mining
he exac posi ion o he welding poin . The me hod is based
on a sys em wi h h ee low cos came as ha a e placed and
ixed in he h ee space axes, and compu es he posi ion
h ough he simul aneous p ocessing o he images aken by
each came a. Each image is bina ised and he welding poin
is ob ained using a p og ammable bina y h eshold. So, o
each came a, he whole image is educed o a single poin . As
he h ee images a e p ocessed simul aneously, e e y
coo dina e is compu ed as pa o wo o hem and hen he
inal coo dina e is gi en as an imp o ed p ecision numbe .
Using a gene al pu pose sys em based on FPGA’s called
UNSHADES [4] we ha e de eloped and es ed he comple e
sys em. Typically his sys em would be made wi h a DSP
sys em ha would spend many ime p ocessing e e y image
in a se ial way and wouldn’ ake ad an age om he
pa allelism in e ms o accu acy.
This pape shows how o ake ad an age om he
UNSHADES-1 sys em. We ha e de eloped a ull con olled
sys em ha compu es he posi ion o he welding poin . In
sec ion II we’ll desc ibe he UNSHADES amewo k, in
sec ion III he Vision sys em is p esen ed, sec ion IV shows
he sys em design and inally he conclusions will be
p esen ed.
II. UNSHADES FRAMEWORK
A Ha dwa e Debugge is a comple e p og ammable
sys em ha can be cus omized o di e en applica ions.
Fig 1. UNSHADES-1 sys em


Hos Con olle
XC4010XL
In e ace &
Con olle
EPP
Da alink
P og ammable
Oscilla o
0 o 100MHz
VIRTEX De ice
XCV50 o XCV800
Low Speed Con ol
Signals
Gene al pu pose Signals
o Sys em In e ace
Main ad an age is ha he comple e sys em can be
moni o ized du ing execu ion ime. Main p ocesso s a e
subs i u ed by an ad anced FPGA and un embedded in o he
sys em i sel . I needed a o ally cus omized sys em can be
buil using he ac ual ne lis .
UNSHADES-1 gene al pu pose amewo k ha is based
on one Xilinx ad anced FPGA o emula ing he cus om
p ocesso o , in his case, image p ocessing. Main ad an age
o UNSHADES sys em is ha he use ha e ull con ol o
he execu ion o he FPGA due o he special cha ac e is ics
o his de ice. UNSHADES means Uni e sidad de Se illa
Ha dwa e Debugge Sys em, and allows a apid p o o yping
and he debugging o he p ocesso ci cui du ing execu ion.
Figu e 1 shows a gene al scheme o he sys em:
Main componen s o UNSHADES sys em a e lis ed below:
- An ad anced FPGA om Xilinx Vi ex amily [3].
This FPGA hos s he cus om design.
- PC so wa e, he human in e ace, ha p o ides a
comp ehensi e way o manipula ing he inne
in o ma ion o he FPGA.
- A p og ammable econoscilla o om Dallas
Semiconduc o DS-1075, ha gene a es he sys em
clock. This de ice can be uned om he PC.
- The e’s ano he FPGA ac ing as he in e ace
be ween he PC and he ad anced FPGA. All he
communica ions, p o ocol adap ing and signal
gene a ion a e suppo ed by de ice, which is smalle
han ad anced FPGA an has ixed i s p og am.
Communica ion be ween PC and UNSHADES ha dwa e is
pe o med using he EPP p o ocol ha can each 2 Mby es/s,
ha is speed enough o ou pu pose.
UNSHADES so wa e is a comple e sys em ha allows a
bi s eam managemen including some in e es ing ools:
downloade , in e nal egis e s inspec o , in e nal egis e
modi ie , oscilla o manage and design selec o , all ully
in eg a ed wi hin he Xilinx s anda d design low. Fo an
op imal speed in he bi s eam ans e , all he w i e and ead
ope a ions a e pe o med using pa ial econ igu a ion
echniques.
III. POSITIONING SYSTEM DESCRIPTION
The aim o his wo k is o build a simple, low cos and
e ec i e 3D digi al ision sys em o acking he posi ion
and p o ide a eedback in posi ioning au onomous obo s.
Sys ems like he one p esen ed can enla ge he e ec i e li e
o he au onomous obo s.
Main d awbacks o cons uc ing hese sys ems a e [5]:
1s . Vision sys ems a e as expensi e as he obo i sel .
2nd. In his kind o obo ics sys ems high h oughpu
echniques ha could gene a e he in o ma ion in eal- ime
a e needed.
3 d. The e’ e many p oblems ela ed wi h isual in o ma ion,
like shadows, illumina ions, uni, bi and h ee-dimensional,
hidden objec s,… e c.
The main objec i e is o o e come mos o hese di icul ies:
1s We p o ide wi h low cos CCD came as and ideo
decode s, he cos o he sys em s ongly depends on he
equi ed p ecision. Fo ou p o o ype we used small digi al
came as.
2nd The p ocessing echniques do no need o p ocess he
whole image o each came a. A simpli ied me hod allows a
esou ce sa ing ha makes a de ice like Vi ex V300 big
enough o he h ee came as. They a e p ocessed in pa allel
and he in o ma ion gene a ed simul aneously, so he esul is
p oduced using he same image ame aken in he same
ins an .
3 d All he imaging p oblems a e sol ed due o he use o
h ee came as placed a di e en posi ions. T inocula s e eo
ision inc eases he geome ic cons ain s and educes he
in luence o heu is ic cons ain s o s e eo ma ching [6-7].
Mos o cases we’ll ha e edundan in o ma ion, bu hidden
objec s will be o one came a, while he o he s will con inue
p oducing he in o ma ion. When he in o ma ion is
edundan , some p ecision imp o emen echniques can be
applied and be e weld acking will be ob ained.
4 h Ano he imp o emen comes om he image dis o ion
due o he lenses ha a e commonly cu e. Fo i s
compensa ion, a dis o ion ac o is in oduced o modi ying
he ypical linea model o he came a. The pa ame e s o
he compensa ion a e measu ed using pho og ame ic
me hods o calib a ion [8-9 ].
Ano he ad an ages a e in oduced when compa ed wi h
o he welding sys ems: The e’s no need o p oduce s imuli
o in e ac ion measu emen . Ou sys em can be conside ed a
passi e sys em, due o he e’s no need emi e s, because he
welding poin ligh ing is he signal aken o measu emen s.
The e’ e ano he echniques o p o iding he in o ma ion
ha ha e been ejec ed o ou pu pose: ul asound a e slow
and a objec s a e bad measu ed wi h many well known
p oblems. X ays need o use be ween he objec and he
senso an ampli ie o ans o ming X ays in o isible
spec um.
Finally, in a ed ha e many p oblems ela ed wi h
abso p ion, many dis o ions coming om he su ounding
emi e s.
In ou sys em he h ee images a e mixed, compensa ed,
s o ed and p ocessed in he same de ice.
IV. PROTOTYPE DESCRIPTION
Ou p o o ype has been designed and implemen ed in he
ela ed FPGA. Ou ex e nal ci cui only needs he ideo
acqui ing sys ems: TVCCD-30M Came as and he digi ize s.
Inside he FPGA, he compensa ions sys em and coo dina e
compu ing blocks a e p og ammed.
Figu e 2 p esen s a block diag am o he PCB boa d. I ’s
compound by he h ee digi ize s, he Phillips SAA7113H,
and ano he communica ion ea u es, like a RS232 line, a
VGA ou pu and a pa allel po .
came a 1
came a 2
came a 3
Video
Decode 3
F
P
G
A
In
e
a
ce
Pa allel
Po Se ial
Po
CDA
connec o
VGA
3D ision boa d
Video
Moni o
Compu e
UNSHADES
Sys em
Boa d
Video
Decode 2
Video
Decode 1
Fig. 2. Main PCB blocks.
Inside he FPGA, a ull digi al ci cui p ocess e e y image.
Figu e 3 shows he block diag am o he p ocesso . The i s
s age is o compensa e he lense dis o ion coming om he
came a lenses. Secondly a bina isa ion o he image sea ches
he welding poin , ead like a small a ea ob ained ha is a
small b igh ci cle. A e , his pa is s o ed in a memo y o
pos p ocessing.
P e iously a ci cui con igu es he digi ize s o an
adequa e image o ma .
The mos impo an block is he image p ocesso . The i s
s ep o he image p ocessing consis o inding he app opia e
h eshold ha would lead o a bina ised ound shape
co esponding o he welding poin p esen in he image.
E e y pixel is compa ed wi h he p e iously compu ed
h eshold, ha is ob ained om he image condi ion i sel .
We ha e selec ed he h eshold alue be ween he mean o
he luminance o he image and i s maximum alue. I he
luminance is below his alue he pixel has 0 alue (black)
and 1 alue (whi e) in ano he case. The line and pixel
numbe o whi e alues a e s o ed in he memo y.
Due o he ixed posi ion o e e y came a e e y decode
can p oduce wo alues (x,y), (y,z) o (z,y). The ideo
decode s p o ide in o ma ion wi h 625 lines and 1440 pixels
pe line. This in o ma ion is use ul o gene a ing e e y
coo dina e using simple coun e s.
Video
acqui e sys em
Dis o ion
compens
i
Dis o ion
compens
Dis o ion
compens
Bina isa ion Bina isa ion Bina isa ion
Memo y Memo yMemo y
P ocesso
Con igu e and
synch onise
decode s
Main
con ol
In e ace
VGA
In e ace
(X,Y,Z) (X,Y,Z)
Robo
Con ol Sys em
di i li ió
Main Blocks in
FPGA
PCB
PCB
Fig. 3. Inside FPGA p ocessing blocks.
The exac cen e o he welding poin is compu ed by
means o a g a i y cen e o each coo dina e, using hese
o mulas:
We’ll ob ain wo numbe s o each coo dina e because
hey’ e epea ed in wo memo ies. Some p ecision
imp o emen s can be p oduced i he came as a e placed
wi h a hal pixel o de ia ion o e e y coo dina e.
∑
∑
=
l
li
l
li
x
x
X
1
2
1
1∑
∑
=
m
mi
m
mi
y
y
Y
1
2
1
1∑
∑
=
n
ni
n
ni
z
z
Z
2
2
2
2
∑
∑
=
q
qi
q
qi
y
y
Y
2
2
2
2∑
∑
=
i
i
z
z
Z
3
2
3
3
∑
∑
=
p
pi
p
pi
x
x
X
3
2
3
3
2
31 XX
X+
=2
21 YY
Y+
=2
32 ZZ
Z+
=
The esul s a e ans e ed o he PC ha p o ides he
in o ma ion o he con olle s. The implemen a ion o his
sys em e esh he coo dina es e e y wo ames, due o he
pos -p ocessing da apa h. The comple e sys em has been
implemen ed in o a Vi ex 300, wi h a 60% o he
occupa ion. Sys em clock uns a 50MHz.
V. CONCLUSIONS
The pape p esen s a simple sys em o de e mine he
absolu e posi ion o a welding poin o an au onomous
obo s. Main ad an ages a e he cos , he p ecision and he
compu ing speed. In he inal e sion o his pape he
compu ing ime will be gi en. I ’s been es ima ed like a
coo dina e compu ing pe wo images, which can be
conside ed in e ms o obo speed eal- ime. This sys em
akes ad an age o he UNSHADES-1 sys em ha p o ides a
comple e, lexible and e y com o able pla o m o he
ision sys em.
V. REFERENCES
[1] D. Rdine Hundsdö e “Cos u as pe ec as. Aumen a el
empleo de la soldadu a po láse en el mecanizado de
chapa”. Re is a de Soldadu a y Tecnología de unión.
Año XII- Num 72, pp. 9-11 No iemb e/Dic 2001.
[2] P. Coi e , M. Chi ouze, “An In oduc ion o Robo
Technology”, Mc G aw Hill, 1983.
[3] Xilinx Da a Book. 2002
[4] M.A. Agui e, J. Tombs, A. To alba, L.G. F anquelo
“HADES-1: A apid p o o yping en i onmen based on
aad ances FPGA´s”. P oceedings o he Design o
Ci cui s and In eg a ed Sys ems, DCSI’01 Opo o 2001.
[5] J. M. Angulo Usa egui “Robó ica P ác ica. Tecnología y
aplicaciones”. Capí ulo 7: “Visión de Máquinas”,
Pa anin o, 5.ª ed., 2000.
[6] Ayache, “T inocula S e eo Vision o Robo ics”,IEEE
T ans. Pa e n Anal. & Machine In elligence, ol. 13,
no. 1, Jan. 1991.
[7] Yachida, “3D Da a Acquisi ion by Mul iple View”,
Robo ics Resea ch: Thi d In . Symp., Fauge as, O.D.
and Gi al , G., Eds. Camb idge, MA, MIT P ess, pp. 11-
18, 1986.
[8] B own D., 1971: Close-Range Came a Calib a ion.
Pho og amme ic Enginee ing, Vol. 37, No.8
[9] El-Hakim, S.F., 1986: Real-Time Image Me ology wi h
CCD Came as. Pho og amme ic Enginee ing and
Remo e Sensing, Vol. 52, No 11, pp. 1757-1766.