Cons ained Hop ield Neu al Ne wo k
o
Real-Time P edic i e Con ol
J.M.
Que o,
C.L.
Jane , and
L.G.
F anquelo,Membe ,
IEEE
Dp o
de Ingenie ia de Sis emas
y
Au omi ica, Uni . de Se illa, A da Reina Me cedes
s/n,
41012
Se illa
-
Spain
phone:
+34-5-4556873
;
ax:
+34-5-4556849
;
e-mail:
que oQg ex02.us.es
Absl cscd-The ha dwa e implemen a ion o
an
op imiza ion ne wo k wi h es ic ions o pe o m
eal- ime Gene alized P edic i e Con ol (GPC)
is desc ibed. The use
o
space-e icien s ochas-
ic a chi ec u e allows a a ealiza ion on a p o-
g ammable logic de ice.
As
a esul
a
p o-
g ammable neu al chip cop ocesso ha sol es
op imiza ion p oblems subjec o es ic ions has
been de elopped. Exp essions o ne wo k pa am-
e e s a e p o ided o implemen GPC. An adap-
i e con olle is achie ed using
RAM
memo ies
o s o e he ne wo k pa ame e s. Expe imen al
esul s om
a
simple implemen a ion o he con-
olle a e included.
I. INTRODUCTION
Robus , eal- ime con ol is essen ial o mos p ocesses
in indus y. Mode n con ol me hods ha e p o ed o be
e y e icien
[l].
Howe e hey a e high- ime consuming
specially when conside ing cons ain sa is ac ion, he e-
o e mo e powe ul p ocesso s a e needed. The GPC al-
go i hm consis s o op imizing a cos unc ion in o de
o achie e he bes expec ed con ol sequence
[2].
New
app oaches
[3]
conside cons ain s in con ol sequences
leading o cons ained op imiza ion p oblems.
We sugges he use o Hop ield ne wo ks o o e ide he
compu a ional e o o implemen such con olle . Ap-
plica ions o his class o neu al ne wo ks can be ound
elsewhe e
[4].
A i icial neu al ne wo ks consis o a se
o e y simple compu ing elemen s highly in e connec ed
ha pe o m an o e all du y. Due o he simplici y o he
compu a ional elemen s i is wo h ying o make spe-
ci ic elec onic implemen a ions a he han using a gen-
e al pu pose digi al compu e o simula e i s beha iou .
Elec onic ealiza ion o neu al ne wo ks can be aced
in di e en ways. On one hand analog ap oaches a e e y
simple in e ms o ci cui y and ha e as con e gence
imes, specially when hey a e compa ed wi h digi al im-
plemen a ions, hu on he o he hand hei p og amming
lexibili y is e y low. Digi al implemen a ions pe o m
high p og amming lexibili y and easy in e ace wi h gen-
e al pu pose compu e s bu hei e iciency in e ms o
consumed silicon a ea is e y low,
as
a
loa ing poin mul-
iplie is needed in e e y neu on o calcula e he p esy-
nap ic ac i i y. One way o ci cum en his p oblem is
employing s ochas icism o e alua e p esynap ic p oduc s
Ou aim is o design a neu al cop ocesso which eleases
he con ol compu e in he ac o y om his algo i hmic
compu a ional bu den. The simplici y o s ochas ic neu al
ne wo ks a chi ec u es
[?I
minimizes he amoun o ha d-
wa e needed, he e o e allowing he use o p og ammable
logic de ices o i s implemen a ion.
This pape is o ganized
as
ollows. In Sec ion I1 i is
discussed how Hop ield’s Neu al Ne wo k can be u ilized
o sol e cons ained linea quad a ic op imiza ion p ob-
lems and how such p oblems can be posed in e ms
o
adimen ional quan i ies, yielding
a
o mula ion ha can
be
implemen ed in s ochas ic a chi ec u e. In Sec ion I11
i is shown ha gene alized p edic i e con olle s apply a
con ol sequence o he sys em ha is being egula ed in
o de
o
minimize a quad a ic cos unc ion. Due o he
ac ha all ac ua o s ha e a limi ed ange o ac ua ions
and slew a es, linea es ic ions mus be imposed o his
op imiza ion p oblem. In Sec ion IV he s ochas ic a chi-
ec u e ha has been used o implemen he Hop ield neu-
al ne wo k and why his a chi ec u e is sui able o GPC
is desc ibed. Sec ion V is de o ed o he ac ual ha dwa e
ha has been used o ealize he neu al ne wo k and why
a p og ammable logic de ice has been used. In Sec ion
VI we conside speci ic examples o demons a e he neu-
al ne cop ocesso pe o mance. Finally, conclusions a e
d awn in sec ion VII.
[51,
PI.
11.
CONSTRAINED
HOPFIELD NEURAL NETWORK.
The way in which neu al ne wo ks could be applied o
sol e linea p og amming ne wo ks was i s desc ibed by
Tank and Hop ield
[4].
They sugges ed an op imiza ion
0-7803-1328-3/94$03.000
1994
IEEE
1727
ne wo k, ha is able o minimize
a
cos unc ion The Gene al P edic i e Con ol algo i hm consis s o
ap
plying
a
con ol sequence ha minimizes
a
mul is age cos
--I
T=A*V
(1) unc ion o he o m
Na
whe e
A'
is
an N-dimensional ec o o coe @ien s o he
J(N1, Nz)
=
E{
Y(
+
j
I
)
-
w(
+
j)12
N
a iables which a e he componen s o
V.
This mini-
s ain s among he a iables:
j=N1
miza ion is accomplished subjec o
a
se o
M
linea con-
Na-d
+
A[Au(
+j
-
(9)
j=l
(2)
whe e
E
{.}
is
he expec a ionbpe a o and
y( +j
I
)
is
an op imum j-s ep ahead p edic ion o he sys em ou pu
on da a up o ime
.
u( ) and y( ) a e he con ol and
Gj
=
[Zjl,
6j2,.
.
.,
3jnlT
(3)
whe e he
6,,
o
each j, con ain he
N
a iable coe icien s
in
a
cons ain equa ion and he
Bj
a e he bounds.
Fo
he case o
a
linea quad a ic op imiza ion p oblem,
i can be conside ed
an
ene gy unc ion o he o m
M-1
whe e
F(z)
is
a
p imi i e o
(z)
gi en by
kz
o
z
>
0
0
o
z
<=
0
(5)
Conside equa ion
4.
We shall deno e he ypical
al-
ues
o
Gj;,
Ij
and
j
by
Go,
IO
and
VO.
These numbe s
a e such ha he adimen ional quan i ies
xji,
pj
and
qj
de ined in
6
ake absolu e alues anging om
0
o
1.
ou pu sequence o he plan .
NI
and
Nz
a e he minimum
and maximum cos ing ho izons.
A
is
weigh ing coe icien
and w( +j)
is
a u u e se -poin
o
e e ence sequence. The
objec i e o p edic i e con ol is o compu e he u u e
con ol sequence u( ), u( +l),
...
in such
a
way ha he
u u e plan ou pu y( +j)
is
d i en close o w( +j). This
is accomplished by minimizing
J(N1, Nz).
Howe e ,
i
can
be compu a ionally p ohibi i e o eal ime applica ions.
In p ac ice, he no mal way o using GPC
is
o compu e
u( )
and apply
i
o he p ocess. I
~( )
iola es he con-
s ain i
is
sa u a ed o he bounds, ei he by he con ol
p og am
o
by he ac ua o . The case o
u(
+
l),
--e,
u(
+
N)
iola ing he cons ain s
is
no e en conside ed
as
in mos cases he signals a e no e en compu ed. This
way o ope a ing es ic s he op imalli yo he GPC when
cons ain s a e iola ed. Fu he mo e, he main pu pose
o he GPC, which
is
o op imize equa ion
(9),
is
no longe
alid and he
bes
expec ed con ol
is
no achie ed.
Mos p ocesses in indus y can be desc ibed by he ol-
lowing ans e unc ion:
Subs i ui ing in
4
Xij
=
Xji
(8)
I
VO
is
chosen
so
ha
=
1
we ob ain he same
linea -quad a ic op imiza ion p oblem in e ms o adimen-
ional quan i ies. This o mula ion is sui able o be im-
plemen ed in
a
s ochas ic a chi ec u e
as
i will be seen
la e .
111
APPLICATION
TO CONSTRAINED GENERAL
PREDICTIVE CONTROL.
I an op imal p edic o is used,
as
shown in
[8],
he dead
ime can be igno ed o designing pu poses.
I we compa e he ene gy unc ion
(4)
o
a
Hop ield ne
o he cos unc ion
(9)
o he GPC, conside ing ha he
ou pu o he neu al ne
V;.
co esponds o he con ol
se
quence
uj,
we ob ain he exp essions o he pa ame e s o
he ne wo k
as
un ions o he sys em pa ame e s, con ol
pa ame e s and he sys em inpu and ou pu sequences o
a
gi en con ol ho izon
[9].
As
an
example, esul s o
his
p ocedu e a e shown in Appendix
I
o sys ems desc ibed
in
(10).
I
is
ema kable ha equa ions which de ine he conduc-
ances
Zj
only depend on he sys ems pa ame e s, whils
equa ions de ining he ne inpu s also depend on he sys-
em inpu and ou pu sequences and he e e ence. The
alues o
Ci
and
R,
a e chosen a bi a ily. We+can_de ine
he cons ain subne iden i ying he e ms o
0,
2
Bj
o each es ic ion.
1728
......................................
!PI--;
PREDICTOR
J
P(Z‘
)
P-l
71’
I
Figu e
1:
Con ol Scheme
The esul ing con ol scheme is shown in Fig.
1.
The
neu al con olle p o ides he inpu signal sequence o he
plan . The iden i ie es ima es he plan model pa ame-
e s om i s inpu and ou pu signals.These alues a e ed
in o block
T
ha ep esen s he calculus o he polynomi-
als o ob ain he conduc ances o he neu al con olle . In
block
I
in ensi y dependen sou ces a e calcula ed acco d-
ing o he cu en e e ence. As i will be explained la e ,
block
I
compu a ions a e also pe o med by he neu al
ne wo k while iden i ica ion and
Tij
e alua ions a e ca -
ied ou by he con ol compu e .
IV
ARCHITECTURE.
A cons ained Hop ield neu al ne wo k will be consid-
e ed in his a icle
as
a
se o sa u a ing linea in eg a o s
whose inpu s a e linea combina ions o o he in eg a o s’
alues and ime a ying e ms. The dynamics
o
hese
e ms a e supposed o be much slowe han ne dynamics
so
ha hey may be ega ded
as
cons an s du ing in e-
g a ion. Analog e sions o his con olle can be ound
elsewhe e
[lo],
bu hey lack o p og ammabili y.
S ochas ic sys ems use s ochas ic signals whose alues
andomly ake he alue
0
o
1.
The a e age o hese
I
.
F -
mm
A
Ii
l==l
mm
:...................................a
Figu e
2:
S ochas ic Neu al Ne P ocesso
alues can be iewed
as
an analog alue in he ange
[-
l,l].
S ochas ic signals can be mul iplied using only a
simple AND ga e.
The e o e, p oduc e ms in he new al ne wo ks can
be cacula ed by a s ochas ic mul iplie . The op imiza-
ion p oblem should be posed in e ms o adimen ional
qua i ies
as
cons an s and a iables a e o be ansla ed
in o s ochas ic s eams o pulses. This has been done in
sec ion
11.
The high-le el a chi ec u e is shown in Fig.
2.
I con-
sis s o
a
s ochas ic neu al p ocesso , which is basically
composed o a s ochas ic p oduc e alua o and a s a e
machine con olle , and wo andom access ead and w i e
memo ies. No ice ha neu on ac i i y alues e ol e due
o ei he o he neu ons’ ac i i y alues
o
ne inpu s. A
any a e hese e ms may be conside ed
as
p oduc s ha
can be compu ed by he s ochas ic p oduc e alua o .
Cons ained Hop ield neu al ne wo k’s in e connec ion
weigh s and ne inpu s’ cons an e ms a e s o ed in one o
he memo ies and neu on ac i i ies and a iable e ms o
he ne inpu s a e s o ed in he emaining one. All hese
magni udes may ei he ake posi i e
o
nega i e alues.
Two kinds o neu ons a e in ol ed in he ne wo k’s e o-
lu ion:
sys em neu ons
and
es ic ion neu ons.
Sys em
neu ons a e sa u a ing linea in eg a o s ha can ei he
ake posi i e
o
nega i e alues. Res ic ion neu ons a e
sa u a ing nonlinea in eg a o s. They ake nega i e al-
ues when he p oblem es ic ions a e iola ed and o ce
he sys em neu ons o e ol e in such a way ha cons ic-
ions a e ul illed.
Neu ons’ in o ma ion (ac i i y alue, sign and ype
o
neu on) a e se ially ad essed by he con olle and loaded
in he up/down coun e
N
and la ches
S
and
T.
The
1729
0
aooo
1000
0
-1000
-2000
----.-.-.-.
.-.-
.-.-
-.-.
-.-.-
--.
I
I
I
I
I
I
T
o
zoo00
~oooo
60000
eoow
io0000
iaoooo
uoooo
160000
iaoooo
ZOQOOO
-3000
I
o
I . . ioP.
Figu e
4:
Con olle ’s chip ansien esponse.
Figu e
3:
Tes
boa d pho og aph.
s ochas ic p oduc e alua o calcula es he inhibi o y
o
exci a o y in luence ha neu ons
(uj,
j
=
l..n)
(and in-
pu ime a ying e ms) ha e o e he one which has been
loaded
(ui).
To
achie e his goal hese alues and hei
ela ed cons an s a e se ially ad essed by he con olle
and ead. Random numbe s a e hen gene a ed and com-
pa ed wi h hem, p oducing wo s ochas ically indepen-
den s eams o pulses ha a e ANDed and ed o logic
block
B.
This logic block ei he inc emen s
o
dec emen s
he up/dowm coun e acco ding o he signs o he h ee
numbe s in ol ed and he kind o neu on ha
is
being in-
eg a ed. I also changes he sign bi
(S
la ch) whene e a
ze o c ossing akes place. When he p ocess has inished
he new compu ed alue
o
o
i
is
w i en in he ex e nal
memo y and ano he neu on
is
loaded.
PHYSICAL REALIZATION.
A neu al op imiza ion p oblem accele a o
has
been de el-
opped (Fig.
3)
and es ed ollowing he desc ibed s uc-
u e. The s ochas ic neu al p ocesso has been imple-
men ed in an E asable P og amable Logic De ice. EPLDs
can be designed and p og amed in-house, elimina ing he
long enginee ing lead imes and high ooling e o s and
cos s o ull cus om
o
semicus om de ices. They a e
also
e y app op ia e
o
sho se ies ab ica ion whe e
an Applica ion-Speci ic In eg a ed Ci cui would be un-
economical. An Al e a EPM5130 de ice has been chosen
o de elop his applica ion
[lo].
The numbe o sys em es ic ion neu ons and ne -
wo k
inpu signals a e ully p og ammable. Two ex e nal
RAMS
ha e been used
o
da a s o age, one o neu ons’
alues and o he o synap ic weigh alues. Ne inpu s’
ime- a ying e ms a e sampled by he hos compu e and
w i en wi hin he neu on
RAM.
Thei associa ed con-
s an s a e plan model pa ame e s which we e s o ed by
he hos compu e in he weigh RAM when he ne was
con igu ed. Ne con igu a ion is accomplished by simply
loading he ne ’s in e connec ion weigh s and ne inpu s’
cons an e ms in one o he ex e nal memo ies and by
loading he ini ial neu ons’ ac i i y alues and ne inpu s’
ime a ying ini ial alues in he o he one.
The sys em’s clock may be ei he ex e nal, in case moni-
o ing he ne ’s e olu ion
is
desi ed,
o
in e nal, o no mal
ope a ion. Mos indus ial p ocesses ha e slow dynamics.
Fo
his eason high speed memo im ha e no been used,
yielding
a
sys em’s clock speed o
5MHz.
VI
APPLICATION.
As an example he sys em o be con olled
has
a
pu e
delay
o
one sampling pe iod and is desc ibed by he ol-
lowing polynomials:
A(.-’)
=
1
-
0.4~-I
-
0.32.~-~
B(z-1)
=
0.1
Fo he i s
se
o
es s
he u u e e e ence was consid-
e ed o be
a
p e iously known squa e wa e. The weighing
ac o
A
was
made equal o
0.0001.
The maximun con ol
signals we e cons ained by
10,
he slew a es we e con-
side ed o be limi ed by
5
and
a
con ol ho izon o
5
has
been used.
The ansien esponse o he neu al ne wo k
is
shown
in Fig.
4
when calcula ing he
i s
con ol signal
se
quence. When he s able s a e
is
eached,
V(5)
ep esen s
he con ol signal
o
be applied o he sys em. R(l) and
R(2) a e he ou pu s o he es icc ions
V(5)
<
10
and
1730
--
-10
1
1
0
5
15
20 a5
30
35
10
I
o
s ep*
Figu e
5:
Sys em's closed loop beha iou .
V(5)
-
V(4)
<
5
espec i ely. This s able s a e ma ches
heo e ical esul and i s compu a ion ime has been
0.3
seconds.
In o de o check he pe o mance o he neu al con-
olle , he esponse o he closed loop sys em has been
s udied when changing he e e ence. In Fig.
5
' e ' ep-
esen s he applied e e nce signal''V(5)' he s eady s a e
o he i s sys em neu on in he con olle and
'Y'
he
ou pu o he plan
as
unc ions o he numbe o he con-
olle 's ac ua ions.
VI1
CONCLUSIONS.
Digi al s ochas ic a chi ec u es a e a e y e icien way
o ealize neu al ne wo ks
as
hey signi ican ly educe he
needs
o
silicon a ea. We ha e implemen ed
a
gene al pu -
pose e sion o a cons ained Hop ield ne wo k on a p o-
g ammable logic de ice. This cop ocesso sol es any con-
s ained quad a ic op imiza ion p oblem. I has been used
o implemen he Gene al P edic i e Con ol p oblem in
o de o elease he con ol compu e om he hea y com-
pu a ional bu den o his algo i hm. Exp essions o ne
pa ame e s a e p o ided o a common plan model which
pe mi an adap i e con ol. The chip has been inco po-
a ed in o a pe sonal compu e . Resul s ob ained wi h
his expe imen al chip ha e been epo ed.
APPENDIX
I
NETWORK PARAMETERS
Fo
his sys em, he ne pa ame e s a e gi en by he ol-
lowing equa ions:
i=l
+
2Au -1
i
j
odd
i
j
e en
2n--i
..
1=J
n-j+l
whe e
odd(i)
=
emainde (i/2)
and
e en(i)
=
1
-
odd(i).
REFERENCES
[l]
M. Gopal.
Model Con ol Sys em
Theo y.
Wiley
Eas e n Limi ed
1985.
[2]
D. W. Cla ke C. Moh adi and P.S.
Tu s.
Gene al-
ized P edic i e Con ol. Pa
I
The Basic Algo i hm.
Au oma ica
~01.23, pp.
137-148. 1988
[3]
E.
F. Camacho. Cons ained Gene alized P edic i e
Con ol
IEEE
T ans.
on
Au oma ic Con ol,
ol 38,
pp.
327-332, 1993.
[4]
D. W. Tank and
J. J.
Hop ield. Simple Neu al Op i-
miza ion Ne wo ks: An A/D Con e e , Signal De-
cision Ci cui , and a Linea P og amming Ci cui .
IEEE
T ans. on Ci cui and Sys ems,
~01.35, pp.
554562, 1988
1731

[SI
D.E.
Van
den Bou and T.K. Mille
I11
A Digi al
A chi ec u e Employing S ocha icism
o
he Sim-
ula ion
o
Hop ield Neu al Ne s.
IEEE
Dans. on
Ci cui and Sys ems,
o1.36, pp. 732-738, 1989
[6]
M.
S.
Mel on,
T.
Phan,
D.
S.
Ree es and
D.
E.
Van
den Bou The TInMANN
VLSI
Chip
IEEE
%ns.
on Neu al Ne wo ks,
~01.3, pp.375384, 1992.
[7]
Y.
Kondo and
Y.
Sawada F’unc ional Abili ies
o
a
S ochas ic Logic Neu al Ne wo ks
IEEE
Dans. on
Neu al Ne wo ks,
o1.3, pp.434443,1992.
[8]
E.
F.
Camacho and
J.M.
Que o. P ecompu a ion
o
Gene alized P edic i e Con olle s.
IEEE
Tkans. on
Au oma ic Con ol,
ol. AG36, pp. 852-859, 1991.
[9]
J.
M. Que o and
E.
F.
Camacho. Neu al Gene al-
ized P edic i e Sel - unning Con olle s.
P oc.
IEEE
Cong ess on Sys ems Con ol,
pp 160-163, 1990
[lo]
J.M.
Que o, L.G. F anquelo and
E.F.
Cama-
cho. “Ne wo ks o cons ained P edic i e Con ol”
.
IEEE
Dans.
on Ci cui s and Sys ems,
ol. 40, 621-
626, 1993.
[ll]
Al e a Co p.
Al e a Da a Book,
1990.
1732