Thing Complex Fuzzy Systems by Supervised Learning Algorithms

Thing Complex Fuzzy Sys ems by Supe ised Lea ning Algo i hms F. J. Mo eno-Velo, I. Ba u one, R. Senhadji, S. Sbnchez-Solano Ins i u o de Mic oelec 6nica de Se illa (IMSE-CNh4) Cen o Nacional de Mic oelec 6nica - CSIC A da. Reina Me cedes, s/n. Edi . CICA, E-41012, Se illa, Spain x uzzy- eam@ imse.cnm.es Abs ac Tuning a uzzy sys em o mee a gi en se o inpu ou pu pa e ns is usually a di icul ask ha in ol es many pa am- e e s. This pape p esen s an s udy o di e en app oaches ha can be applied o pe o m his uning p ocess au oma - ically, and desc ibes a CAD ool, named x sl, which allows applying a wide se o hese app oaches: (a) a la ge numbe o supe ised lea ning algo i hms; (b) di e en p ocesses o simpli y he lea ned sys em; (c) uning only speci ic pa am- e e s o he sys em; (d) he abili y o une hie a chical uzzy sys ems, sys ems wi h con inuous ou pu (like uzzy con- olle ) as well as wi h ca ego ical ou pu (like uzzy classi- ie s), and e en sys ems ha employ use -de ined uzzy unc ions; and, inally, (e) he abili y o employ his uning wi hin he design low o a uzzy sys em, because x sl is in- eg a ed in o he uzzy sys em de elopmen en i onmen X uzzy 3.0. 1. In oduc ion Tuning he sys em beha io is o en one o he mos di - icul ask in he design low o a uzzy sys em. Much o he uning e o is dedica ed o sea ch a p ope con igu a ion o he sys em pa ame e s, because he e is usually a la ge numbe o pa ame e s. To con on his ask, supe ised lea ning algo i hms a e commonly used as au oma ic uning me hods. In supe ised lea ning echniques, he desi ed sys- em beha io is desc ibed by a se o inpu ou pu pa e ns and he objec i e is o minimize he e o be ween he de- si ed and he cu en sys em beha io . The pape is s uc u ed as ollows. Sec ion 2 p esen s he di e en e o unc ions ha can be employed in supe ised lea ning. Sec ion 3 desc ibes b ie ly some amilies o supe - ised lea ning algo i hms. The p oblem o uning unde cons ain s is add essed in Sec ion 4, while Sec ion 5 sum- ma izes some simpli ica ion p ocesses ha can be easily pe o med a e uning. Mos o hese possibili ies has been included in o a CAD ool, named x sl, so as o au oma e he uning p ocess o complex uzzy sys ems. This ool is b ie - ly desc ibed in Sec ion 6. Finally, Sec ion 7 and 8 show se - e al examples o illus a e he uning o sys ems wi h ei he con inuous o ca ego ical ou pu s. 2. The e o unc ion The i s s ep in elabo a ing a supe ised lea ning p oc- ess supposes desc ibing sys em de ia ion by means o a unc ion, known as e o unc ion. A e y commonly used e o unc ion is he mean squa e e o (MSE): whe e N is he numbe o da a pa e ns, M is he numbe o ou pu a iables in he sys em, y, is he j- h ou pu gene a - ed by he sys em o he i- h pa e n, j.. is he co ec ou - pu exp essed by he aining pa e n, and j is he ange o he j- h ou pu ha is used o no malize he de ia ions. I can be use ul o he designe o selec he ela i e in- luence o e e y ou pu a iable on he global de ia ion om i s in ended beha io . The ollowing unc ion can be used in his case: V whe e wj is he weigh o he j- h ou pu a iable on he glo- bal sys em e o . These weigh s should be no malized so as o sum 1. I can be also use ul o employ he absolu e alue ins ead o he quad a ic e o , wi h he co esponding op ions o a iable no maliza ion and weigh accoun ing: The abo e exp essions assume a nume ical ou pu om he uzzy sys em. Howe e , i is possible o de ine uzzy sys ems whose ou pu s a e linguis ic labels, as is he case o classi ie s. In hese sys ems, he ou pu alue is he linguis- ic label p esen ing he highes ac i a ion deg ee as a esul o he in e ence p ocess. A common de ini ion o he de i- This wo k has been padally suppo ed by he Spanish CICYT P ojec TIC2001-1726. 0-7803-7810-5/03/517.00 WO03 IEEE 226 The IEEE In e na ional Con e ence on Fuzzy Sys ems a ion in he beha io o his kind o sys em is he numbe o classi ica ion e o s: 11 CE = -. -. z 6.. N M i, II (4) whe e Si is 1 when he classi ica ion o he pa e n has been inco ec and 0 o he wise. This ype o unc ion conside s equally all classi ica ion ailu es, wi hou aking in o ac- coun he dis ance om a co ec classi ica ion. To conside his in o ma ion i is necessa y o add a new e m like he ollowing: ACE = -' Z6.. NM+I i,i J (5) wi h whe e Gj is he ac i a ion deg ee o he co ec label, and ai is he ac i a ion deg ee o he label selec ed by he sys- em. Ano he way o aking in o accoun he dis ances om igh classi ica ions is o conside he ollowing classi ica- ion squa e e o , which is a di e en iable unc ion: wi h 1 i y..=j.. 0 'I 0 i y.. j.. { 'J [J p.. = The choice o an adequa e e o unc ion o he lea ning p ocess depends bo h on he ype o uzzy sys em o be uned and on he algo i hm selec ed o pe o ming he p ocess. Fo example, i he lea ning algo i hm belongs o he amily o g adien descen algo i hms, he e o unc ion mus be de i able, so ha he classi ica ion e o s CE and ACE can no be used. 3. Supe ised lea ning algo i hms Since he objec i e o supe ised lea ning algo i hms is o minimize an e o unc ion, hey can be conside ed as al- go i hms o unc ion op imiza ion. Some supe ised lea n- ing algo i hms ha can be used o une uzzy sys ems a e b ie ly desc ibed in he ollowing. 3.1. G adien descen algo i hms The equi alence be ween uzzy and neu al ne wo ks led o apply he neu al lea ning p ocesses o uzzy in e ence sys ems. In his sense, a well-known algo i hm employed in uzzy sys ems is he BackP opaga ion algo i hm, which modi ies he pa ame e alues p opo ionally o he g adien o he e o unc ion in o de o each a local minimum. Since he con e gence speed o his algo i hm is slow, se - e al modi ica ions we e p oposed like using a di e en lea ning a e o each pa ame e o adap ing heu is ically he con ol a iables o he algo i hm, hus leading o Back- P opaga ion wi h Momen um, Adap i e Lea ning Ra e, Adap i e S ep Size o Manha an algo i hms. An in e es ing modi ica ion ha imp o es g ea ly he con e gence speed is o ake in o accoun he g adien alue o wo successi e i - e a ions. This idea is ollowed by he algo i hms Quickp op and RP op [l]. 3.2. Conjuga e g adien algo i hms Since he g adien indica es he di ec ion o maximum unc ion a ia ion, i may be con enien o gene a e no only one s ep bu se e al s eps which minimize he unc ion e o in ha di ec ion. This idea, which is he basis o he's eep- es -descen algo i hm, has he d awback o p oducing a zig- zag ad ancing because he op imiza ion in one di ec ion may de e io a e p e ious op imiza ions. The solu ion is o ad ance by conjuga e di ec ions ha do no in e e e each o he . The se e al conjuga e g adien algo i hms epo ed in he li e a u e (like Polak-Ribie e, Fle che -Ree es, Hes enes-S ie el, and One-s ep Secan ) di e in he equa- ions used o gene a e he conjuga e di ec ions. The main d awback o he conjuga e g adien algo i hms is he imple- men a ion o a linea sea ch in each di ec ion, which may be cos ly in e ms o unc ion e alua ions. The line sea ch can be a oided by using second-o de in o ma ion, as done by he scaled conjuga e g adien [2]. 3.3. Second-o de algo i hms A o wa d s ep owa ds speeding up he con e gence o lea ning algo i hms is o make use o second-o de in o ma- ion o he e o unc ion. Since he calculus o he second de i a i es is complex, one solu ion is o app oxima e he Hessian by means o he g adien alues o successi e i e - a ions. This is he idea o he algo i hms o B oyden-Fle ch- e -Golds@-Shanno and Da idon-Fle che -Powell [3]. An special case is when he unc ion o minimize is a quad a ic e o because, in his case, he Hessian can be app oxima ed by only he i s de i a i es o he e o unc ion, as done by he Gauss-New on algo i hm. Since his algo i hm can lead o ins abili y when he app oxima ed Hessian is no de ined posi i e, he Ma qua d -Le enbe g algo i hm sol es his p oblem by in oducing an adap i e e m. 3.4. Algo i hms wi hou de i a i es The g adien o he e o unc ion can no be always cal- cula ed because i can be oo cos ly o no de ined. In hese cases, op imiza ion algo i hms wi hou de i a i es can be 227 The IEEE In e na ional Con e ence on Fuzzy Sys ems employed. An example is he Downhill Simplex algo i hm, which conside s a se o unc ion e alua ions o decide a pa- ame e change. Ano he example is Powell’s me hod, which implemen s linea sea ches by a se o di ec ions ha e ol e o be conjuga e 141. These algo i hms a e oo much slowe han he p e ious ones, A bes solu ion can be o es- ima e he de i a i es om he secan s o o employ no he de i a i e alue bu i s sign (as RP op does), which can be es ima ed om small pe u ba ions o he pa ame e s. 3.5. S a is ical algo i hm All he abo e commen ed algo i hms do no each he global bu a local minimum o he e o unc ion. The s a is- ical algo i hms can disco e he global minimum because hey gene a e di e en sys em con igu a ions ha sp ead he sea ch space. One way o b oadening he space explo ed is o gene a e andom con igu a ions and choose he bes o hem. This is done by he blind sea ch algo i hm whose con e gence speed is ex emely slow. Ano he way is o pe o m small pe u ba ions in he pa ame e s o ind a be - e con igu a ion as done by he algo i hm o i e a i e im- p o emen s. A be e solu ion is o employ simula ed annealing algo i hms [4]. They a e based on an analogy be- ween he lea ning p ocess, which is in ended o minimize he e o unc ion, and he e olu ion o a physical sys em, which ends o lowe i s ene gy as i s empe a u e dec eases. Se e al annealing schemes (like linea , exponen ial, classic, as o adap i e) ha e been p oposed, p oducing di e en e sions o he simula ed annealing algo i hm. 4. Tuning uzzy sys ems unde cons ain s The pa ame e s o adjus in a uzzy sys em usually ha e o mee se e al cons ain s. Fo ins ance, when uning he pa ame e s o a Gaussian membe ship unc ion, he lea ning algo i hms should always ejec a nega i e alue o he pa- ame e ep esen ing he wid h o he unc ion. The con- s ain s ha usually appea when uning a uzzy sys em pa ame e .pi, a e he ollowing: “pi <= cons an ”, < con- s an ”, ‘pi >= cons an ”, “pi > cons an ”, o ‘>i < pj”. The la e ones appea , o ins ance, be ween he h ee poin s ha can de ine a iangula membe ship unc ion. They a e he mos di icul cons ain s o main ain and should be a oided as much as possible. In his sense, a iangula unc- ion is be e de ined by i s cen e , wid h, and slope. S a is ical algo i hms manage cons ain s easily, by di- ec ly ejec ing he andom con igu a ion gene a ed i i does no mee he cons ain s. On he o he hand, he algo- i hms based on any kind o g adien descen gene a e he same (de enninis ic) displacemen a a gi en i e a ion, so ha he solu ion is no o ejec i i i is o bidden (i would be again gene a ed in he nex i e a ion) bu o change i in o ano he displacemen accep ed by he sys em. The usual so- P2 P2 I I c I I Figu e 1: Lea ning unde cons ain s. lu ion is o y an smalle displacemen in he same di ec- ion, as shown in Figu e la. The p oblem is ha his solu ion does no dis inguish be ween pa ame e s, so ha he con- s ain on one pa ame e can limi no only he displacemen o ha pa ame e bu also o o he ones (as shown in Figu e la). This p oblem can be a oided by managing he pa ame- e s independen ly. This means, in he example o Figu e I, ha he sys em could be mo ed o he inal poin shown in Figu e Ib. Fo hose cons ain s ela ing se e al pa ame e s (as hose o he iangle shown in Figu e IC), a good solu ion is o conside hem as pa icles subjec o inelas ic collisions in a one dimensional space (Figu e Id). Ano he in e es ing poin o ema k is ha he s able poin eached by he sys em a e lea ning unde cons ain s will p o ide o no he minimum possible e o depending on he uning algo i hm employed. When he p oposed dis- placemen ollows he g adien di ec ion, he sys em e ol es o a poin whe e he g adien componen s o e he non cons ained pa ame e s a e null, ha is, o he poin o he on ie whe e he e o is minimum ( he poin “b” in Figu e 2). On he o he hand, i he lea ning algo i hm gen- e a es a di ec ion which is no pa allel o he g adien (like ha shown wi h a dashed line in Figu e 3). he s able poin o he sys em will no p o ide a minimum e o ( he poin “ 1, . a in Figu e 2). 5. Simpli ica ion p ocesses Supe ised leaming can be used no only o une uzzy sys ems bu also o help ob aining uzzy models in iden i i- o bidden egion Figu e 2: S able poin s depending on he algo i hm. 228 The IEEE In e na ional Con e ence on Fuzzy Sys ems ca ion p oblems. Simpli ying he uzzy sys em ob ained a e a uning p ocess allows ex ac ing a aluable in o ma- ion abou he logical s uc u e o he sys em. One o hese simpli ica ion p ocess consis s in de ec ing and dele ing hose uzzy ules and membe ship unc ions ha a e ne e ac i a ed su icien ly by any o he aining inpu lou pu pa e ns. A ypical esul o he uning p ocess is ha he membe - ship unc ions co e ing he ou pu a iables o e lap each o he in a high deg ee. A clus e ing p ocess o e hese unc- ions d i es o a good and simple uzzy sys em. This clus- e ing can be made au oma ically by means o he Ha d C- means algo i hm, whe e he numbe o clus e can be ixed manually, o can be selec ed by some clus e e alua ion unc ions [5]. 6. The X sl ool In o de o au oma e he uning p ocess o a uzzy sys em we ha e de eloped he ool x sl. X l includes all he e o unc ions and supe ised lea ning algo i hms desc ibed in Sec ions 2 and 3, and apply he solu ions desc ibed in Sec- ion 4 o allow lea ning e icien ly unde cons ain s. Sim- pli ica ion me hods desc ibed in Sec ion 5 a e also included and can be execu ed p io o o a e he lea ning algo i hms. In addi ion, he sys em pa ame e s o une can be selec ed by a g aphical in e ace. Xjsl allows he use o apply supe ised lea ning algo- i hms o uzzy sys ems speci ied wi h he XFL3 language [6], he o mal language o X uzzy 3.0 [7]. XFL3 pe mi s he desc ip ion o complex uzzy sys ems wi h hie a chical ule bases. Besides, he e is no limi a ion in he numbe o ules wi hin a ule base, linguis ic a iables, o linguis ic labels co e ing he a iables. The ules suppo complex logic ela ions in he p emise pa (wi h conjunc ions, dis- junc ions, and linguis ic hedges), and hese ope a o s as well as he implica ion ope a o s, membe ship unc ions, o de uzzi ica ion me hods can be de ined eely by he use . The language XFL3 is he nexus be ween he di e en Figu e 3: Main window o he ool xjsl. 229 X uzzy 3.0 ools (dedica ed o desc ip ion, lea ning, e i i- ca ion, o syn hesis o uzzy sys ems). Figu e 3 illus a es he main window o x sl. This win- dow is di ided in o ou pa s. The le uppe co ne is he a ea o con igu e he lea ning p ocess. The p ocess s a e is shown a he igh uppe pa . The cen al a ea illus a es he e olu ion o he lea ning, and he bo om pa con ains se - e al con ol bu ons o un o s op he p ocess, o sa e he e- sul s, and o exi . 7. Fuzzy sys ems wi h con inuous ou pu Since uzzy sys ems wi h con inuous ou pu can be seen as in e pola o s, le us conside , as example, he p oblem o app oxima ing he unc ion shown a Figu e 4a. We con- side a wo-inpu uzzy sys em wi h 7 Gaussian membe - ship unc ions pe inpu , hus con aining 49 ules. The Weigh ed Fuzzy Mean is selec ed as de uzzi ica ion me hod. Ini ially, inpu membe ship unc ions a e homoge- neously dis ibu ed in hei uni e se o discou se, while he 49 ou pu unc ions a e equal and cen e ed in hei uni- e se. All he pa ame e s o hese membe ship unc ions as well as he weigh s a e going o be uned, which means 126 pa ame e s. Figu es 4b, c, and d show he e olu ion o he sys em beha io while being uned by he di e en lea ning algo- i hms p o ided by x sl. The g adien descen algo i hms a e shown in Figu e 4b. I can be seen ha modi ica ions o he BackP opaga ion algo i hm no o iously inc ease he con e gence speed. Figu e 4c is dedica ed o he conjuga e g adien and second o de algo i hms. As i is shown on his igu e, hese algo i hms a e signi ican ly as e han he s eepes descen algo i hm, especially BFGS and Ma - qua d -Le enbe g algo i hms. Algo i hms wi hou de i a- i es and s a is ical algo i hms a e shown in Figu e 4d. These algo i hms a e se e al o de s o magni ude slowe han he p e ious ones, so hei use is only ecommended when hose a e disca ded (in non-de i able sys ems, o ins ance). I can be seen ha Powell's algo i hm is much as e han Downhill Simplex algo i hm. Conce ning he s a is ical algo i hms, Simula ed Annealing inc eases he con e gence speed wi h espec o Blind Sea ch o I e a i e Imp o emen algo i hms. The exis ence o non-linea pa ame e s gene a es he p esence o se e al local minima in he uning p ocess. The e o e, i is no possible o asse wha is he bes algo- i hm, since a e y as algo i hm may be some imes d i en o a local minimum a away om he op imum beha io . A solu ion o his p oblem is o make se e al uning p ocesses wi h di e en andom ini ial con igu a ions, selec ing he bes o he lea ning esul s. Wi hin he lea ning p ocess. he membe ship unc ions o he ou pu a iable end o g oup a ound some common The IEEE In e na ional Con e ence on Fuzzy Sys ems 3 Adap i e Lea ning Ra e 'I lo3 lo4 CPUTime 1 ob RMSE (b) 100 10' d 1 U- 10.' 1 0-2 10-3 104 10-5 1 S eepes Descen 2 Ccnjuga eG adien 3 Scaled Conjuga e- G adien 4 BFGS 5 1 5DFP 1 IO*$ 10 10' le lo3 IO' CPU Time RMSE (C) 1 Downhill Simplex 2 Powell 3 Blind Sea ch 4 I e a i e Impm e- 5 Sim. Annealing men o* 10 10' le 10' IO4 10 CPUTime (d) Figu e 4: Compa ison o he di e en algo i hms. o ms (Figu e 5a). Applying he clus e ing p ocess sup- po ed by x l, he 49 membe ship unc ions a e educed o 6 (Figu e 5b). This educ ion leads o he simpli ied ule base shown in Figu e 5c. The use o hie a chical s uc u es allows simpli ying he desc ip ion o a sys em because complex beha io s can be gene a ed by composing simple ule bases. A ele an ad- an age o he x s/ ool is i s abili y o adjus hie a chical sys ems. To illus a e his ype o lea ning p ocess, we will conside again he p oblem o app oxima ing he beha iou a Figu e 4a, bu now using a hie a chical uzzy sys em wi h wo cascaded ule bases, like ha shown in Figu e 6a. The ini ial desc ip ion we ha e aken o he i s ule base is e y simple: wo uzzy se s o each inpu a iable and ou single on alues o he ou pu , hus gi ing 4 ules (Figu e 6b). The second ule base employs only one uzzy se o he 21 21 22 23 zS 26 26 zl zl 21 B 23 z5 26 Figu e 5: Rule base ob ained a e lea ning and clus e ing. inpu because wo o he ones a e gene a ed by using linguis- ic hedges, and h ee single on alues o he ou pu (Figu e 6c). The de uzzi ica ion me hod pe o med by bo h ule bases is he Fuzzy Mean me hod. Since ini ially all he ou - pu alues a e equal, he inpu -ou pu ela ion p o ided by his sys em is la . The in luence o he pa ame e s on he global beha io o a hie a chical sys em is complex. In e ms o lea ning, his means he exis ence o a lo o local minima which make no use ul he applica ion o g adien based lea ning algo i hms. Con a y o he p e ious example, he s a is ical algo i hms p o ide now be e esul s. In pa icula , we ha e used he Blind Sea ch algo i hm ollowed by Ma qua d -Le en- be g's algo i hm. In he la e algo i hm, x sl does no com- pu e he de i a i es (i is no possible in hie a chical sys ems) bu es ima es hem om small pa ame e modi i- ca ions. X Y (b) (4 Figu e 6. Tuning a hie a chical sys em 230 The IEEE In e na ional Con e ence on Fuzzy Sys ems A e lea ning, he global sys em beha io app oxima es he a ge beha io wi h an RMSE o 0.41%. Wha is e y in e es ing is ha he ule bases ha e lea n he in insic composi ion o he a ge unc ion. The i s ule base iden- i ies a sub ac ing ela ion be ween y and x (wi h a ce ain scaling ac o ), while he second ule base iden i ies an s ep- wise ela ion. Be ween 25 and 36 ules a e equi ed by a g id-based uzzy sys em (using he Fuzzy Mean de uzzi i- ca ion me hod) o pe o m as well as a hie a chical sys em wi h only 7 ules, hus showing he impo ance o uning hi- e a chical desc ip ions o a CAD ool. 8. Fuzzy sys ems wi h disc e e ou pu A uzzy classi ie can be seen as a uzzy sys em in which he membe ship unc ions o he ou pu a iable ep esen he di e en ca ego ies o which he ou pu may belong. The ou pu p o ided by a uzzy classi ie is gene ally he ou pu membe ship unc ion wi h he highes ac i a ion deg ee. Since he ou pu alues o hese sys ems a e ca ego ies, he lea ning p ocess aces an addi ional obs acle. I should modi y he pa ame e s o he membe ship unc ions associ- a ed o he inpu a iables o imp o e he success a e o he classi ica ion. Con a y o he case o uzzy in e pola o s, uzzy classi ie s can pe o m be e when educing hei ule bases since classi ica ion bounda ies no pa allel o he g id pa i ion can be ob ained. As an example o uning a uzzy classi ie , le us consid- e he p oblem illus a ed in Figu e 7. I shows a se o 80 da a g ouped in o 4 di e en ca ego ies wi h 20 da a each one (Cl, C2, C3, and C4). The uzzy classi ie o be uned by ~ sl con ains 9 ules ini ially, wi h 3 membe ship unc- ions co e ing each inpu a iable, as shown a he op and le pa s o Figu e 7. Figu e 7: Example o a classi ica ion p oblem. Applying he p uning p ocess o x sl, he 9 ules a e e- duced o 6, and he membe ship unc ions a e lea ned as shown a he bo om and igh pa s o Figu e 7. The classi- ica ion bounda ies implemen ed by he 6 ules each a clas- si ica ion a e o 100%. as shown in Figu e 7. 9. Conclusions The ool x sl p esen ed he ein ep esen s an impo an e - o owa ds he.au oma iza ion o he lea ning p ocess in he design o uzzy sys ems. The wide se o algo i hms in- cluded ( om g adien -based o s a is ical) allows sol ing many applica ion p oblems. The inco po a ed me hods o clus e ing and p uning pe mi s he simpli ica ion o he uzzy sys em conside ed. I s capabili y o uning hie a chi- cal uzzy sys ems makes i also possible o simpli y he de- sc ip ion o a sys em because complex beha io s can be usually gene a ed by composing simple ule bases. I s abil- i y o wo k wi h sys ems ha employ linguis ic hedges al- lows adjus ing he sys em as well as main aining i s linguis ic meaning, which is e y in e es ing when ex ac - ing knowledge om da a. Since he ool is in eg a ed in o he en i onmen X uzzy 3.0, i is possible no only o une a sys em bu also using o he ools o g aphically de ine i , o ep esen i s beha io by 2-D o 3-D plo s, and o simula e, moni o o syn hesize so wa e desc ip ions o i . As pa o X uzzy 3.0, x sl is dis ibu ed eely unde he GNU Gene al Public License om he X uzzy o icial web page (h p:// www.imse.cnm.es/X uzzy/). Re e ences [I] S. Haykin, “Neu al Ne wo ks. A Comp ehensi e Founda- ion”, IEEE P ess Macmillan, 1994. [2] Mglle , M.F., “A Scaled Conjuga e G adien Algo i hm o Fas Supe ised Leaming”, Neu al Ne wo ks, Vol. 6, pp. 525- 533.1993. [3] Scales, L.E.. “In oduc ion o Non-Linea Op imiza ion”, Sp inge -Ve lag New Yo k Inc., 1985. [4] P ess, W.H., Teukolsky, S.A., Ve e ling, W.T., Flmne y, B.P., Nume ical Recipes in C, Camb idge Uni e si y P ess, 1997. [SI Bezdek, J.C., Pal, N.R., “Clus e alida ion wi h gene alized Dum’s indices”, P oc. 2nd NZ In . Two-S eam Con . on ANNES, pp. 190-193, Dunedin, 1995. [6] F. I. Mo eno-Velo, S. Sinchez-Solano, A. Baniga, 1. Ba u one, D. R. Lopez, “XFL3: A New Fuzzy Sys em Speci- ica ion Language”, Ma hwa e & So Compu ing, pp. 239- 253, Decembe 2M)I. [7] F. J. Mo eno-Velo, I. Ba u one, S. Sinchez-Solmo, A. Ba i- ga, “Rapid Design o Fuzzy Sys ems wi h XFUZZY, sub- mi ed o FUZZ-lEEE2003. 231 The IEEE In e na ional Con e ence on Fuzzy Sys ems