Steel bar counting from images with machine learning

elec onics A icle S eel Ba Coun ing om Images wi h Machine Lea ning Ana Ca en He nández-Ruiz * , Ja ie Alejand o Ma ínez-Nie o and Julio Da id Buldain-Pé ez   Ci a ion: He nández-Ruiz, A.C.; Ma ínez-Nie o, J.A.; Buldain-Pé ez, J.D. S eel Ba Coun ing om Images wi h Machine Lea ning. Elec onics 2021,10, 402. h ps://doi.o g/ 10.3390/elec onics10040402 Academic Edi o : Gwanggil Jeon Recei ed: 18 Decembe 2020 Accep ed: 3 Feb ua y 2021 Published: 7 Feb ua y 2021 Publishe ’s No e: MDPI s ays neu- al wi h ega d o ju isdic ional clai- ms in published maps and ins i u io- nal a ilia ions. Copy igh : © 2021 by he au ho s. Li- censee MDPI, Basel, Swi ze land. This a icle is an open access a icle dis ibu ed unde he e ms and con- di ions o he C ea i e Commons A - ibu ion (CC BY) license (h ps:// c ea i ecommons.o g/licenses/by/ 4.0/). Depa men o Elec onic Enginee ing and Communica ions, Uni e si y o Za agoza, 50018 Za agoza, Spain; alma inez@uniza .es (J.A.M.-N.); buldain@uniza .es (J.D.B.-P.) *Co espondence: anaaca en@uniza .es; Tel.: +34-602-803-025 Abs ac : Coun ing has become a undamen al ask o da a p ocessing in a eas such as mic o- biology, medicine, ag icul u e and as ophysics. The p oposed SA-CNN-DC (Scale Adap i e— Con olu ional Neu al Ne wo k—Dis ance Clus e ing) me hodology in his pape is designed o au oma ed coun ing o s eel ba s om images. I s design consis s o wo Machine Lea ning ech- niques: Neu al Ne wo ks and Clus e ing. The sys em has been ained o coun ound and squa ed s eel ba s, ob aining an a e age de ec ion accu acy o 98.81% and 98.57%, espec i ely. In he s eel indus y, coun ing s eel ba s is a ime consuming ask which highly elies on human labou and is p one o e o s. Reduc ion o coun ing ime and esou ces, sa e y and p oduc i i y o employees and high con idence o he in en o y a e some o he ad an ages o he p oposed me hodology in a s eel wa ehouse. Keywo ds: con olu ional neu al ne wo ks; s eel ba s; coun ing; clus e ing; machine lea ning 1. In oduc ion Coun ing is a ime-consuming ask and a key ac o in keeping ack o he in en o y o any ma e ial. When alking abou objec s wi h di e en shapes and sizes, he ask becomes mo e challenging. In he s eel indus y, he s eel ba is one o he mos widely used p oduc in he wo ld o building cons uc ion and o ge. Du ing he manu ac u ing p ocess, he ba s a e usually coun ed by using images, hus allowing o dis ance, ligh ing and angle con ol. Howe e , once hey lea e he ac o y, hese hea y and la ge ma e ials mus be s acked and s o ed in wa ehouses o e ails, whe e a hos ile en i onmen p e ails in o de o ack a smoo h and eliable in en o y. T adi ional s eel ba coun ing is based on human calcula ion; howe e , due o he shi ing condi ions and low manoeu abili y, he manual coun ing is qui e slow and labou -in ensi e wi h low accu acy a e. The e o e, an au oma ic sys em capable o coun hese ma e ials ega dless o he physical condi ions is equi ed in o de o imp o e he e ec i eness and eliabili y o he s eel ba s coun ing p ocess. Basically, i is possible o dis inguish wo main app oaches o coun ing asks. Image p ocessing echniques implemen algo i hms based on ma hema ical unc ions o ans o m an image. Fil e s, h eshold segmen a ion, edge de ec ion and ma ching a e commonly used echniques [ 1 – 4 ]. Al hough hese echniques a e highly accu a e, hey a e bounded o speci ic condi ions such as cons an ligh ning and backg ound, o special came a e- qui emen s. Mo eo e , hey a e limi ed o ound s eel ba s wi h ixed shape and size, assuming hei shape is quasici cula , hus lacking obus ness [ 5 ]. No e ha some o hese me hods a e bounded o he p oduc ion line in s eel ab ics whe e physical sepa a ion o he ma e ials is iable [6,7]. Simila ly, o he s eel ba coun ing algo i hms based on image p ocessing a e mainly based on a ea and empla e algo i hms. Bo h o hese me hods a e easible, bu he e a e some disad an ages. The esul s o he i s me hod canno di ec ly loca e he s eel ba in he coun ing esul , so i makes g ea incon enience in e o analysis o he algo i hm [ 8 , 9 ]. Elec onics 2021,10, 402. h ps://doi.o g/10.3390/elec onics10040402 h ps://www.mdpi.com/jou nal/elec onics Elec onics 2021,10, 402 2 o 19 Templa e ma ching me hod hea ily depends on he shape o he empla e and a ge objec , so he adap i e abili y is limi ed [10,11]. In he case o machine lea ning (ML) echniques, con olu ional neu al ne wo ks (CNN) ha e demons a ed o be highly accu a e and as enough o image p ocessing. Well-known CNN a chi ec u es such as Fea u e Py amid Ne wo k, Visual Geome ic G oup and Incep ion-ResNe a e used as classi ie s and eg esso s in o de o ob ain he o al numbe o elemen s [ 12 – 16 ]. As o indus ial applica ions, neu al ne wo ks ha e p o en o achie e high accu acy and p o ide a as - ack solu ion o p oblems in hos ile en i onmen s [17,18]. A me hodology ha esembles how humans coun is p esen ed in [ 19 ]. A CNN im- plemen ed as a Bina y Classi ie is used o de ec each ba in he image and ma k i as a candida e cen e. Once e e y candida e cen e is de ec ed, a clus e ing echnique is applied in o de o ex ac he ac ual geome ic cen e o he ba s. The inal coun is he numbe o cen es de ec ed. This me hodology, e e ed o as CNN-DC, al hough i achie es 99.26% o accu acy in 3.58 s, i is es ic ed o a cons an backg ound among he images and a ixed pa ch size, hus implying ha he dis ance om he came a is cons an . On he o he hand, a deep lea ning usion model o de ec ing objec s is p oposed in [ 20 ]. Localisa ion and segmen a ion o s eel ba s is done by using a combined model. The model achie es a 98.17% in F1 sco e (ha monic mean alue o p ecision and ecall) in objec de ec ion in 0.03 s; howe e , he p oposed Incep ion-RFB-FPN a chi ec u e is qui e complex wi h many laye s and equi es high compu a ional esou ces o i s deploymen , hus making i una o dable o an embedded and po able sys em. Mos o he implemen a ions ocus on coun ing a single objec kind wi h speci ic cha - ac e is ics and cons an backg ound. Al hough some machine lea ning based wo ks ha e a high pe o mance, a po able sys em migh no be iable due o he equi ed amoun o compu a ional esou ces. By conside ing hese issues, a CNN-based coun ing me hodology app oach, namely Scale Adap i e Con olu ional Neu al Ne wo k Dis ance Clus e ing (SA-CNN-DC), is p esen ed in his pape in o de o coun s eel ba s ega dless o hei size and shape by adop ing a compac design wi h a minimum numbe o pa ame e s. The p oposed coun ing me hodology is desc ibed in Sec ion 2. Sec ion 3summa ises he image p ocessing echniques applied o gene a e he da ase o each ne wo k in he sys em. A desc ip ion o he implemen ed neu al ne wo k a chi ec u es and dis ance clus e ing algo i hm is shown in Sec ion 4, whe e a aining pe o mance o each one is also p e- sen ed. The mos ele an esul s and he SA-CNN-DC o e all pe o mance alida ion a e p esen ed in Sec ion 5. A compa ison wi h simila implemen a ions is also made in his sec ion. The desk op app implemen a ion is desc ibed in Sec ion 6. Finally, conclusions a e d awn in Sec ion 7. 2. P oposed SA-CNN-DC Coun ing App oach A a ie y o s eel ba shapes can be ound in he s eel indus y, and each ba kind usually is manu ac u ed wi h di e en dimensions. The di e si y o sizes and shapes is a challenge ha he p oposed me hodology add esses by adding an ex a inpu called densi y. This pa ame e is a undamen al ac o in o de o ob ain an e ec i e ba classi ica ion as well as i s accu a e localisa ion. The p oposed me hodology a emp s o imp o e he CNN-DC amewo k p esen ed in [ 19 ] which equi es speci ic condi ions o he images conside ed, as well as a ixed size and shape ba . In addi ion, he p oposed me hod is designed o be po able, epli- cable and obus o na u al a iabili y o he condi ions in he wa ehouse, such as noise, ligh and scale. Basically, he p oposed SA-CNN-DC me hodology au oma ically classi ies he s eel ba ype, loca es each ba cen e and p o ides an ou pu wi h he o al amoun o ba s. Bo h he ma e ial image and he densi y a e he inpu s equi ed o he sys em. I is wo h men ioning ha based on an analysis o he s o ing ma e ial physical condi ions wi hin he wa ehouse, h ee ypes o densi y we e de ined (low, medium and high) in o de o Elec onics 2021,10, 402 3 o 19 ensu e obus ness o he a iabili y o he s eel ba dimensions. Figu e 1shows an example o each o he possible s eel ba densi ies. (a) (b) (c) Figu e 1. Possible s eel ba densi ies: (a) Low, (b) medium and (c) high. The SA-CNN-DC main co e consis s o h ee neu al ne wo ks and a clus e ing ech- nique. Each ne wo k sol es an speci ic ask: ba classi ica ion, image esizing and cen e localisa ion. The modula design no only allows po abili y bu also i is eplicable, hus p o iding he capabili y o add new ma e ials by using he same me hodology. Bea - ing his in mind, ound and squa ed ba s we e conside ed o coun ing pu poses; howe e , angled and ec angula ba s we e also used o aining. A g aphical ep esen a ion o he p oposed me hodology is shown in Figu e 2. Thei main s ages a e desc ibed as ollows: Image Densi y O iginal Image Inpu s Resized Image Fi e C ops Fi e Resized C ops Class Resizing Fac o Candida e Cen e s Dis ance Clus e ing Pa ches 64 ×64 Final Coun Cen e s Visualiza ion Slidding Window 22 Isola ion Fil e Scale Adap ed Image MEDIUMLOW 44 66 55 11 33 HIGH Bina y Classi ie Linea Reg esso Classi ie Bina y Classi ie Linea Reg esso Classi ie Bina y Classi ie Linea Reg esso Classi ie Fla en o Op imal C op Fla en o Op imal C op 0 / 1 CNN CNN 64 ×64 Figu e 2. SA-CNN-DC me hodology block diag am. 1. P ep ocessing: The inpu image is esized o a size limi be ween 3500 and 1700 pixels. Acco ding o i s densi y ype, i e c ops o di e en sizes a e ex ac ed om he cen e Elec onics 2021,10, 402 4 o 19 o he image and esized o he CNN inpu size. Fo high densi y, he c ops ex ac ed a e smalle and ice- e sa o he low densi y. 2. Classi ica ion: These i e c ops a e eed- o wa d in o he classi ie o ob ain he so max ou pu . The class is chosen acco ding o he i e p edic ions ob ained h ough o ing, hus he class wi h he highes numbe o p edic ions is selec ed. F om he co ec p edic ions he one wi h he highes p obabili y is selec ed. A highe p obabili y means a mo e eliable p edic ion and i implies he ne wo k is able o de ec ea u es wi h high ce ain y. The la en ec o o he esponse wi h he highes p obabili y, which is gene a ed a e he con olu ional laye s and con ains all he in o ma ion o he image condensed, is s o ed as he op imal la en o he nex s ep. 3. Linea Reg esso : This ne wo k ou pu s a esizing ac o , R , which is used o esize he image. This ac o de e mines how much an image mus be esized so ha a single ba comple ely i s in a pa ch o a ixed size, as seen in Figu e 3. Then, he scale o he esul ing image is adap ed o he nex s age. The con olu ional laye s o he classi ie a e used o ain a simple mul ilaye pe cep on wi h a linea ou pu ha ac s as eg esso o he esizing ac o . This p ocess, called ans e lea ning , helps o d as ically educe he aining da a and ime o he eg esso ne wo k. 4. Bina y Classi ica ion: A ixed-size sliding window mo es h ough he scale adap ed image wi h a small s ide. A bina y-ou pu ne wo k classi ies whene e he esul ing pa ch con ains a ba o no . I a posi i e de ec ion is made, he cen e coo dina e o he pa ch is s o ed as a candida e cen e. 5. Dis ance Clus e ing: Candida e cen es a e il e ed acco ding o hei ho izon al and e ical p oximi y wi h o he candida e cen es. I a candida e cen e is comple ely isola ed, i is dele ed. The dis ance clus e ing algo i hm measu es he Euclidean dis ance be ween candida e cen es and g oups hem wi hin a h eshold dis ance. Fo each clus e , a cen e coo dina e is s o ed, ideally his is he geome ic cen e o he ma e ial [19]. 6. Ou pu : The inal coun is he o al numbe o cen e coo dina es. Fo isualisa ion pu poses, he cen es a e ma ked wi h a colou do in he image. (a) (b) (c) Figu e 3. Resized image: (a,b) show a w ong esizing, while (c) a co ec ly esized image. 3. Da ase Acquisi ion Since each neu al ne wo k has an speci ic ask wi hin he p oposed sys em, h ee di - e en da ase s we e buil o each ne wo k: Classi ie -Da ase , Reg esso -Da ase and Bina y-Da ase . I is wo h men ioning ha a single inpu size was ixed o he ne wo ks, hus ob aining a modula design. This inpu size de e mines how much GPU memo y will be equi ed o he ke nel weigh s in he ne wo k laye s, as well as he ba ch size and he aining ime. When an image wi h la ge dimensions and high esolu ion is esized o a conside ably smalle size, i ends up dis o ed and he s eel ba s lose hei shape. In consequence, he CNN is unable o lea n cha ac e is ics o he ma e ials, ins ead i lea ns o ex ac in o ma ion om he noise in he image. By conside ing a ade-o be ween he equi ed GPU memo y and image dis o ion, he inpu size was se o 64 × 64 pixels. Mo e- Elec onics 2021,10, 402 5 o 19 o e , since he RGB channels do no con ain ele an in o ma ion abou he shape and hey also equi e a la ge memo y consump ion, he images we e con e ed o g ayscale. 3.1. Classi ie -Da ase The classi ie was buil o dis inguish be ween s eel ba ypes. Fo his ne wo k i is impo an ha he samples con ain he ele an shape cha ac e is ics o he ba s. No e ha an angled, ec angula , ound and squa ed s eel ba s wi h di e en size and colou , as shown in Figu e 4, we e conside ed as possible classes. I is impo an o emphasise ha only he ound and squa ed ba s we e used o coun ing, while he angled and ec angula ca ego ies we e in oduced as ejec ion classes, bu hey will be conside ed o u u e esea ch. (a) (b) (c) (d) Figu e 4. S eel ba classes o he classi ie : (a) angled, (b) ec angula , (c) ound and (d) squa ed. Fi s , s eel ba pho og aphs wi h di e en dimensions we e manually collec ed in an o dina y wa ehouse. To ensu e an app op ia e ep esen a ion o he possible condi ions in he place, i was equi ed o collec se e al images wi h a ia ions in he ligh , ame, angle and posi ion. In his way, a ound 400 pho og aphs we e aken om each single ma e ial pile, hus ob aining a small se o 12,793 images. F om hese collec ed images and by conside ing he a o emen ioned densi y pa ame e , a iable size c ops (wi h a andom inc ease o dec ease) we e ex ac ed om di e en coo dina es wi hin he images. Once ex ac ed, hey we e esized o 64 × 64 pixels and con e ed o g ayscale. This echnique helps o inc ease he amoun o da a and p e en s dis o ion due o esizing. Figu e 5shows an example o he mos app op ia e esized c op (Figu e 5d) ob ained om he o iginal image (Figu e 5a). The c op size was selec ed acco ding o bo h he densi y and he image dimensions. Table 1shows he da ase c ea ed o he CNN. The ca ego ies a e qui e balanced and he amoun o da a is enough o a oid o e i ing when using a i ed CNN size. (a) (b) (c) (d) Figu e 5. Squa ed s eel ba s c ops wi h di e en dimensions: ( a ) O iginal image wi h la ge dimensions and high esolu ion image, (b) esized small c op, (c) esized big c op and (d) esized co ec c op. Elec onics 2021,10, 402 6 o 19 Table 1. Classi ie -Da ase sample dis ibu ion. S eel Ba Numbe o Samples Pe cen (%) Angled 15,204 25.14 Rec angula 14,995 24.80 Round 15,329 25.35 Squa ed 14,945 24.71 To al 60,473 100 3.2. Reg esso -Da ase As men ioned be o e, only ound and squa ed ba s we e conside ed o coun ing. Fo he Linea Reg esso , new pho og aphs wi h a iable dimensions we e aken ho izon- ally in on o he ma e ial piles whe e he s eel ba s we e uni o mly pain ed. The esul ing 263 images we e manually labelled wi h he ba size in pixels and hei co esponding densi y. The ba size is de e mined by he heigh and diame e o wid h o he ba ’s sec ion and i is a single numbe . In his way, he esizing ac o could be compu ed wi h he equa ion shown in Equa ion (1). Mo eo e , o a ions and shi s we e used o conside ably inc ease he amoun o da a. The esul ing sample dis ibu ion is summa ised in Table 2 and some samples a e p esen ed in Figu e 6. R=ba size 64 (1) Table 2. Reg esso -Da ase sample dis ibu ion o ound and squa ed s eel ba s. S eel Ba Numbe o Samples Round 5318 Squa ed 2114 (a) (b) Figu e 6. Samples o he Reg esso -Da ase o ound s eel ba s wi h ( a ) R= 1.18 and ( b ) R= 3.37. 3.3. Bina y-Da ase The Bina y Classi ie classi ies each ba in he image. Inpu pa ches gene a ed by a sliding window we e classi ied in o wo classes: ze os and ones. The i s one e e s o images con aining backg ound, incomple e elemen s o join s be ween hem, as shown in Figu e 7a, and he g oup labelled as ones con ains images wi h cen ed and comple e elemen s, as shown in Figu e 7b. An image o each size o he ound and squa ed s eel ba s we e conside ed o he da ase building (12 images in o al). Ba cen es we e manually ma ked by using an image edi o wi h a 10% b ush size o he ba size. No e ha his size de e mines how many pixels will be conside ed as cen e. Nex , a sliding window o he s eel ba sizes pass h ough he image wi h a s ide o 5% in o de o ex ac he equi ed pa ches. I he cen e o he pa ch Elec onics 2021,10, 402 7 o 19 ma ches a cen e o an elemen , which is ecognisable by i s ma ke colou , he pa ch is sa ed as one in g ayscale. On he con a y, i he pa ch cen e is no a s eel ba cen e, he pa ch is sa ed as ze o. (a) (b) Figu e 7. Samples om he Bina y Classi ie Da ase o ound s eel ba s: (a) Ze os and (b) Ones. The sample dis ibu ion o he Bina y-Da ase is p esen ed in Table 3. I is wo h men- ioning ha he unbalanced da a shown o he ound ba s is no an issue o his ne wo k because he da a can be chosen andomly in o de o balance bo h classes. Mo e impo - an ly, he addi ion o new ma e ials is easily done because he wo equi ed da ase s a e au oma ically p ocessed wi h he implemen ed algo i hms. Labelling he images is also a simple ask jus by changing he ilename and using a simple as e g aphics edi o , such as Mic oso Pain . This ensu es he eplicabili y o he me hodology and p o ides a as - ack addi ion o di e en s eel ba ypes. Table 3. Bina y-Da ase sample dis ibu ion o ound and squa ed s eel ba s. S eel Ba Ones Samples Ze os Samples Round 25,397 45,446 Squa ed 26,399 24,398 4. Neu al Ne wo ks Pe o mance The p oposed ne wo ks we e e icien ly designed wi h he leas numbe o neu ons and laye s in o de o c ea e a modula and po able a chi ec u e. The mo e con olu ion laye s a e added, he mo e abs ac in o ma ion is ex ac ed. Howe e , i he numbe o con olu ional laye s exceeds he one equi ed, ea u es wi h new in o ma ion a e no c ea ed because he e is no u he in o ma ion o lea n. The e o e, i is no ecommended o add a la ge numbe o con olu ional laye s. In 1987, Lippmann demons a ed ha a mul ilaye pe cep on wi h wo hidden laye s is enough o o m a bi a y decision egions [ 21 ]. These simple guidelines we e aken in o conside a ion o he a chi ec u es design. The classi ie p o ides he con olu ional laye s o he o he wo ne wo ks: he Linea Reg esso and he Bina y Classi ie , which we e ained o each single s eel ba ype wi h wo new da ase s. These wo ne wo ks a e smalle and equi e less da a and aining ime. This cha ac e is ic is an ad an age i new s eel ba ypes need o be added. Ins ead o aining he whole model, only aining o wo new mul ilaye pe cep ons is equi ed. This me hodology o eusing he ea u e-ex ac ion pa o a ained model wi h a pa icula goal o be used in o he model wi h di e en goal is commonly known as ans e lea ning [ 22 , 23 ]. Figu e 8shows how his p ocess is ca ied ou : he classi ie con olu ional laye s emains he same and only he small mul ilaye pe cep ons co esponding o bo h, he Reg esso and he Bina y Classi ie , a e ained. Elec onics 2021,10, 402 8 o 19 Inpu CNN Fea u e Ex ac ion Fla en o Op imal C op Classi ie Class 0 / 1 Bina y Classi ie Linea Reg esso Con olu ional Laye s Fully Connec ed Laye s Figu e 8. Block diag am o he implemen ed Neu al Ne wo ks a chi ec u es. Fi s , he classi ie is ained o classi y angled, ec angula , ound and squa ed s eel ba s. Then, ans e lea ning is applied o ain he Linea Reg esso and he Bina y Classi ie o ound and squa ed ba s. No e ha he CNN ea u e ex ac ion pa (con olu ional laye s) o he classi ie , is ozen and eused o ain he p e iously men ioned ne wo ks. Fo each ne wo k, he da a was di ided in o 70% o aining, 20% o alida ion and 10% o es in o de o c oss- alida e esul s. Se e al simula ions we e ca ied ou o de e mine he bes a chi ec u e and he hype pa ame e s o each ne wo k by using a NVIDIA GeFo ce GTX 1050 GPU wi h Ke as [24,25]. 4.1. Classi ie The classi ie was designed and ained o classi y he ou di e en s eel ba ypes. This ne wo k also wo ks as a ea u e ex ac o o encode , which means i educes he in o ma ion o a la ge inpu in o a compac ( la en) ec o . I was ained o 12 epochs wi h a ba ch size o 108 samples and i s a chi ec u e consis s o h ee con olu ional laye s and wo ully connec ed laye s, as shown in Figu e 9. The alida ion and es accu acies a e 99.28% and 99.35%, espec i ely, and he aining ime was ba ely 84 s. Inpu Con olu ional Laye 1 Con olu ional Laye 2 Con olu ional Laye 3 kk k Fla en Fully Connec ed Laye 1 Fully Connec ed Laye 2 Ou pu RELU RELU RELU RELU So max 64 ×64 5 ×514 ×14 6 ×6 2304 ×1 12 4 Classes Image 64 channels 64 con olu ional il e s 64 channels 64 channels 64 con olu ional il e s 64 con olu ional il e s 30 ×30 s = 2 s = 2 3 ×3 s = 2 3 ×3 Figu e 9. Classi ie a chi ec u e. Th ee con olu ional laye s and wo ully connec ed laye s, wi h 12 and 4 neu ons, a e used o classi y he ou s eel ba ypes. Elec onics 2021,10, 402 9 o 19 The con usion ma ix is used as pe o mance me ic o he classi ica ion ne wo k. The class p edic ions made by he ne wo k wi h he es da a a e compa ed wi h he ac ual class in Table 4. No e ha ound and squa ed ba s a e he classes mos likely o be con used. Table 4. Con usion ma ix o he classi ie . P edic ed Class Angled Squa ed Round Rec angula Ac ual Class Angled 1554 1 3 2 Squa ed 1 1554 8 3 Round 0 9 1544 0 Rec angula 9 4 0 1477 4.2. Linea Reg esso The selec ed a chi ec u e o he Linea Reg esso ne wo k consis s o one hidden laye wi h ou neu ons and an ou pu laye wi h a single neu on, as shown in Figu e 10. The inpu co esponds o he la en ec o gene a ed by he con olu ional encode o he classi ie and he linea ou pu is a nume ical alue which ep esen s he esizing ac o . The loss is calcula ed by using he Mean Squa ed E o (MAE), while he Mean Absolu e E o (MAE) is he me ic conside ed o his ne wo k. Fla en Hidden Laye 1 Ou pu Laye Ou pu ReLU Linea 2304 × 1 4 1 Inpu Figu e 10. Linea Reg esso modula a chi ec u e. A ba ch size o 256 ound and squa ed ba samples was conside ed, esul ing in a aining ime o 80 and 100 epochs, espec i ely. Once ained, p edic ions o he alida ion and es se s we e compu ed by using he Sciki -Lea n’s linea eg ession algo i hm [ 26 ], hus ob aining bo h, he linea eg ession ( W ) and de e mina ion ( R2 ) coe icien s. The bes possible sco e o he R2 coe icien is 1, which means ha he p edic ions a e equal o he eal alues and is calcula ed by: R2=1− ∑(y ue −yp ed)2 ∑(y ue −y ue)2(2) whe e y ue co esponds o he eal alues and yp ed a e he alues p edic ed by he ne wo k. Figu e 11 shows he eal and p edic ed alues o he es se o bo h ma e ial ypes, while Table 5summa ises hei esul ing me ics and coe icien s. Elec onics 2021,10, 402 16 o 19 Table 11. Compa ison wi h o he implemen a ions by conside ing di e en ound ba s da ase s. Me hod P ecision Recall F1 Accu acy (%) In e ence Time (s) Zhang e .al. [1] 0.9360 0.8864 0.9103 94.69 0.3023 Ying e .al. [28] 0.8417 0.9617 0.8975 85.68 0.2404 Liu e .al. [11] 0.6833 0.8123 0.7420 80.99 0.0313 Fan e .al. [19] 0.9976 0.9951 0.9963 99.26 3.5862 Zhu e .al. [20] 0.9753 0.9881 0.9817 98.72 0.0306 P oposed * 0.9926 0.9888 0.9906 98.81 25 * NVIDIA GeFo ce GTX 1050 GPU. Table 12. Compa ison be ween he p oposed me hodology and he CNN-DC p esen ed in [ 19 ] by conside ing bo h, he same compu a ional esou ces and same images. Me hod Pa ame e s Accu acy (%) In e ence Time (s) Fan [19] * 2,899,138 91.15 96 P oposed * 149,356 98.81 25 * NVIDIA GeFo ce GTX 1050 GPU. 6. Desk op App A use iendly desk op app was de eloped in o de o implemen he p oposed me hodology. Mino imp o emen s we e conside ed wi hin he app o c ea e a p ac ical use in e ace. The possibili y o de ine he elemen size as well as he a ea o in e es wi hin he aw image was added. The pa ch size used o he bina y ne wo k can be de ined jus by selec ing he elemen size wi h a bounding box in he image (Figu e 14a). While educing he a ea o in e es (Figu e 14b), he image is c opped so ha he possible noise in he o iginal image ames can be educed. On he o he hand, once he p ocessing is comple ed and he inal coun is ob ained, he app shows he possible coun ing e o s highligh ed in ed colou (Figu e 14c), so he use can easily e i y i hose e o s a e ela ed wi h clu e ing o clus e ing p oblems. (a) (b) (c) Figu e 14. Sc een-sho s o he app: (a) elemen selec ion, (b) a ea o in e es selec ion and (c) possible e o s highligh ed. Figu e 15 shows he low diag am o he desk op app implemen a ion. Fi s , he app asks o he use name and passwo d o login. The use can access he ile di ec o y and selec s he image. The image is displayed in he densi y selec ion window whe e he use mus choose he image densi y. A his s age, he use has he op ion o ei he selec a bounding box o an elemen o o he s ack o pile o ba s, o bo h. Once he p ocessing is comple ed, an ou pu window p o ides he image wi h ma ked cen es (whe e he possible e o s also appea ), he ma e ial ype, he inal coun and he p ocessing ime, as shown in Figu e 16. Elec onics 2021,10, 402 17 o 19 Login Image selec ion Densi y selec ion Image analysis Final coun and image wi h ma ked cen e s A ea and elemen c op Figu e 15. Flow diag am o he de eloped desk op app whe e he g ey s ep is op ional. Figu e 16. Desk op app ou pu window. The ex on op indica es he ba ype and he inal coun while he p ocessing ime is shown a he bo om. Red ma ke s a e displayed due o ma e ial clu e ing. 7. Conclusions Ba coun ing is a ime-consuming and edious ask o he wo ke s in he s eel wa e- house and i becomes mo e complica ed due o he di e en sizes o each ba ype. Con olu- ional neu al ne wo ks ha e become sui able o his ask because hey a e able o pe o m as ea u e ex ac o s, jus exploi ing hei inhe en capabili y o lea n abs ac concep s and de ec ing di e en shape objec s. The e o e, he p oposed wo k is a machine lea ning based me hodology capable o iden i y he ba ype and coun he numbe o elemen s om an image, which has been used in a eal s eel wa ehouse wi h high le el o use sa is ac ion. The addi ion o challenging s eel ba s shapes, such as angled and ec angula ba s, has been conside ed o u u e wo k. Collec ing mo e images, gene al imp o emen s and in e ence ime educ ion ha e been also aken in o accoun . Mo e impo an ly, he implemen a ion o he cu en ne wo ks in an embedded sys em is a high p io i y goal. The SA-CNN-DC me hodology shows a modula and po able design implemen ed by h ee mul ilaye ne wo ks, p ocessing he esponses o a common con olu ional encode and a clus e ing echnique. A compac design app oach was adop ed wi h a minimum numbe o pa ame e s in o de o educe he compu a ional esou ces bu wi hou comp o- mising i s pe o mance and accu acy. Tes simula ions we e ca ied ou in o de o alida e i s p ope pe o mance, so ha di e en wa ehouses images we e conside ed o e i y he gene alisa ion capabili y. Simu- la ion esul s showed a good pe o mance by conside ing ound and squa ed ba s coun ing, Elec onics 2021,10, 402 18 o 19 wi h an accu acy o 98.81% and 98.57% espec i ely. Compa ed wi h he implemen a ions ound in li e a u e, he p oposed me hodology is capable o achie ing compe i i e esul s wi h he minimum compu a ional esou ces. Mo eo e , i s modula design allows he addi ion o new ba ypes in a simple way, hus ul illing he eal wa ehouses expec a ions. The usage o he desk op app in he s eel wa ehouse has d as ically educed he coun ing ime o ound and squa ed ba s. Mo eo e , he e is a highe con idence in he in en o y due o he low e o a e and he possible e o s ma ks. Pain ing ma e ials o manual coun ing is no longe necessa y which esul s in a signi ican educ ion o esou ces and en i onmen al damage. Finally, p oduc i i y is imp o ed since employees can ocus on o he ac i i ies, while a oiding he exhaus i e ac i i y o coun ing a la ge amoun o elemen s in a hos ile en i onmen . Au ho Con ibu ions: Concep ualisa ion, A.C.H.-R. and J.D.B.-P.; Me hodology, A.C.H.-R. and J.D.B.-P.; So wa e, A.C.H.-R.; Valida ion, A.C.H.-R.; Fo mal analysis, A.C.H.-R. and J.D.B.-P.; In- es iga ion, A.C.H.-R. and J.D.B.-P.; Resou ces, A.C.H.-R. and J.D.B.-P.; Da a Cu a ion, A.C.H.-R.; W i ing—o iginal d a p epa a ion, A.C.H.-R., J.D.B.-P. and J.A.M.-N.; W i ing— e iew and edi ing, A.C.H.-R., J.D.B.-P. and J.A.M.-N.; Visualisa ion, A.C.H.-R., J.D.B.-P. and J.A.M.-N.; and Supe ision, A.C.H.-R. and J.D.B.-P.; Funding Acquisi ion, A.C.H.-R. and J.D.B.-P. All au ho s ha e ead and ag eed o he published e sion o he manusc ip . Funding: This esea ch wo k has been pa ially suppo ed by he Na ional Council o Science and Technology (CONACYT) by he M.Sc. G an wi h CVU No. 855821. Acknowledgmen s: The au ho s would like o hank DIACO and TEX s eel wa ehouses o hei suppo du ing his esea ch, especially o he employees who helped ake pho og aphs when needed. Con lic s o In e es : The au ho s decla e no con lic o in e es . The unde s had no ole in he design o he s udy; in he collec ion, analyses, o in e p e a ion o da a; in he w i ing o he manusc ip , o in he decision o publish he esul s. Re e ences 1. Zhang, D.; Xie, Z.; Wang, C. Ba sec ion image enhancemen and posi ioning me hod in on-line s eel ba coun ing and au oma ic sepa a ing sys em. In P oceedings o he 2008 Cong ess on Image and Signal P ocessing, Sanya, China, 27–30 May 2008; pp. 319–323. [C ossRe ] 2. Zhao, J.; Xia, X.; Wang, H.; Kong, S. Design o eal- ime s eel ba s ecogni ion sys em based on machine ision. In P oceedings o he 2016 8 h In e na ional Con e ence on In elligen Human-Machine Sys ems and Cybe ne ics (IHMSC), Hangzhou, China, 27–28 Augus 2016; pp. 505–509. [C ossRe ] 3. O su, N. A h eshold selec ion me hod om g ay-le el his og ams. IEEE T ans. Sys . Man Cybe n. 1979,9, 62–66. [C ossRe ] 4. Wu, Y.; Zhou, X.; Zhang, Y. S eel ba s coun ing and spli ing me hod based on machine ision. In P oceedings o he 2015 IEEE In e na ional Con e ence on Cybe Technology in Au oma ion, Con ol, and In elligen Sys ems (CYBER), Shenyang, China, 8–12 June 2015; pp. 420–425. [C ossRe ] 5. Wang, J.; Hao, C.; Xiaoqing, X. Pa e n ecogni ion o coun ing o bounded ba s eel. In P oceedings o he Fou h In e na ional Con e ence on he Applica ions o Digi al In o ma ion and Web Technologies (ICADIWT 2011), S e ens Poin , WI, USA, 4–6 Augus 2011; pp. 173–176. [C ossRe ] 6. Su, Z.; Fang, K.; Peng, Z.; Feng, Z. Reba au oma ically coun ing on he p oduc line. In P oceedings o he 2010 IEEE In e na ional Con e ence on P og ess in In o ma ics and Compu ing, Shanghai, China, 10–12 Decembe 2010; pp. 756–760. [C ossRe ] 7. Nie, Z.; Hung, M.-H.; Huang, J. A no el algo i hm o eba coun ing on con eyo bel based on machine ision. J. In . Hiding Mul imed. Sign. P ocess 2016,7, 425–437. 8. Youlian, Z. Resea ch o image ecogni ion a i hme ic in au oma ic coun ing sys em o s eel ba s. J. Wuhan Eng. Ins . 2008 ,20, 31–34. 9. Chi, L. A coun ing me hod o bundled s eel ba s based on image p ocessing. J. Shenyang Uni . Technol. 2016 ,38, 551–554. [C ossRe ] 10. Dahou, Z.; Sba aï, Z.M.; Cas el, A.; Ghoma i, F. A i icial neu al ne wo k model o s eel–conc e e bond p edic ion. Eng. S uc . 2009,31, 1724–1733. [C ossRe ] 11. Xiaohu, L.; Jineng, O. Resea ch on s eel ba de ec ion and coun ing me hod based on con ou s. In P oceedings o he 2018 In e na ional Con e ence on Elec onics Technology (ICET), Chengdu, China, 23–27 May 2018; pp. 294–297. [C ossRe ] 12. Cohen, J.P.; Bouche , G.; Glas onbu y, C.A.; Lo, H.Z.; Bengio, Y. Coun -cep ion: Coun ing by Fully Con olu ional Redundan Coun ing. In P oceedings o he 2017 IEEE In e na ional Con e ence on Compu e Vision Wo kshops (ICCVW), Venice, I aly, 22–29 Oc obe 2017; pp. 18–26. [C ossRe ] Elec onics 2021,10, 402 19 o 19 13. Rahnemoon a , M.; Sheppa d, C. Deep Coun : F ui Coun ing Based on Deep Simula ed Lea ning. Senso s 2017 ,17, 905. [C ossRe ] [PubMed] 14. Sam, D.B.; Su ya, S.; Babu, R.V. Swi ching Con olu ional Neu al Ne wo k o C owd Coun ing. In P oceedings o he 2017 IEEE Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4031–4039. [C ossRe ] 15. Zeng, L.; Xu, X.; Cai, B.; Qiu, S.; Zhang, T. Mul i-scale con olu ional neu al ne wo ks o c owd coun ing. In P oceedings o he 2017 IEEE In e na ional Con e ence on Image P ocessing (ICIP), Beijing, China, 17–20 Sep embe 2017; pp. 465–469. [C ossRe ] 16. Seguí, S.; Pujol, O.; Vi ià, J. Lea ning o coun wi h deep objec ea u es. In P oceedings o he 2015 IEEE Con e ence on Compu e Vision and Pa e n Recogni ion Wo kshops (CVPRW), Bos on, MA, USA, 7–12 June 2015; pp. 90–96. [C ossRe ] 17. Li, X.; Jia, X.; Wang, Y.; Yang, S.; Zhao, H.; Lee, J. Indus ial Remaining Use ul Li e P edic ion by Pa ial Obse a ion Using Deep Lea ning Wi h Supe ised A en ion. IEEE/ASME T ans. Mecha on. 2020,25, 2241–2251. [C ossRe ] 18. Li, X.; Jia, X.; Yang, Q.; Lee, J. Quali y analysis in me al addi i e manu ac u ing wi h deep lea ning. J. In ell. Manu . 2020 ,31, 2003–2017. [C ossRe ] 19. Fan, Z.; Lu, J.; Qiu, B.; Jiang, T.; An, K.; Joseph aj, A.N.; Wei, C. Au oma ed s eel ba coun ing and cen e localiza ion wi h con olu ional neu al ne wo ks. a Xi 2019, a Xi :1906.00891. 20. Zhu, Y.; Tang, C.; Liu, H.; Huang, P. End-Face Localiza ion and Segmen a ion o S eel Ba Based on Con olu ion Neu al Ne wo k. IEEE Access 2020,8, 74679–74690. [C ossRe ] 21. Lippmann, R. An in oduc ion o compu ing wi h neu al ne s. IEEE ASSP Mag. 1987,4, 4–22. [C ossRe ] 22. Weiss, K.; Khoshgo aa , T.M.; Wang, D. A su ey o ans e lea ning. J. Big Da a 2016,3, 9. [C ossRe ] 23. Alom, M.Z.; Taha, T.M.; Yakopcic, C.; Wes be g, S.; Sidike, P.; Nas in, M.S.; Hasan, M.; Van Essen, B.C.; Awwal, A.A.S.; Asa i, V.K. A S a e-o - he-A Su ey on Deep Lea ning Theo y and A chi ec u es. Elec onics 2019,8, 292. [C ossRe ] 24. Bell, N.; Ga land, M. E icien Spa se Ma ix-Vec o Mul iplica ion on CUDA; NVIDIA Technical Repo NVR-2008-004; NVIDIA Co po a ion: San a Cla a, CA, USA, Decembe 2008. 25. Ke as: Simple. Flexible. Powe ul. A ailable online: h ps://ke as.io (accessed on 12 Oc obe 2020). 26. Ped egosa, F.; Va oquaux, G.; G am o , A.; Michel, V.; Thi ion, B.; G isel, O.; Blondel, M.; P e enho e , P.; Weiss, R.; Dubou g, V.; e al. Sciki -lea n: Machine lea ning in Py hon. J. Mach. Lea n. Res. 2011,12, 2825–2830. 27. Sa ang Na khede. Unde s anding AUC-ROC Cu e. 2018. A ailable online: h ps:// owa dsda ascience.com/unde s anding- auc- oc-cu e-68b2303cc9c5 (accessed on 12 Oc obe 2020). 28. Ying, X.; Wei, X.; Pei-xin, Y.; Qing-da, H.; Chang-hai, C. Resea ch on an Au oma ic Coun ing Me hod o S eel Ba s’ Image. In P oceedings o he 2010 In e na ional Con e ence on Elec ical and Con ol Enginee ing, Wuhan, China, 25–27 June 2010; pp. 1644–1647. [C ossRe ]