PERSIMMON: a visual dataflow language for machine learning

Uni e sidad Complu ense Facul ad de In o má ica Ingenie ia In o má ica Tecnología Especí ica de Compu ación June 16, 2017 Pe simmon A isual da a low language o machine lea ning Ál a o Be mejo Ga cía Supe ised by Manuel F ei e Mo an Cosupe ised by Pablo Mo eno Ge Abs ac Pe simmon is a isual p og amming in e ace ha le e ages sciki -lea n o p o- ide a d ag and d op in e ace o de eloping Machine Lea ning and Da a Mining pipelines. I is based on he da a low p og amming p inciples, gi ing he use a unc ional isual language wi h a ype sa e y sys em ha checks connec ions a w i e ime, non-s ic e alua ion, ask pa alleliza ion, and execu ion isualiza ion. I has been e alua ed by pa icipan s on a h ee- ask o m, o e all ecei ing good e iews, being p aised by he use o colo s o indica e ypes, consis en design, easy o na iga e and shallow lea ning cu e. Keywo ds: Machine Lea ning , Da a Mining , Visual P og amming , Da a low P o- g amming , Func ional P og amming . 2 Con en s 1. In oducción 7 Desc ipción ..................................... 7 Mo i ación ..................................... 9 Obje i os ...................................... 9 Quenoeselp oyec o................................ 10 Es uc u adelamemo ia ............................. 10 2. In oduc ion 11 Desc ip ion ..................................... 11 Mo i a ion ..................................... 12 Objec i es...................................... 13 Wha hep ojec isno .............................. 14 P ojec S uc u e.................................. 14 3. Focus 15 4. Li e a u e Re iew 16 OnMachineLea ning ............................... 16 OnDa a lowP og amming............................. 17 OnVisualP og amming.............................. 18 S a eo hea ................................... 18 5. Wo kflows 21 Simple........................................ 21 Regula ....................................... 21 Complex....................................... 22 6. Miles ones 23 T ee ......................................... 23 Gan Cha ..................................... 24 De elopmen Me hodology............................. 25 Sou ceCode..................................... 25 7. Risk Analysis 27 S akeholde s..................................... 27 P e en ion&Mi iga ion.............................. 27 3 8. In e ace Design 29 Ske ches....................................... 29 Colou Pale e ................................... 31 Typog aphy..................................... 31 9. Implemen a ion 32 Fi s I e a ion.................................... 32 SecondI e a ion .................................. 33 Thi dI e a ion ................................... 33 ModelViewCon olle ............................... 35 MakingaConnec ion................................ 36 Visualizing heDa aFlow............................. 38 Bina yDis ibu ion................................. 39 10. Type Checking 41 G adualTyping................................... 41 W i eTime ..................................... 41 The wolanguages ................................. 41 Ac ualTypes .................................... 42 In e media e Rep esen a ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 11. E alua ion 45 Me hod ....................................... 45 P oposedTasks................................... 45 E alua ionResul s ................................. 46 12. Conclusiones 49 Re isióndeObje i os ............................... 49 Re ospec i a.................................... 50 Conclusión ..................................... 50 T abajoFu u o................................... 51 13. Pos mo em 52 Objec i esRe iew ................................. 52 Re ospec i e.................................... 53 Conclusion ..................................... 53 Fu u eWo k .................................... 54 Bibliog aphy 55 A. Package O ganiza ion 59 Backend....................................... 59 View......................................... 59 4 B. How was his documen made? 60 P ocess ....................................... 60 Diag ams ...................................... 60 Re e ences...................................... 60 C. Pe simmon E alua ion 61 P epa a ion..................................... 61 P e iousQues ions................................. 61 Tasks ........................................ 61 Addi ionalFeedback ................................ 62 5 Lis o Figu es 4.1. G aph Execu ion algo i hm ......................... 17 4.2. Azu e ML S udio web in e ace ....................... 19 4.3. Un eal Engine 4 Bluep in sys em ...................... 20 5.1. Jus alida ion o he model ......................... 21 5.2. P edic ion using he whole da ase ..................... 21 5.3. Adjus men o hype pa ame e s ....................... 22 6.1. Miles ones T ee ................................ 23 6.2. Gan Diag am o he p ojec de elopmen . ................ 24 8.1. Ske ch o he i s in e ace .......................... 29 8.2. Ske ch o he second in e ace ........................ 30 8.3. Types colo s .................................. 31 9.1. Implemen a ion o he i s in e ace .................... 32 9.2. Second i e a ion implemen a ion ....................... 34 9.3. Thi d i e a ion in e ace showing a wa ning ................ 35 9.4. Widge T ee .................................. 36 9.5. Connec ions be ween elemen s ........................ 37 9.6. Connec ion modi ica ion handling ...................... 38 10.1. Type hie a chy ................................ 43 10.2. IR de ini ion on Haskell ........................... 44 11.1. Pa icipan s amilia i y ............................ 46 11.2. Task sco e pe ask pe pa icipan ..................... 47 12.1. Pe simmon en el ex anje o ......................... 50 13.1. Chinese machine lea ning o um ....................... 53 A.1. Pe simmon package hie a chy ........................ 59 6 1. In oducción En es e capí ulo se p esen a Pe simmon, así como los obje i os y las mo i aciones del p oyec o. También se incluye una sección sob e emas elaciones con el p oyec o pe o que quedan ue a del ámbi o del mismo. Finalmen e se encuen a una b e e e isión de la es uc u e de la memo ia. Desc ipción El campo de Da a Science ha is o un inc emen o exponencial de me cado en los úl- imos años, con p edicciones a icinando que has a un millón de cien í icos de da os se án necesa ios pa a 2018 (Rajpu ohi , 2016). Los cien í icos de da os se encuen an en una si uación excepcional, pa a el Ha a d Business Re iew es “ he sexies job o he 21s cen u y” (Da enpo and Pa il, 2012). Y sin emba go, a pesa de odo es o al an p o esionales que puedan cub i es os pues os, la disciplina es inhe en emen e mul idisciplina ia (Taylo , 2016), incluyendo conocimien o de es adís icas, ma emá icas, p og amación y del dominio. Es o hace que el camino pa a con e i se en un expe o sea la go y complejo, lo cual desemboca en las llamadas “cazas de unico nios” (Ha is and Ei el-Po e , 2015) y (P ess, 2015). He amien as como sciki -lea n1, Weka o Tableau pe mi en un acceso simpli icado y de al o ni el a las he amien as necesa ias pa a hace Da a Science, sua izando la cu a de ap endizaje y aumen ando la o e a de p o esionales capaces de desa olla análisis de da os. Es as he amien as po o o lado equie en p og amación, se cen an en a eas de limpieza y p e-p ocesamien o de da os, o p o een una in e az muy limi ada. Pe simmon p e ende p opo ciona una in e az isual pa a sciki -lea n, dando la habili- dad de c ea complejos p ocesos de análisis sin esc ibi una sola línea de código, dando al usua io una exp esi idad compa able a la p og amación adicional a la ez que se le ayuda median e es ímulos isuales. Pa a pode consegui es o el p oyec o explo a las siguien es disciplinas, 1Sciki -lea n es una lib e ía de Py hon que ae una mul i ud de algo i mos de ap endizaje au omá ico a una API que pe mi e el uso y compa ación del os mismos en un al o ni el de abs acción. 7 • Da a low P og amming. Es e pa adigma ep esen a p og amas como g a os acicli- cos di igidos, iniciado en los 60 en el MIT y los labo a ios Bell (Kelly e al., 1961). Modela los p og amas como un lujo de da os que pasa po una se ie de in ucciones en ez de una se ie de ins ucciones que ope an en unos da os ex e nos, i.e. los da os luyen po las ins ucciones, no al e és (de ahí el nomb e de da a low). Es o p o- duce p og amas pa alelos po na u aleza, más ce canos al pa adigma uncional que al impe a i o y a la a qui ec u a Von Neumann (sección 15, Backus, 1978). • P og amación Visual. La elección po na u aleza pa a ep esen a un lenguaje de da a low es una in e az isual, pudiendo ep esen a el g a o de o ma cla a y p ecisa (Shu, 1988). Mejo a más a anzadas que se pueden implemen a g acias a la p esen ación isual incluyen comp obación de ipos en iempo de esc i u a, indicado de ejecución (señalando que unciones se es án ejecu ando en el momen o), e c. • Expe iencia del Usua io. El p oyec o se nu e de la expe iencia de los pa icipan es en los expe imen os con el p o o ipo. La in e az debe indica el camino pa a ealiza la acción deseada po el usua io, dando acilidades pa a educi la di icul ad de uso. • Ingenie ía del So wa e. Comunicación con múl iples lib e ías y amewo ks, de ini- ción de in e aces y o ganización del código median e ecnicas de p og amación o ien ada a obje os y modulos. • Ap endizaje Au omá ico. Aunque no se implemen an los algo i mos en sí, es nece- sa io ex enso conocimien o de la implemen ación, ya que hay que p opociona un pun o de acceso a los hype pa ame os y o os ipos de con igu ación que pe mi e sklea n (Va oquaux e al., 2015). • T ans o mación de Da os. Algunas p econdiciones sob e los da os han de se asum- idas o el usua io ha de se p o is o con las he amien as necesa ias pa a ealia las ans o maciones necesa ias. • Compilado es. El g a o isual que el usua io dibujo iene que se compilado a codigo uen e en Py hon (T anscompilación). La hipó esis del p oyec o es que la ep esen ación isual del p og ama y los concep os asociados puede ayuda con el ap endizaje y uso de écnicas de ap endizaje au omá ico, así como acele a el abajo de explo ación emp ana ípico del análisis de da os. Es a hipó esis con e ge con el espí i u de sklea n (Va oquaux e al., 2015, pp29) en el he- cho de que in en a simpli ica el uso y acceso a he amien as de ap endizaje au omá ico. Es a es a egia pa ece habe uncionado pa a sklea n, con i iéndose en uno de los p oyec os de ap endizaje au omá ico de código lib e más impo an es, con más de 16000 es ellas en Gi hub, siendo usado po compañías como Spo i y, Facebook o E e no e (sciki -lea n, 2016). 8 Mo i ación T as cu sa Ap endizaje Au omá ico el año pasado u e una beca en una emp esa de ading algo í mico como pa e del equipo de quan s2. Allí mi p incipal esponsabilidad e a eesc ibi pa e de las he amien as de MATLAB a Py hon, du an e ese p oceso obse é como algunos de los in eg an es del equipo expe i- men aban di icul ades con el cambio de lenguaje. Todos los in eg an es enían de disciplinas más “pu as” (Física, Ma emá ica, Es adís ica, Ingenie ía Ae oespacial, e c..). Los expe os de es os campos es án acos umb ados a abaja con lenguajes de dominio especí ico como MATLAB, R, Simulink o Julia, y el cambio a un lenguaje deu so gene al ae di icul ades como la p og amación o ien ada a objec os, complejas es uc u as de da os, op imización o ipos más “ ue es”. La si uación es aún mas di ícil pa a aquellos que comienzan el ap endizaje, ya que no solo ienen que lidia con la ba e a de la p og amación, sino que además ienen que supe a la di icul ad de los algo i mos en sí. Obje i os Es udio Viabilidad: El p oyec o iene que explo a el espacio de posibles soluciones isuales de ap endizaje au omá ico, e aluando dis in as es a egias en el on y backend de la aplicación. Diseño y Usabilidad: El sis ema ha de se diseñado aco de a los eque imien os, an o en é minos de hace sencillo el p og eso a a es de miles ones, como p oduciendo so wa e usable en cada elease. En odos los casos se debe balancea la compleji- dad con a la exp esi idad del sis ema, p o iniendo al usua io deu na he amien o po en e sin p oduci una in e ace compleja. E aluación: El sis ema se á e aluado po pa icipan es que pe enecen a la audiencia po encial del so wa e, un o mula io debe se p epa ando de allando las ac i idades que end án que ealiza , así como se án a ados sus da os. He amien a de ap endizaje: El so wa e debe ayuda con la ba e a de p og amación, a- cili ando el ap endizaje de Machine Lea ning, ayudando al es udian e a cen a se en las conexiones, in uiciones y bases ma emá icas de los algo i mos y no en los de alles de implemen ación y peculia idades del lenguaje. Acele a análisis explo a o io: P o eyendo una in e az isual ácil de usa con la capaci- dad de a as a y sol a el usua io puede p o a una plé o a de algo i mos, aju- s ando los hype pa áme os aco de a la e aluación sin esc ibi una sola línea de código. 2Analis a Cuan i a i o, en inglés Quan i a i e Analys , ab e iado Quan . 9 4. Li e a u e Re iew On his chap e he main sou ces used o he p ojec a e explained as well as some o he lea ning needed in o de o build he p ojec . On Machine Lea ning Al hough he p ojec aims o p o ide a e y high-le el ool o machine lea ning wi hou needing o ge oo deep in o he algo i hms, i is necessa y o unde s and he lib a y ha is used o pe o ming he ac ual machine lea ning ( om he e onwa ds e e ed as ml). While om he concep ion o he p ojec py hon was se as he main language, a com- pa ison be ween ml lib a ies was done in o de o e alua e sciki -lea n agains he com- pe i o s. The e compa ison o e di e en solu ions (Ryan, 2016), bu hey mos ly look a deep lea ning amewo ks. In ac , while deep lea ning is going h ough a golden age igh now (no doub helped by he push om companies such as Google o Facebook) i is a bleeding edge ield (Gschwind, 2017). Neu al ne wo ks wi h many laye s and complex connec ions be ween hem a e also e y di icul o isually ep esen compa ed o adi ional s a is ical me hods ha can be ep esen ed as unc ions mo e easily, and whole amewo ks a e dedica ed jus o ep esen hem such as Tenso low (Abadi e al., 2016). On he o he hand, adi ional machine lea ning lib a ies a e ei he embedded on pu pose-speci ic languages (such as R,Ma lab,Julia) o ha e less use s han o he s (To ch has only 7k Gi hub s a s). And inally, clus e -o ien ed compu ing amewo ks like Spa k o Hadoop a e usually in compiled languages like Ja a o C++ o pe o mance easons. Pe simmon main ool is sciki -lea n (Va oquaux e al., 2015), sciki -lea n (also known as sklea n) is based on Numpy (a n-dimensional a ay o Py hon (Wal e al., 2011)) and scipy (a scien i ic compu ing amewo k (Jones e al., 2014)). Pe simmon also uses pandas (McKinney and o he s, 2010) o inpu and ou pu handling. O he s pape s ela ed o he pi alls o machine lea ning ha p o ed use ul when ana- lyzing wo k lows we e (Hughes, 1968), (Khabaza, 2005). 16 On Da aflow P og amming A e e iewing da a low seminal pape Kelly e al. (1961), and Sousa (2012) i was clea he undamen al s ep o ha e a wo king sys em was w i ing a compila ion algo i hm om he isual ep esen a ion o py hon code. The e a e di e en ways o implemen da a low p og amming compile s, o now le ’s jus conside he language ep esen a ion as o med by blocks ha ha e pins. Pins on he le side o a block a e called inpu pins and each mus come om a single ou pu pin. Pins on he igh side a e called ou pu pins and one can be connec ed o mul iple inpu pins. This esul s in wha is e ec i ely a di ec ed acyclic g aph, in o de o compile and un he p og am (ac ually i is heo e ically possible o ha e mul iple pa allel p og ams on he same blackboa d) he g aph has o be explo ed, checking he dependencies o each block, execu ing hem i necessa y, execu ing he unc ion and adding he nex blocks o be execu ed un il he e is no block le o be execu ed. Requi e: G is a Di ec ed Acyclic G aph ha does no b eak ype sa e y on all he ela ionships. 1: unc ion execu e(G:G aph) 2: queue ←Queue() 3: seen ←Map() 4: queue.pu (G.ge inpu blocks()) ▷We can s a in a andom e ex 5: while ¬queue.emp y() do 6: queue,seen ←explo e(queue.ge (), queue, seen) 7: 8: unc ion explo e(cu en :V e ex, queue :Queue, seen :Map)→Queue, Map 9: o all in pin ∈cu en .ge in pins() do 10: co esponding ←in pin.o igin.uid 11: i ¬seen.has(co esponding) hen 12: dependency ←co esponding.block 13: i dependency ∈queue hen ▷Remo e i al eady in queue 14: queue. emo e(dependency) 15: queue,seen ←explo e(dependency, queue, seen) 16: in pin. alue ←seen.ge (co esponding) 17: cu en . unc ion() ▷ unc ion uses in pins and se s ou pins 18: o all ou pin ∈cu en .ge ou pins() do 19: seen.pu (ou pin, ou pin. alue) 20: queue.add(pin.des ina ions) 21: e u n queue,seen Figu e 4.1.: G aph Execu ion algo i hm The algo i hm looks each inpu pin on he block. I he co esponding alue has al eady been compu ed (i.e. is al eady on a hash able) i is assigned, else ha block is p ocessed i s and hen he execu ion o he cu en block esumes. Then he unc ion inside he 17 block is execu ed and a e ha he alue o each ou pu pin is sa ed on he hash able. The e is an al e na i e way o doing he compila ion wi hou needing o check depen- dencies when compiling/execu ing. Th ough a opological so on he g aph he g aph can be p ocessed “ o wa d only”, no ecu si e s ep is needed, bo h app oaches a e O(N), mo e closely hey a e O(n∗m)whe e n is he numbe o blocks and m he numbe o pins. On Visual P og amming Fo designing he in e ace many no es we e aken om Shu (1988), bu mos impo an ly om he bluep in sys em (Shah, 2014) and Azu e ML s udio web in e ace (Ba ga e al., 2015), all hese in luences a e discussed on he s a e o he a sec ion, and he in e ace design i sel along wi h he ske ches can be seen on he In e ace Design chap e . S a e o he a Be o e implemen ing he sys em i was necessa y o look a exis ing solu ions on he ield o isual p og amming and isual Machine Lea ning o inspi a ion and a oiding common pi alls. Mic oso Azu e ML S udio (Ba ga e al., 2015) is one o he mos di ec inspi a ions o his p ojec ; i is a Mic oso cloud-based pla o m o c ea ing p edic i e analy ic solu ions on da a using a d ag and d op in e ace. The e is plen y o like, lo s o di e en p e-p ocessing s eps, mul i ude o es ima o s, uns on he cloud, and a web in e ace ha uns on any pla o m. Howe e , some o hese ea u es a e also sho comings, he web in e ace eels basic, especially on he classi ica o s pa ame e s iew, lack o na i e suppo means ha d agging and d opping do no eel as smoo h as hey should. Cloud suppo is e y good, as i in eg a es wi h he es o Mic oso ’s Azu e pla o m, bu o sensi i e da a such as inancial o medical eco ds a sel hos ed e sion is a mus . The a ie y o algo i hms is in e es ing, bu he limi ed abili y o ex end hem is a sho coming, azu e is w i en on compiled languages (E icsson e al., 2017), unlike mos ml ha is w i en on ei he Ro Py hon (Puge , 2017), and unning cus om code is e y limi ed, as sc ip s a e ea ed as black boxes. This in u ns se e ely handicaps he ex ensibili y o he gi en p imi i es in any meaning ul way. Weka (Hall e al., 2009) is a popula machine lea ning sui e, w i en in Ja a and de el- oped a he Uni e si y o Waika o. I p o ides bo h a command line in e ace and a g aphical in e ace. 18 Figu e 4.2.: Azu e ML S udio web in e ace Howe e i is s a ing o show i s age, he in e ace eels da ed and he composi ion o algo i hms h ough g aphical means is e y es ic ed. Because i is w i en on Ja a i also means ha i need he JVM1, which is a bi o a disad an age, especially in p oduc ion se e s whe e dependencies b ing a long and a duous p ocess o e iew and app o al (Zmud, 1980). Epic’s Un eal Engine 4 (Shah, 2014) in oduced Bluep in s as an al e na i e o C++ p o- g amming. I ep esen s all he p og amming s uc u es as blocks ha can be connec ed, o example an “and” is a block ha akes o inpu s and e u ns one ou pu . Because i p o ides wha is essen ially a gene al-pu pose p og amming language i has cons uc s o ep esen s a e, because o his i also needs a explici low mechanism, meaning ha blocks do no only need o be connec ed h ough da a bu also by execu ion o de , his is necessa y because he o de in which side-e ec s a e pe o med is impo an , and many p ocedu es do no e u n meaning ul alues. Wi h his knowledge, i is clea ha in o de o no ha e an explici low line he isual language ep esen ed mus be pu e, con- s aining side e ec s o ei he he s a o he end o a pipeline (McB ide and Pa e son, 2008). 1The Ja a Vi ual Machine is he unde lying pla o m whe e he Ja a language is usually un on op o . I p o ides a single pla o m in which is abs ac ed o he unde lying ha dwa e a chi ec u e a he cos o paying some pe o mance o e head. 19 Figu e 4.3.: Un eal Engine 4 Bluep in sys em Bluep in s p o ides an in ui i e in e ace, when one cable is d agged om a block and a p omp appea s wi h only he blocks ha make sense o be connec ed o he p e ious block. Ano he example is how di e en ypes a e ep esen ed by di e en colo s in bo h pins and cables, making i easie o p edic whe he a connec ion makes sense o no wi hou e en ying o c ea e i . These small de ails imp o e he use expe ience, making i as e and easie o use. 20 5. Wo kflows A wo k low in he con ex o his p ojec e e s o he ypical ML explo a o y wo k analysis, i.e. he pipelines ha a e used ea ly on he p ojec when i is s ill no known wha s a egies will wo k bes o he gi en da a. This concep is gene aliza ion o sklea n pipelines. Simple The simples wo k lows a e hose ha in ol e no p e-p ocessing, no adjus men , and jus ei he es how good he model wo ks ( alida e) o p edic using bo h he ain ile and ano he ile wi hou class ea u e. .cs Es ima o Valida ion Figu e 5.1.: Jus alida ion o he model .cs Es ima o Valida ion P edic ion .cs .cs Figu e 5.2.: P edic ion using he whole da ase Regula A mo e usual wo k low in ol es also unning he hype -pa ame e s o he selec ed hype - pa ame e s, his in ol es making a g id o he possible hype -pa ame e s and ying all 21 o hem, esul ing in inding he bes possible alue. .cs Es ima o Adjus men Valida ion P edic ion .cs .cs Figu e 5.3.: Adjus men o hype pa ame e s Complex Mo e complex wo k lows in ol e p e-p ocessing, o au oma ing mul iple classi ie s hype - pa ame e uning a he same ime h ough he use o pipelines, his a ies widely on a case by case basis, and can o en in ol e da a cleaning, ea u e enginee ing (such as combining wo ea u es in o one) o dimensionali y educ ion (like PCA). Howe e , he e a e e en u he examples o pipelines whe e he whole p ocess is au- oma ed o he maximum, e en going as a as iden i ying he sui able da a ea u es, selec ing classi ie s o bagging, boos ing and o he me a-classi ie s, e c… (Thaku , 2016). I should be no ed ha his kind o wo k low is ou side he scope o he p ojec , as his is a away om explo a o y wo k, and ei he equi es manual da a cleaning anyway o an ex emely complex pipeline. In ac , his kind o use case would esul unwieldy and messy on a isual o m, i- sual p og amming ge s oo bloa ed when ep esen ing p og ams ha a e oo complex. On Dalke (2003) some wo ka ounds a e p oposed, such as modules, di e en shapes o di e en kinds o blocks, e c… Bu e en wi h hese echniques isual p og amming languages ne e uly ul illed hei p omises and gained mains eam adop ion (Simões, 2015). Howe e , isual languages managed o become ele an in small niches such as PLCs design (Minas and F ey, 2002) o music composi ion (Twells, 2016). P esumably because he complexi y can be p edic ed and accoun ed o when he numbe o ac ions is limi ed, his is he basis o he p ojec p og amming in e ace being limi ed on he numbe o blocks, as no o allow he g aphs o become insc u able, and as men ioned on he in oduc ion his also allows making assump ions abou he in e ace which educe he complexi y such as no needing an explici low line, mo e on he explici low line can be ead in he implemen a ion chap e . 22 6. Miles ones In o de o gua an ee he deli e y o he so wa e an inc emen al app oach has been chosen, his implies b eaking down he objec i es in o smalle miles ones ha can be accomplished mo e easily, so in case he las miles one is no eached he e is s ill a subs an ial p oduc o submi . T ee Capped Pa i y Compila ion Ou o scope Web Syn hesis Figu e 6.1.: Miles ones T ee Capped is mo e han a minimum iable p oduc , a ex ensi e p oo -o -concep , wi h a ew limi ed algo i hms and he abili y o inpu ing .cs iles, wi h a es ic ed in e ace in which algo i hms a e no d agged and d opped bu me ely selec ed h ough bu ons. Pa i y means a mo e o less comple e pa i y in e ms o ea u es and isual in e ac ion. I is no e y impo an o ha e he same numbe o unde lying algo i hms because ha ’s no he ocus o he p ojec , and c ea ing new blocks ha ing he unde lying algo i hm is easy. And he inal miles one is Compila ion, he abili y o ge he py hon sou ce code om he isual ep esen a ion, also imp o ing he in e ace o ha e a be e low, mo e akin o Un eal Engine, as discussed on he li e a u e e iew chap e , s a e o he a sec ion. This miles one would b ing Pe simmon u ili y beyond he ealm o lea ning ool, as i would be a con enien ool o he explo a o y wo k o any ML solu ion (business case, 23 a Kaggle1compe i ion, e c…). Ou o scope, bu possible u he applica ions o he sys em a e web/junype in eg a- ion, which would mean he sys em would be accessible om a websi e in e ace, and sc ip syn hesiza ion, which is he opposi e o compila ion, in o he wo ds he abili y o ansla e a py hon sou ce ile o he Pe simmon isual ep esen a ion. Gan Cha Wi h he de ined miles ones a Gan cha o he p ojec de elopmen was d awn. 2016 2017 Oc obe No embe Decembe Janua y Feb ua y Ma ch Ap il Dis il Idea Planning Implemen a ion I e a ion 1 Capped I e a ion 2 Pa i y I e a ion 3 Compila ion Repo Building Re inemen Figu e 6.2.: Gan Diag am o he p ojec de elopmen . 1Kaggle.com 24 De elopmen Me hodology The chosen me hodology is based on agile me hodologies such as Sc um o Ex eme P og amming, meaning ha he e is no a comple e model o he desi ed sys em like in model d i en de elopmen (Selic, 2003), no a comple e planning o e e y de elop- men de ail a he s a o de elopmen , such as on Wa e all (Pe e sen e al., 2009), ins ead he e a e con inuous i e a ions, as e and smalle han adi ional de elopmen i e a ions ha allow o mo e oppo uni y o eac and adap o change (Beck e al., 2001). These i e a ions las wo weeks and a e called sp in s, and a boa d is used o keep ack o all cu en and u u e asks. On a adi ional Sc um me hodology, he p oduc owne pu s uses cases (i ems) in o he p oduc backlog. Each sp in he sc um mas e and he de elopmen eam ha e a mee ing called Sp in Planning e en (Schwabe and Beedle, 2002), whe e i ems he cu en sp in i ems om he p oduc backlog o be done a e decided and b oken down in o asks o be done. I ems can also be pushed back in o he backlog i hey a e no achie able o ha e a lowe p io i y. Howe e , his me hodology does no eally i he de elopmen o his p ojec , since he e is no eam, he e is no need o supe luous and unnecessa y p ocesses. The e is no e ospec i e a e each sp in and he e is no speci ic weigh o cos assigned o each ask. Du ing a sp in he nex sp in asks a e mo ed om he p oduc backlog in o he sp in planning column and b oken down u he i necessa y. Task a e de ined by use cases and can be b oken down u he by using checklis s on he asks. I a ask is no ully comple ed i can be mo ed back on o he p oduc backlog. The planning boa d can be ound a h ps:// ello.com/b/JmG3xy0U/pe simmon Sou ce Code The sou ce code o his p ojec is hos ed on h ps://gi hub.com/Al a Be /Pe simmon, he o ganiza ion o he code ollows he ea u e b anch wo k low (A lassian, 2014), i can be desc ibed in e ms o i s b anches. Mas e b anch. The mas e is he main b anch, meaning ha i is he de aul on he emo e web in e ace, and he only b anch whe e deploymen s happen, he e is no ac ual de elopmen apa om ho ixes, inse ead i me ges commi s om de , o ming a elease on each me ge. De b anch. The de b anch ep esen s he mos ecen commi s, commi s a e made usually di ec o his b anch. Tes a e un when commi s om his b anch a e pushed o he epo, bu no deploymen . 25 9. Implemen a ion On his chap e he implemen a ion o he sys em is de ailed, explained wha was done in each i e a ion. A e he i e a ions Pe simmon in e media e ep esen a ion is explained. Finally, some o he mos complex echnical p oblems along hei espec i e solu ions a e de ailed. Fi s I e a ion Figu e 9.1.: Implemen a ion o he i s in e ace 32 Fo he i s i e a ion, he p io i y was o ge a p oo o concep in o de o see whe e he di icul ies can appea , wi h a ew simple classi ie s and c oss- alida ion echniques. As such a bu on-based in e ace wi h e y limi ed wo k low c ea ion was chosen. The chosen classi ie s we e simple and well-unde s ood me hods such as K-Nea es Neigh- bo s, Logis ic Reg ession, Nai e Bayes, Suppo Vec o Machines and Random Fo es , which a sligh ly mo e complex me hod ha in ol es ensemble o Decision T ees, bu gi es good esul s in wide a ie y o p oblems. All hese classi ie s ha e ew pa ame e s on hei espec i e sklea n implemen a ions, and o his p o o ype he in e ace did no allow modi ying any o hem, as he i would ha e clu e ed and i was no a necessa y ea u e. Also, all o hem a e classi ie s, as i simpli ies he in e ace, since eg ession and clus e ing ha e some incompa ibili ies. Apa om he empo a y in e ace he backend had o be buil . Since he wo k low was ixed he backend simply ecei ed he node as a gumen s and execu ed hose, meaning he p e iously explained execu ion algo i hm was no needed o his i e a ion. Second I e a ion Fo he second i e a ion he d ag and d op eel was he main p io i y. As such a e de eloping he ab panel d aggable boxes we e de eloped, hese boxes needed o be connec ed h ough pins. The logic behind he pins and he blocks is qui e hea y, as he e is a igh coupling be ween he blackboa d1, blocks and he pins on hem, as all o hese pa s elay in o ma ion o each o he while he use is d agging a cable be ween wo pins, his is u he explained on he “Making a connec ion” sec ion. This igh coupling means he e is a no iceable lag when mo ing he cable oo as on low-end compu e s, he e a e se e al solu ions o his, bu he mos con enien is op imizing he me hod. I mo e op imiza ion is needed o his pa icula unc ion ools such as Numba2o Cy hon could be used. Thi d I e a ion Fo he hi d and inal i e a ion, he ocus was on imp o ing he isual aspec , adding help ul aids o he use expe ience. The main addi ion being adding a no i ica ion sys ems ha gi es eedback o he use abou he ou come o hei ac ions and he ype sys ems ha p e en s c ea ing mal o med pipelines. O he mino imp o emen s o he sys em we e he addi ion o a wa ning when he in ended connec ion is no possible, by 1Blackboa d is whe e he blocks and connec ions eside. 2Numba is a py hon lib a y ha allows he compila ion and ji ing o unc ions in o bo h he CPU and he GPU h p://numba.pyda a.o g/ 33 Figu e 9.2.: Second i e a ion implemen a ion 34 changing he colo line o ed, and a wa ning showing up when a block has only some o hei inpu s connec ed. Figu e 9.3.: Thi d i e a ion in e ace showing a wa ning Model View Con olle Since he beginning o de elopmen sepa a ion o logic and p esen a ion has been a p i- o i y. Fo his eason, he Model View Con olle 3pa e n has been applied, sepa a ing Model ( ep esen ed by he subpackage backend), View ( ep esen ed by he .py iles on iew subpackage) and Con olle (co esponding o he .k iles on iew subpackage). This way coupling is kep as minimal as possible, enabling swapping he cu en ki y amewo k o ano he one by jus changed he iew, no modi ica ions o he backend needed. 3Model View Con olle is a so wa e pa e n. 35 In o de o a oid epe i ion ex ensi e use o classes coupled wi h eusable cus om ki y Widge s we e used. This o example mean ha each indi idual pin on each block is a class, his p o ed use ul o de ining ma ching pins in di e en blocks (like when connec ion a pin ha sends da a o a pin ha ecei es i ). Fo mo e in o ma ion abou in e nal package dis ibu ion check appendix A. Making a Connec ion One o he mos complex pa s o he sys em is s a ing, econnec ing and dele ing a connec ion be ween blocks, i in ol es se e al ac o s, asynch onous callbacks and a e y s ong coupling be ween all elemen s. Blackboa d Block Ou Pin Connec ion Block InPin Figu e 9.4.: Widge T ee In o de o unde s and how connec ions a e made i is necessa y o unde s and how Ki y handles inpu . A su ace le el Ki y ollows he adi ional e en -based inpu managemen , wi h he e en p opaga ing downwa ds om he oo . Howe e , while adi ionally inpu s e en s a e only passed down o componen s ha a e on he e en posi ion Ki y passes he e en s o almos all child en by de aul , his is done because in phones (one o Ki y a ge s is And oid) ges u es end o s a ou side he ac ual widge hey in end o a ec . On Ki y he e a e h ee main inpu s e en s, on_ ouch_down ha ge s called when a key is is p essed, on_ ouch_mo e ha is no i ied when he ouch is mo ed, i.e. a inge mo es ac oss he sc een, o on his cases when he mouse mo es, and on_ ouch_up ha is i ed when he ouch is eleased. Le ’s ep esen he possible ac ions as use cases, he ou e * ep esen s on_ ouch_down, - ep esen s on_ ouch_mo e, and he inne * on_ ouch_up: • (On pin) S a a connec ion. • (On connec ion) Modi y a connec ion. –Follow cu so . –(On pin) Type check. * (On a pin) Es ablish connec ion i possible. 36 * (Elsewhe e) Remo e connec ion. Logic is spli in wo big cases, c ea ing a connec ion and modi ying an exis ing one. C ea ing a connec ion in ol es c ea ing one end o he connec ion, bo h isually and logically and p epa ing he line ha will ollow he cu so . On he o he hand, modi ying a connec ion means emo ing he end ha is being ouched. These wo cases can be handled by di e en classes, pin on he i s case and connec ion o he las . Mo ing and inishing he connec ion use he same code o bo h. Connec ion Block Ou Pin end Block InPin s a Figu e 9.5.: Connec ions be ween elemen s Wi hou ge ing oo deep in o implemen a ion de ails, ends canno jus be emo ed, he e a e isual binds ha ha e o be unbinded and emo ed om he can as, and when a connec ion is des oyed ( his only happens inside on_ ouch_up, bu i can be ei he he pins o he blackboa d on_ ouch_up depending i he connec ion is des oyed because he pin iola es ype sa e y o he e is no pin unde he cu so espec i ely) i has o unbind he logical connec ions o he pins hemsel es. Fo his eason, connec ion has high-le el unc ions ha do he unbind, ebind and dele ion o ends, as long as he necessa y elemen s a e passed (dependency injec ion pa e n). This is he econnec ing logic, no ice how he econnec ing is o wa d o backwa ds depending on which edge he ouch has happened, o cou se i nei he has been ouched he ouch e en is no handled. de on_ ouch_down(sel , ouch): """ On ouch down on connec ion means we a e modi ying an al eady exis ing connec ion, no c ea ing a new one. """ i sel .s a .collide_poin (* ouch.pos): sel . o wa d = False sel .unbind_pin(sel .s a ) sel .unci cle_pin(sel .s a ) sel .s a .on_connec ion_dele e(sel ) ouch.ud['cu _line']=sel sel .s a = None e u n T ue eli sel .end.collide_poin (* ouch.pos): sel . o wa d = T ue sel .unbind_pin(sel .end) sel .unci cle_pin(sel .end) sel .end.on_connec ion_dele e(sel ) 37 ouch.ud['cu _line']=sel sel .end = None e u n T ue else: e u n False Figu e 9.6.: Connec ion modi ica ion handling Visualizing he Da a Flow One o he la es ea u es ha made i in o Pe simmon is he isualiza ion o he da a lowing h ough he cables be ween blocks, his was an in e es ing echnical p oblem, since i in ol ing elaying da a back om he backend in o he on end (p e iously he communica ion be ween on and backend was unidi ec ional). Bu in o de o p ese e he decoupling be ween bo h he backend IR had o emain un ouched. Fo his eason i was decided ha he backend has an e en whe e i announces i has inished execu ing a block and he on end has o subsc ibe o i . Bu he on end does no ecei e he block, only he hash, since ha is all he backend has, and i has o compa e wi h all block hashes o ind he ac ual block. A e his, he backend has o make he ou going connec ions o ha block pulse, mean- ing o example changing he alue o he wid h o he line be ween ce ain alues, a unc ion ha wo ks well o his is he sin unc ion. The icky pa is ha each ime he unc ion is called i has o emembe he p e ious alue in o de o g ow o dec ease he wid h acco dingly, his canno be done on a egula unc ion since using sleep would eeze he en i e applica ion, and he bes way o main ain s a e be ween execu ions is using a gene a o (also known as semi-co ou ines). Bu wha happens when co ou ine needs o be s opped om being called? Ki y has a mechanism whe e i he scheduled unc ion e u ns False i will s op calling, by de aul ou co ou ine does no e u n any meaning ul alue, bu i is possible o yield a inal False ha will s op he calls. Bu how is ha yield igge ed? The p ope solu ion solu ion is using a ull co ou ine (ei he a gene a o -based one o he newe asyncio ones), bu hen concu ency issues appea s, such ha since he co ou ine is being called 20 imes pe second i he co ou ine is called while i is execu ing he scheduled in e al i will igno e he second call. The solu ions comes om execu ions, simila o a as in e up in ha dwa e i is possible o h ow a execu ion on a co ou ine ha (maybe) is unning, his also mean ha he h owing hijacks he cu en execu ion, leading o wo di e en e u ns needed, one o he in e up execu ion and ano he o he p e ious unning execu ion (i i was unning, i no i will be on he nex scheduled call). 38 Wi h he h ow solu ion he e is no need o a ull co ou ine anymo e, and a gene a o can be used again. de pulse(sel ): sel .i = sel ._change_wid h() # C ea e i e a o Clock.schedule_in e al(lambda _: nex (sel .i ), 0.05)# 20 FPS de s op_pulse(sel ): sel .i . h ow(S opI e a ion)# Hijacking execu ion de _change_wid h(sel ): y: o alue in sel ._wid h_gen(): sel .lin.wid h = alue yield excep S opI e a ion: sel .lin.wid h = 2# Re u n wid h back o de aul yield # This yield is o he hijacking execu ion yield False # And his o he egula execu ion de _wid h_gen(sel ): """ In ini y oscilla ing gene a o (be ween 2 and 6) """ al = 0 while T ue: yield 2* np.sin( al) + 4 al += pi / 20 Bina y Dis ibu ion The in e p e a i e na u e o Py hon does no make c ea ing an execu able bina y easy, pa icula ly cPy hon he s anda d implemen a ion and e e ence p o ides no ooling o c ea e an execu able bina y. Fo his ask PyIns alle was chosen, he p ocess o c ea ing a bina y is mos ly au o- ma ed, gi en a sc ip i ies o ead he impo s and include hem, inally i embeds a small in e p e e o un his code. The p oblem wi h his app oach is ha Py hon allows o al e na i e ways o impo ing, i also b eaks esou ce loading a execu ion ime (since i has o c ea e a empo a y olde ). This esul s in manually speci ying hidden dependencies and non py hon iles (on his case mos ly k iles). Un o una ely, his p ocess has o be done on a windows sys em, and as such canno be done on he CI4se e , o see how Pe simmon u ilizes CI check he appendix B. 4Con inuous In eg a ion is a e m ha e e s o he idea o es ing, building, gene a ing documen a ion 39 and e en deploying au oma ically h ough a commi on he e sion con ol sys em. 40 10. Type Checking Al hough Py hon has no obus ype checking s ep i is possible o ou isual language o ha e ha d gua an ees o co ec ness a w i e ime, meaning ha he building o inco ec pipelines can be a oiding al oge he . G adual Typing Py hon allows o g adual yping since 2014 (Rossum e al., 2014), meaning ha unc ion pa ame e s can be speci ied and ools such as mypy will check o possible ype e o s, i some pa ame e o unc ion ype is no speci ied he ool will simply igno e he associa ed checks. These ools p o ide a use ul ool o in oduce ype checking in cu en and new py hon code, howe e hey un ou side he py hon execu ion (i.e. hey un on he non-exis en py hon compile ime) and Pe simmon needs un ime ype checking o dynamic block connec ions. Ne e heless, his is a use ul ool o imp o ing he code quali y, specially o he backend code, because i is much pu e ha he on end. I is also a e e ence o Pe simmon ype sys em. W i e Time On he p e ious sec ion un ime ype checking was men ioned, his is because on he Py hon side he ype checks ha e o be done a un ime due o blocks being spawned and connec ed dynamically. Bu om he isual language pe spec i e he checks a e done e en be o e compile ime (on he li e a u e e e ed as w i e ime). The wo languages As seen on he p e ious sec ions and he implemen a ion chap e Py hon and Pe simmon a e essen ially wo di e en languages, bu jus how di e en a e hey? 41 posed, such as adding a zoom abili y, a bubble spawning block sys em ins ead o abs. Also some new ideas we e p oposed, such as undo unc ionali y, o isualiza ion op ions. On he o he hand pa icipan s p aised he d ag and d op na u e o he in e ace, he wide selec ion o ml algo i hms and es op ions, he use o colo s o indica e ypes, consis en design, easy o na iga e and shallow lea ning cu e. The e o handling and he esilience o he applica ion we e men ioned, as well as he simple ins alla ion p ocess wi hou he need o dependencies ins alla ion. 48 12. Conclusiones T as la e aluación es una opo unidad pa a obse a lo que el sis ema ha conseguido. Re isión de Obje i os Es udio de Viabilidad: La e aluación pa ece demos a que es posible c ea una in e az de ap endizaje au omá ico isual que es lexible a la ez que ela i amen e ácil de usa , incluso pa a es udian es, incluyendo un sis ema de ipos y no i icación de e o es en iempo de esc i u a. Diseño y Usabilidad: La implemen ación inal sigue los bosquejos iniciales, demos ando que el diseño inicial enía undamen os sólidos. Los buenos esul ados de la e alu- ación, incluyendo los comen a ios inales de los pa icipan es, pa ecen indica que la in e az sa is ace los obje i os man eniendo una in e az simple. E aluación: A pesa del bajo núme o de pa icipan es la e aluación esul ó en esul ados mayo men e posi i os, incluyendo eedback que in luyó la ase inal de desa ollo. He amien a de Ap endizaje: Con la mayo pa e de los obje i os cumplidos el sis ema ha alcanzado un es ado en el cual iene su icien e uncionalidad como pa a se usado como he amien a de ap endizaje, especialmen e g acias al sopo e de los wo k lows más simples y usados. Incluso dos pa icipan es eseña on la acilidad de uso y la capacidad de ealiza acciones complejas (como ajus e de hype -pa ame os) de mane a sencilla compa ado con o os amewo ks y lib e ías. Acele a análisis explo a o io: Al igual que en el úl imo obje i o, el sis ema ha alcanzado un ni el de uncionalidad su icien e en el que ealiza análisis de da os e i e a sob e dis in os mé odos es ela i amen e ápido (simplemen e desengancha y en- gancha las conexiones a o o bloque). Cuando un bloque necesa io no es aba im- plemen ando la implemen ación e a ela i amen e sencilla (la mayo ía de bloques son menos de 20 lineas de código). Implemen ación: Al inal del p oyec o los eque imien os no- uncionales han sido cumpli- dos, culminando en un ejecu able sin dependencias que los pa icipan es han usado pa a la e aluación. Es e p oceso de ácil ins alación ha sido comen ado po a ios pa icipan es, así como el endimien o del sis ema, man eniendo la in e az sensible al inpu mien as se ende izan múl iples bloques y el p oceso se ejecu a simul ánea- men e (mul ihilo). 49 Re ospec i a Con más de 7000 líneas de código, 10 eleases, y más de 200 commi s, Pe simmon se ha con e ido en un p oyec o de amaño medio, desde su concepción ha llamado la a ención, con más de 3000 isi as y 100 es ellas en Gi hub. Ha apa ecido en múl iples,páginas web, e incluso ha ganado el p emio al mejo p oyec o en el compshow 2017 en la uni e sidad de He o dshi e. Figu e 12.1.: Pe simmon en el ex anje o Conclusión En conclusión el sis ema ha conseguido alcanza un es ado es eable en el cual los pa ic- ipan es han e aluado la usabilidad, lexibilidad y po encial, alo ándolo posi i amen e. Es o pa ece indica que es posible mejo a la si uación de he amien as isuales de ap endizaje au omá ico con pequeñas mejo as que impac an la expe iencia de usua io. Ca ac e ís icas como el menú de búsqueda in eligen e usa la in ospección pa a suge i bloques adecuados, usando el sis ema de ipos pa a ayuda al usua io a c ea p ocesos más ápida y ácilmen e. Es o se co esponde con la hipó esis del p oyec o, así como con el obje i o de que el sis ema no debe ía solo hace di ícil o imposible c ea p ocesos inco ec os, sino hace más ácil y ápido c ea g a os co ec os. 50 Da más pode al usua io no signi ica complica la in e az, de hecho puede se lo con a io. T abajo Fu u o • Expone pa áme os opcionales. • Puli aspec os isuales. –Ca ego ías en el menú de búsqueda. –Más indicado es du an e acciones de a as e. • Se ialización de los g a os. • Sopo e de mo imien o y zoom sob e el g a o. • Gene ación au omá ica de bloques desde unciones en Py hon. • Capacidad de deshace (Command pa e n). • Selección en á ea. • C eación de wo lows comunes median e plan illas. • Uni /In eg ación/End o end es ing. • Deploymen au omá ico en Windows • In eg ación con igua. • Cacheado de esul ados simila es a un REPL1. 1Un Read E al P in Loop es una consola in e ac i a p o enien e de LISP que pe mi e la ejecución in e ac i a de exp esiones, gua dando los esul ados in e medios pa a el uso explo a o io. 51 13. Pos mo em A e he e alua ion i is ime o make a e ospec i e, look wha Pe simmon has achie ed. Objec i es Re iew Feasibili y: E alua ion seems o show ha i is possible o c ea e a machine lea ning isual in e ace ha is bo h lexible and ela i ely easy o use, e en o lea ne s, including a ype sys em and e o s in compila ion ime. Design and Usabili y: The inal implemen a ion closely ollowed he ini ial ske ches, p o - ing he ini ial design had solid undamen als. The good e alua ion sco es, and inal ema ks gi en by pa icipan s, seem o demons a e ha he in e ace has accomplished i s objec i es o p oducing a powe ul ye simple o use in e ace. E alua ion: Despi e ha ing a low numbe o pa icipan s he e alua ion esul ed in a mos ly unanimous good e iews o he so wa e, as well as p o iding e y use ul eedback o u u e imp o emen s. Lea ning Tool: Because mos o he miles ones we e achie ed he inal sys em has eached a s a e whe e i is use ul enough o i s use as a lea ning ool hanks o suppo ing he simples (and mos common) wo k lows, i was e en ema ked by wo pa ici- pan s how easy i was o use, and how easy i was o do complex ac ions (such as hype -pa ame e uning) compa ed o o he amewo ks/lib a ies. Fas e Explo a o y Wo k: Like las objec i e hanks o he cu en s a e o he sys em i is p e y as o pe o m ea ly ml analysis, when limi ed by he lack o a block i was p e y easy and as adding a block ha sol ed he p oblem (in a ound ~20 lines o code). Implemen a ion: A he end o he p ojec he non- unc ional equi emen s ha e been me , deli e ing a windows single execu able ile ha pa icipan s used o he e al- ua ion, while keeping a good pe o mance, handling many blocks wi hou a hi ch, and keeping he ame a e s eady while modi ying connec ions and unning he execu ion o he pipeline simul aneously. 52 Re ospec i e Wi h o e 7k lines o code, 10 eleases, and mo e han 200 commi s, Pe simmon s ands as a medium size codebase, since i s incep ion i has ga he ed a en ion, wi h o e 3000 isi s, and mo e han 90 s a s on Gi hub. I has been ea u ed on mul iple,websi es, and e en won bes p ojec a he 2017 comp- show a Uni e si y o He o dshi e. Figu e 13.1.: Chinese machine lea ning o um Conclusion In conclusion he sys em has managed o each a es able s a e in which pa icipan s ha e ema ked i s usabili y, lexibili y and po en ial. This seems o indica e ha is is possible o small imp o emen s on isual machine lea ning ools do make an impac on he use expe ience Fea u es like he sma bubble ha use in ospec ion o sugges sui able blocks o connec le e age he ype sys em o help he use c ea e he pipelines as e and easie . This co esponds wi h he hypo hesis o he p ojec , as well as he objec i e ha he sys em should no only make i ha d o impossible o cons uc inco ec g aphs, bu should make i easie and as e o c ea e co ec g aphs. 53 Gi ing mo e powe o he use does no mean con olu ing he in e ace, in ac i can be he opposi e. Fu u e Wo k • Su ace o op ional pa ame e s. • Visual Polish. –Sma Bubble b eakdown by ca ego y. –Mo e indica o s when d agging/d opping. • G aph Se ializa ion. • Suppo mo e and zoom in backg ound. • Au oma ic block gene a ion om Py hon unc ion. • Undo unc ionali y (Command pa e n). • A ea d ag selec . • Skele ons o common wo k lows. • Uni /In eg a ion/End o end es ing. • Au oma ic windows deploymen . • Con inuous in eg a ion. • Cache esul s simila o a REPL1. 1A Read E al P in Loop is an in e ac i e console many mode n p og amming languages ha allows o he in e ac i e execu ion o exp essions, sa ing he esul s in a local session. 54 Bibliog aphy Abadi, M., Aga wal, A., Ba ham, P., B e do, E., Chen, Z., Ci o, C., Co ado, G.S., e al. (2016), “Tenso low: La ge-scale machine lea ning on he e ogeneous dis ibu ed sys ems”, a Xi P ep in a Xi :1603.04467. A lassian. (2014), “Fea u e b anch wo k low”, a ailable a : h ps://www.a lassian.com/ gi / u o ials/compa ing-wo k lows# ea u e-b anch-wo k low (accessed 22 Ap il 2017). Backus, J. (1978), “Can p og amming be libe a ed om he on neumann s yle?: A unc ional s yle and i s algeb a o p og ams”, Communica ions o he ACM, ACM, Vol. 21 No. 8, pp. 613–641. Ba ga, R., Fon ama, V., Tok, W.H. and Cab e a-Co don, L. (2015), P edic i e Analy ics wi h Mic oso Azu e Machine Lea ning, Sp inge . Beck, K., Beedle, M., Van Bennekum, A., Cockbu n, A., Cunningham, W., Fowle , M., G enning, J., e al. (2001), “Mani es o o agile so wa e de elopmen ”. Biancuzzi, F. and o he s. (2009), Mas e minds o P og amming: Con e sa ions wi h he C ea o s o Majo P og amming Languages, “ O’Reilly Media, Inc.” Boehm, B.W. (1991), “So wa e isk managemen : P inciples and p ac ices”, IEEE So wa e, IEEE, Vol. 8 No. 1, pp. 32–41. Dalke, A. (2003), “Visual da a low p og amming”, a ailable a : h p://www. dalkescien i ic.com/w i ings/dia y/a chi e/2003/09/22/VisualP og amming.h ml (accessed 21 Feb ua y 2017). Da enpo , T.H. and Pa il, D. (2012), “Da a scien is : The sexies job o he 21s cen- u y”, a ailable a : h ps://hb .o g/2012/10/da a-scien is - he-sexies -job-o - he-21s -cen u y (accessed 25 Feb ua y 2017). Ennals, R. and Jones, S.P. (2003), “Op imis ic e alua ion: An adap i e e alua ion s a - egy o non-s ic p og ams”, in ACM Sigplan No ices, Vols. 38, ACM, pp. 287–298. E icsson, G., Glo e , D., P ice, A. and F anks, L. (2017), “Azu e machine lea ning equen ly asked ques ions: Billing, capabili ies, limi a ions, and suppo ”, a ailable a : h ps://docs.mic oso .com/en-us/azu e/machine-lea ning/machine-lea ning- aq (accessed 11 Ap il 2017). Gschwind, M. (2017), “Powe AI: Looking back on a yea o deep lea ning inno a ion - and in o he u u e”, a ailable a : h ps://www.ibm.com/de elope wo ks/communi y/ 55 blogs/ e313521-2e95-46 2-817d-44a4 27eba32/en y/Powe AI_Looking_back_on_a_ yea _o _Deep_Lea ning_inno a ion_and_in o_ he_ u u e?lang=en (accessed 12 Ap il 2017). Hall, M., F ank, E., Holmes, G., P ah inge , B., Reu emann, P. and Wi en, I.H. (2009), “The weka da a mining so wa e: An upda e”, ACM SIGKDD Explo a ions Newsle e , ACM, Vol. 11 No. 1, pp. 10–18. Ha is, J.G. and Ei el-Po e , R. (2015), “Da a scien is s: ’As a e as uni- co ns”’, a ailable a : h ps://www. hegua dian.com/media-ne wo k/2015/ eb/12/ da a-scien is s-as- a e-as-unico ns (accessed 3 Ap il 2017). Hughes, G. (1968), “On he mean accu acy o s a is ical pa e n ecognize s”, IEEE T ansac ions on In o ma ion Theo y, IEEE, Vol. 14 No. 1, pp. 55–63. Jones, E., Oliphan , T. and Pe e son, P. (2014), “SciPy: Open sou ce scien i ic ools o py hon”. Kelly, J.L., Lochbaum, C. and Vysso sky, V.A. (1961), “A block diag am compile ”, Bell Sys em Technical Jou nal, Wiley Online Lib a y, Vol. 40 No. 3, pp. 669–676. Khabaza, T. (2005), “Ha d ha s o da a mine s: My hs and pi alls o da a mining”, a ailable a : h p://www.spss.ch/upload/1113911601_da a_mining_khabaza (3).pd (accessed 16 Decembe 2016). Launchbu y, J. (1993), “A na u al seman ics o lazy e alua ion”, in P oceedings o he 20 h Acm Sigplan-Sigac Symposium on P inciples o P og amming Languages, ACM, pp. 144–154. McB ide, C. and Pa e son, R. (2008), “Applica i e p og amming wi h e ec s”, Jou nal o Func ional P og amming, Camb idge Uni P ess, Vol. 18 No. 01, pp. 1–13. McKinney, W. and o he s. (2010), “Da a s uc u es o s a is ical compu ing in py hon”, in P oceedings o he 9 h Py hon in Science Con e ence, Vols. 445, an de Voo S, Millman J, pp. 51–56. Minas, M. and F ey, G. (2002), “Visual plc-p og amming using signal in e p e ed pe i ne s”, in Ame ican Con ol Con e ence, 2002. P oceedings o he 2002, Vols. 6, IEEE, pp. 5019–5024. Pe e sen, K., Wohlin, C. and Baca, D. (2009), “The wa e all model in la ge-scale de- elopmen ”, in In e na ional Con e ence on P oduc -Focused So wa e P ocess Imp o e- men , Sp inge , pp. 386–400. P ess, G. (2015), “The hun o unico n da a scien is s li s sala ies o all da a ana- ly ics p o essionals”, a ailable a : h ps://www. o bes.com/si es/gilp ess/2015/10/09/ he-hun - o -unico n-da a-scien is s-li s-sala ies- o -all-da a-analy ics-p o essionals/ #38147ccc5258 (accessed 3 Ap il 2017). Puge , J.-F. (2017), “The mos popula language o machine lea ning and da a science is 56 …”, a ailable a : h p://www.kdnugge s.com/2017/01/mos -popula -language-machine-lea ning-da a-science. h ml (accessed 11 Ap il 2017). Rajpu ohi , A. (2016), “Businesses will need one million da a scien is s by 2018”, a ail- able a : h p://www.kdnugge s.com/2016/01/businesses-need-one-million-da a-scien is s-2018. h ml (accessed 25 Feb ua y 2017). Rossum, G. an and Le ki skyi, I. (2014), “PEP 483 – he heo y o ype hin s”, a ailable a : h ps://www.py hon.o g/de /peps/pep-0483/ (accessed 20 Ap il 2017). Rossum, G. an, Leh osalo, J. and Langa, Ł. (2014), “PEP 484 – ype hin s”, a ailable a : h ps://www.py hon.o g/de /peps/pep-0484/ (accessed 20 Ap il 2017). Ryan, F. (2016), “A look a popula machine lea ning amewo ks”, a ailable a : h p: // edmonk.com/ yan/2016/06/06/a-look-a -popula -machine-lea ning- amewo ks/ (accessed 12 Ap il 2017). Sau o, J. (2012), “10 hings o know abou he single ease ques ion (seq)”, Measu ing U, 2012. Sau o, J. and Dumas, J.S. (2009), “Compa ison o h ee one-ques ion, pos - ask usabili y ques ionnai es”, in P oceedings o he Sigchi Con e ence on Human Fac o s in Compu ing Sys ems, ACM, pp. 1599–1608. Schwabe , K. and Beedle, M. (2002), Agile So wa e De elopmen wi h Sc um, Vols. 1, P en ice Hall Uppe Saddle Ri e . sciki -lea n. (2016), “Who is using sciki -lea n?”, a ailable a : h p://sciki -lea n.o g/ s able/ es imonials/ es imonials.h ml (accessed 8 Feb ua y 2017). Selic, B. (2003), “The p agma ics o model-d i en de elopmen ”, IEEE So wa e, IEEE, Vol. 20 No. 5, pp. 19–25. Shah, R. (2014), Mas e ing he A o Un eal Engine 4-Bluep in s, Lulu.com. Shu, N.C. (1988), Visual P og amming, Van Nos and Reinhold New Yo k. Simões, T. (2015), “Visual p og amming is unbelie able… he e’s why we don’ belie e in i ”, a ailable a : h ps://www.ou sys ems.com/blog/ isual-p og amming-is-unbelie able. h ml (accessed 11 Feb ua y 2017). Sousa, T.B. (2012), “Da a low p og amming concep , languages and applica ions”, in Doc o al Symposium on In o ma ics Enginee ing, Vols. 130. S aníček, P. (n.d.). “Pale on. com (aka colo scheme designe 4)”. Taylo , D. (2016), “H p://www.kdnugge s.com/2016/10/ba le-da a-science- enn- diag ams.h ml”, a ailable a : h p://www.kdnugge s.com/2016/10/ba le-da a-science- enn-diag ams. h ml (accessed 7 June 2017). Thaku , A. (2016), “App oaching (almos ) any machine lea ning p oblem”, a ailable a : h p://blog.kaggle.com/2016/07/21/app oaching-almos -any-machine-lea ning-p oblem-abhishek- haku / 57