Herramienta de enseñanza de aritmética de computadores de nueva generación. Caso de estudio de posit

Herramienta de enseñanza de aritmética de computadores de nueva generación. Caso de estudio de posit. E-learning tool for next generation computer arithmetic. Study case of posit. Trabajo de Fin de Grado Curso 2022–2023 Autor David Godoy Ruiz Director Guillermo Botella Juan Raúl Murillo Montero Grado en Desarrollo de Videojuegos Facultad de Informática Universidad Complutense de Madrid Herramienta de enseñanza de aritmética de computadores de nueva generación. Caso de estudio de posit. E-learning tool for next generation computer arithmetic. Study case of posit. Trabajo de Fin de Grado en Desarrollo de Videojuegos Autor David Godoy Ruiz Director Guillermo Botella Juan Raúl Murillo Montero Convocatoria: Septiembre 2023 Grado en Desarrollo de Videojuegos Facultad de Informática Universidad Complutense de Madrid 15 de septiembre de 2023 Dedicatoria A toda mi gente, estéis donde estéis v Agradecimientos A la peña del Joyfe por aguantarme todos estos años. A los del Gredos por todo lo performático. A los de los laguitos por juntarnos en los lugares más raros. A los panas de la uni por hacer magia. A los chavales de LAG y de ASCII por ser mucho más que la vida universitaria. A mi familia. A toda la gente que me habré dejado. A la gente que ha pasado por mi vida y me ha dejado sin querer un pedacito suyo. A todos vosotros, que me habéis hecho como soy. vii Resumen Herramienta de enseñanza de aritmética de computadores de nueva generación. Caso de estudio de posit. El estándar IEEE para la aritmética binaria de coma flotante (IEEE 754) lleva siendo la norma para la aritmética de computadores desde 1985 y es usado en los ordenadores de todo el mundo. En 2017 John L. Gustafson presentó un nuevo formato, los posit, con el objetivo de reemplazar a la coma flotante, diciendo que ofrece mayor precisión o precisión similar con un menor número de bits. A medida que ha ido avanzando el tiempo el formato ha ido reuniendo cada vez más seguidores que lo apoyan si no como alternativa directa a la coma flotante como un formato que puede coexistir con ella. Sin embargo los recursos disponibles para aprender este nuevo formato son pocos y están dispersos lo que hace que un usuario nuevo que quiera aprenderlo tenga que buscar información repartida entre varios documentos, algunos de ellos muy técnicos. En este Trabajo de Fin de Grado se ha creado una aplicación de e-learning para facilitar el aprendizaje sobre los posit. Tras investigar detalladamente el funcionamiento del formato y los recursos de aprendizaje disponibles actualmente tanto para coma flotante como para posit se ha desarrollado una aplicación de escritorio para Windows que permite al usuario aprender la estructura del formato y sus operaciones básicas. Para conlcuir se ha hecho un análisis de las características de la aplicación desarrollada en comparación con el resto de herramientas disponibles. Palabras clave Sistema numérico posit, Coma flotante, E-learning, Aritmética computacional, Aplicación de escritorio ix 5.5. PositTool main window . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.6. PositTool operation window for posit information, a one parameter operation ................................. 33 5.7. PositTool operation window for addition/subtraction, a two parameteroperation ............................... 34 5.8. Full display of the addition operation and the glossary . . . . . . . . 34 List of tables 3.1. Comparison of features available in e-learning applications for the floating-point and posit formats . . . . . . . . . . . . . . . . . . . . . 18 5.1. Comparison of Qt, wxWidgets and GTK’s features . . . . . . . . . . 28 6.1. Feature comparison of the proposed application with state of the art e-learningresources............................ 37 xvii Chapter 1 Introduction 1.1. Motivation The IEEE 754 Standard for Floating-Point Arithmetic (IEEE 754) (4) has covered the constant need for computers to represent real numbers with a limited, fixed amount of memory space for decades. But even if it has metaphorically held that weight for that long, when taught this format is barely skimmed over in some educational institutions, to the point that many programmers do not even know the internal workings of the format, just the bare minimum: that it is used to represent real numbers, and that it is somewhat imprecise. If a significant amount of programmers know the bare minimum of an incredibly widespread format, it is no surprise that only people with an express interest in computer arithmetics know that a new format was created, and that it is taking steps to knock the floating point off its high throne: the posit format. Posit mode Type III unums (which will be referred to as posits from now on) were presented as a direct replacement of the current format (1). Unlike its two predecessors, posits were designed to have a simple hardware and software implementation using the same type of low-level circuit constructs that IEEE 754 floats use, even taking less chip area. Because of its fairly low amount of followers posits are lacking hardware support, as it is fairly recent and few people know about it. Anyone interested in learning and/or teaching it will just find a lack of educative resources about this incredible new format. A very important step in order to standardize this incredible new format is creating learning resources in order to accelerate the increase of its spread. 1 2Chapter 1. Introduction 1.2. Objectives The objective of this work is to investigate the posit format and its inner workings in order to develop an E-Learning application to ease the process of obtaining information about them. In order to do so the application must show the user the internal parts of a posit, and explain step by step the basic operations that can be carried out with them: conversion to decimal, addition, subtraction, multiplication and division. All of this must be accompanied by graphics to help visualize the process that is being carried out. In addition, as posit have a great amount of terminology that can be confusing, a glossary will be added in order to facilitate learning. The main goals of this work are: 1. Understand posits and their inner workings in comparison to floats. 2. Study available learning resources to the one that is pretended to develop, their features and flaws. 3. Develop a desktop application that explains the posit format and its main operations. 1.3. Work plan In order to achieve the objectives explained in the earlier section, the next plan will be followed: 1. Study posits and their inner workings in comparison to floats. To investigate about posits and floats. To develop a basic in-house implementation of floats. To look into software implementations of posits and their internal workings. To develop an implementation of posit operations along with step by step explanations. 2. Study available learning resources to the one that is pretended to develop, their structure and quality. To investigate similar apps for posits as well as floats. To study their functions and the resources they offer. 3. Develop a desktop application that explains the posit format and its main operations. To look for a suitable environment to develop desktop applications. 1.4. Document structure 3 To develop the window navigation environment of the application. To adapt the implemented console methods to work in the new environment. To develop a glossary for the application. To develop graphics that facilitate following the already implemented explanations. 1.4. Document structure The document follows the research and development processes that led to the final application. In Chapter 2 the main characteristics of the floating point and the posit format are explained as well as some improvements that posits make over floating point numbers. Next in Chapter 3 the features of the available e-learning resources available for posits are explained and compared with some of the ones available for floating point. In Chapter 4 the development of an own floating point implementation in order to understand the format as well as a posit implementation based on the Universal Numbers Library. Following that in Chapter 5 the implementation of the desktop application is explained entirely. Lastly, in Chapter ?? an evaluation of the final result of the desktop application is shown as well as a comparison with the former studied applications. Finally in Chapter 7 the conclusions and future work are discussed. Chapter 2 Floating Point and Posit In this chapter the background of posits is shown as well as floats, which were considered necessary in order to understand their intended successor. 2.1. Floating Point: the IEEE 754 Standard The floating-point format is used to represent binary real numbers as a string of bits. As shown in Figure 2.1, floats are composed of three components: a sign bit that indicates if the number is positive or negative, an exponent that indicates the position of the radix point and the fraction which usually indicates the fractional part of the normalized number, leaving the first 1 implicit. This is not always the case as denormalized numbers exist, which do not have an implicit 1 and instead have an implicit 0. The IEEE-754 Standard for Binary Floating-Point Arithmetic (4) describes a floating point standard for computer representation. It was the first of its kind and has been used by the majority of computers since its creation in 1985 by the Institute of Electronics and Electrical Engineering, even though it has been revised twice, one in 2008, and another one in 2019. This standard specifies formats and methods and it defines exception conditions and their standard handling in order to represent real numbers in a limited amount of bits. S Exponent E Unsigned significand M Figure 2.1: Float Format They have multiple special values: two zeros (-0, +0) and multiple NaN (Not a Number) values as well as denormalized numbers. 5 6Chapter 2. Floating Point and Posit 2.2. Posit The Type III unum, better known as posit, is an arithmetic format presented by John L. Gustafson and Isaac Yonemoto in 2017 (1). Unlike the Type I and II unums (which were designed to represent either a number or a specific range in which a number is located, working similarly to interval arithmetic), the posit format was designed to be a direct replacement of the IEEE 754 standard in computers, by making its hardware implementation easier than its other unum counterparts. The posit format is shown in Figure 2.2. Figure 2.2: Posit format (1) It is constituted by 4 different bit fields in order of decreasing significant bits: 1. The sign bit S. It is the same as in signed float or integers: 0 for positive numbers and 1 for negative numbers. If the number is negative it’s necessary to take the 2’s complement of the remaining bits before interpreting them. 2. The regime bit field Rwhich consists of k(minimum 1) identical bits terminated by R0negated as shown in the Figure 2.2. Rrepresents −kif R0= 0, or k−1if R0= 1.kis used to calculate a the scale of useedkwhere useed is 22es . The regime is a variable-length field. The longer the regime, the more bits of other fields are not represented and are considered 0 bits. 3. The exponent bit field Ehas a maximum length of es, dictated by the specific posit format, but one or more bits can be beyond the least significant bit (LSB) and be considered to have value 0. It represents the value ewhich is treated as an unsigned integer. eis used to calculate the scaling value 2e. In the 2022 standard es is always 2 (5). 4. The fraction bit field Fwhich has a maximum length of (n−3−es), but as with the exponent any of those bits beyond the LSB are considered 0. The number of explicit bits is considered m.Frepresents the fraction value f. The value of a posit pis obtained following the next process: 1. If all the bits except Sare 0 it is one of two special values: If S= 0 then p= 0 If S= 1 then p=NaR.NaR(Not a Real) is a special posit value that is used to represent infinity or invalid values (such as the result of 1/0). 2.3. Beating Floating Point 7 2. Otherwise the value of p is obtained from Equation 2.1. p= (−1)sign ·(1 + f)·useedk·2e(2.1) If rounding is needed, it follows the next process: 1. If xis exactly expressible in the posit format no rounding is needed. 2. If x > maxpos, it rounds to maxpos. 3. If x < minpos, it rounds to minpos. 4. Else round to nearest even. Rounding is simple because there is only one form of rounding: nearest value, ties to nearest even. The way posits are designed also cause that there is no overflow or underflow, instead if a posit were to underflow/overflow it rounds to the values minpos/maxpos, which are the smallest nonzero value and the greatest representable value in the format. This makes the handling of overflow or underflow situation easier to manage for programmers than the equivalent in IEEE 754. The problem with posit numbers is the lack of hardware implementations and the fact that the further you go from scale 0 the less precision they have. And, because they are so new, there are fewer resources, especially when comparing to a standard that has been around since 1985. 2.3. Beating Floating Point The posit format has many improvements over the standard floats (1): 1. If the sign bit is 1, the posit bit string is negated (treating it as a standard 2’s complement integer) before decoding the remaining fields. This eliminates the need for “negative zero” and all the complications of having two distinct bit patterns represent the same real value. 2. A machine instruction that tests if two integers are equal will also serve to compare two posits, whereas float instructions need an additional exception tests for negative zero (equal even though bit patterns are different) and also a check for NaN values (not equal, even when bit patterns are identical). 3. There are no denormalized numbers. The implicit "hidden bit" before the fraction bits is always 1. 4. Floats have up to five different rounding modes (nearest-to-even, directed to 0, round to ∞, round to −∞, and nearest-away-from-0), which can be chosen up to the preference of the user as seen in Figure 2.3. This in turn causes that different devices can give different results to the same calculation. Posits only 14 Chapter 3. E-Learning Resources Figure 3.4: Real wheel for posit<5,1> 3.3.2. RacFP RacFP (21) is a project created by students of the Universidad Politécnica de Valencia published in 2006. This desktop application was made with a similar philosophy as the one which this application was approached with: facilitating the learning of an arithmetic format. As such, it has been an inspiration for this work. It was developed with the purpose of helping students understand floating-point real numbers, their related algorithms and the implementation of circuits based on these algorithms. It was designed to be used in undergraduate-level courses of computer engineering, with a purely educational purpose. The tool classifies its theme into three abstraction levels: The first level, internal representation of real numbers; the second level, generic algorithms such as addition and multiplication; and the third level, which introduces the students to real-world examples that show how the format is applied in market available circuits. As seen in Figure 3.8, RacFP has many interactive buttons, each row corresponding to one of the aforementioned levels. When pressed, a pop-up window appears depending on the feature. The first level offers the option of conversion, which asks for user input, and the 3.3. Floating Point Applications 15 Figure 3.5: Float Toy showing πin multiple float formats Figure 3.6: Harald Schmidt’s converter showing the example value 12.6 formats of both input and output. The input (shown in Figure 3.9) can either be in decimal, binary and hexadecimal. For the output, it can be in single precision (32-bit float), double precision (64-bit double) or in a custom format in which the user can specify the number of mantissa and exponent bits. Once entered, the window changes to the one shown in Figure 3.10 to show the binary representation of the number separating the sign, exponent and mantissa fields. It also shows the representation of the number in hexadecimal, and decimal, and the specific details of the format used and the number, like if it’s normalized, denormalized, a zero, NaN, or an infinite while giving a brief explanation of these details. This level also includes sign tables for effective operation, addition/substraction and multiplication/division as seen in Figure 3.11. The second level consists of the generic algorithms for arithmetic operations: addition/subtraction and multiplication/division. First a flowchart (3.12 for the former and 3.8 for the latter) appears with all the possible steps of said operation and a text box below it that offers a brief explanation of the current step and the intermediate results. By pressing the next button the user input window appears asking for both operands, the rounding mode and the specific operation (addition/- subtraction, or multiplication/division). It also offers the format options present in 16 Chapter 3. E-Learning Resources Figure 3.7: Exploring Binary showing the conversion of the example value 27.124 to decimal, normalized binary scientific notation and raw binary the conversion window. After entering the values the window closes and the next button now advances a step at a time through the process highlighting the current step, changing the description, or both depending on what fits the explanation at that moment. An important usability problem with this implementation is that there is no option to return to a previous step or check the explanations after they are overwritten by the next ones, forcing the user to restart the process in order to look at them. The third and last level shows the hardware implementation of the arithmetic operations explained before using flowcharts that simulate HP chips 3.14. The input is similar to the second level but the flowchart is revealed by step instead of being shown all the time and the explanation, this time in a separate window, can show the information of each point of the diagram if the user hovers the mouse over it. All these explanations are also offered in both Spanish and English. 3.3. Floating Point Applications 17 Figure 3.8: RacFP main screen Figure 3.9: Input screen for conversion The comparison between the features for floating-point resources and the posit ones is shown in Table3.1, which shows the shortcomings of the latter. The posit tools have less options of user interaction and practically no text explanations of how they work or the theory behind it. Together it all makes that a new user of the format has no other option other than read through multiple documents. Furthermore this documents have the basic information buried between complicated blocks of text, adding to the confusion. Besides, all the features that posit has over floating point such as the representation of the posit wheel or the lookup tables are not needed for the latter because of its ease of understanding in comparison to the former. 18 Chapter 3. E-Learning Resources Figure 3.10: Conversion result Floating-point applications features Posit applications features Visual Conversion Tables Step by step explanation of: Calculator -Addition Visual Representation -Subtraction -Multiplication -Division -Hardware Implementation Table 3.1: Comparison of features available in e-learning applications for the floating-point and posit formats 3.3. Floating Point Applications 19 Figure 3.11: Sign tables 20 Chapter 3. E-Learning Resources Figure 3.12: Addition flowchart 3.3. Floating Point Applications 21 Figure 3.13: Multiplication and division flowchart 22 Chapter 3. E-Learning Resources Figure 3.14: Addition hardware flowchart Chapter 4 Console Implementation of Float and Posit Structures Two small console applications that simulated the internal structure and operations of both formats. The goal of this was to complement the theoretic learning of both formats (especially the internal step by step process of the operations, which is not clearly included in any document) and as a test for our understanding of the process. In order to do so, an in-house console application that replicated the structure of the formats, the conversion from decimal and a simple operation which was the addition. It was developed in console in order to develop and iterate as fast as possible. C++ was chosen as the language to develop the application from its first iteration. It allows for an easy and fast development process and there are many libraries available for both posit and desktop application development. 4.1. Understanding Float via Console Implementation A console application was developed that simulated the internal workings of floating point numbers, including extracting each binary field from a C++ float and the basic operation of addition. The console methods were an important step in order to understand the differences between floats and posits and completely understand the conversion and addition processes as they are similar to posit but simpler and easier to comprehend. It is not present on the final application as it was a part of the investigation process and the application is focused in posits. 23 30 Chapter 5. Desktop Application Substeps instead have a string vector that is filled line by line with the text explanation of the substep of that step of that operation. 1. NextStep: adds a step to the operation with a given title, which is considered the current step 2. NextSubstep: adds a substep with a given title to the current step added to the operation 3. Add: adds a text line to the current substep added to the current step of the operation 4. AddHighlight: adds the number that corresponds to a flowchart box to the current step. 5.4.3. Flowcharts The flowcharts are created by the main window on operation start and show the flow of the operation in question as seen in Figure 5.3. They receive the current step of the explanation and highlight the relevant steps. Figure 5.3: Addition flowchart 5.4.4. Glossary The glossary is opened with a button in the bottom right corner that shows a list of terms and their definitions as shown in Figure 5.4. 5.4. The Desktop Application Structure 31 The glossary’s terms and descriptions are pulled from a .json file. Figure 5.4: Glossary 5.4.5. Application flow The application flow is seen in Figure 5.1. When the application starts the main window (seen in Figure 5.5 shows the user the options for each of the operations. When clicked, the window changes to the operation window widget shown in Figure 5.6. It shows one text line for input and a go button for one-parameter operations like the Figure (conversion and information breakdown). For two-parameter operations (addition and multiplication), a second text line input is shown for the second operand as well as the option to do the opposite operation (subtraction and division respectively) as can be seen in Figure 5.7. 32 Chapter 5. Desktop Application Figure 5.5: PositTool main window On clicking the start button a text box is created that shows the first step of the corresponding explanation, and the next button (on the bottom right) adds another text box for the next step’s explanation. All the text boxes are distributed along a scroll allowing the user to check all the previous steps. The start button also opens the flowchart, which shows the process of the whole operation and highlights the latest step that is shown to the user. At any point during or after the explanation, the user can change the input and press the button to start again or go back to the main menu to go to another operation. The whole process can be seen in Figure 5.8 which shows the text boxes for the addition operation of 7 + 15 with the flowchart and glossary. The glossary is opened with a button on the top right of the main window. It opens a separate window filled with terms used in the explanations and their definitions. 5.4. The Desktop Application Structure 33 Figure 5.6: PositTool operation window for posit information, a one parameter operation 34 Chapter 5. Desktop Application Figure 5.7: PositTool operation window for addition/subtraction, a two parameter operation Figure 5.8: Full display of the addition operation and the glossary Chapter 6 Evaluation of the Application In this chapter, an evaluation of the final application will be compared against the applications previously studied. 6.1. Characteristics The developed application offers many characteristics that were not previously available for learning the posit format: It offers step-by-step explanations of the main operations of the format: conversion, addition, subtraction, multiplication and division. This is a great help for users to learn the internal process of posits with practical examples. Among the operations the most important one is the conversion which is the big difference from floating point. It shows how every field of a posit is extracted from a decimal number, how it is rounded and all possible exceptions. The glossary offers the user easy access to a list of common concepts that may be confusing for beginners. It is a desktop application, which means that once downloaded it can work without the need to access the internet. The application is open-source which means that everyone can access the source code and make contributions to it. It is available in https://github. com/dagodoy/TFG_2023. 6.2. Limitations Even though the application offers a vast array of utilities for learning the format it also comes with its own shortcomings: 35 36 Chapter 6. Evaluation of the Application The application is missing features already available in other posit applications such as posithub’s lookup tables. The explanations are plain text which can be confusing to get used to. The structure of the application does not allow for colored text or most kinds of format. Requiring previous installation also means that it is only compatible with Windows 10 onward and is harder to access than a web application. The application does not allow the user to choose the posit type freely unlike other posit resources, limited only to 8 or 16 bits and an es of 1 or 2. The application is only available in English. These are all improvements to add in future works. Because the application is open-source anyone who is interested in it can collaborate in order to point out and improve its limitations. 6.3. Use cases A main use of the application is to help people doing individual research of the format. If someone on the internet is interested in posits they now have access to all the features available in order to learn the basics of the format: its structure and how conversion, addition, subtraction, multiplication and division are carried out. Another main use is as a complement to computer science courses, specially in universities. It is important that university courses on computer science incorporate posits (27) either focusing on its structure on the computer arithmetics side, its possible hardware implementation or its uses in neural network training. The application would be a useful tool to help students in understanding the format and, in time, help incorporate it in a introductory level along floating point. 6.4. Comparison with previous tools As RacFP was the biggest reference in the development of this application, it is important to compare the end result with it. A feature comparison is made in Figure 6.1 where it shows that it has some additional important features such as the glossary and a step-by-step explanation of conversion (while they may be considered unnecessary for floats as the process and terminology are much simpler). The sign tables are not present in the application because they were not considered as relevant as the authors for RacFP. Some usability improvements were made taking into account RacFP’s few shortcomings in that regard. The most notable one is that it allows the user to see the whole explanation of the operations at all times unlike RacFP’s explanations and so 6.5. Evaluation with users 37 allows to check previous steps, unlike RacFP. It also allows to manipulate all open windows without needing to close their secondary windows. Features Proposed RacFP Posit calculator Posit lookup tables Well rounded Float converters Explanations of: -Addition ✓ ✓ ✗ ✗ ✗ ✗ -Subtraction ✓ ✓ ✗ ✗ ✗ ✗ -Multiplication ✓ ✓ ✗ ✗ ✗ ✗ -Division ✓ ✓ ✗ ✗ ✗ ✗ -Conversion ✓ ✗ ✗ ✗ ✗ ✓ -Hardware implementation ✗ ✓ ✗ ✗ ✗ ✗ -Structure of the format ✓ ✓ ✗ ✗ ✗ ✗ Glossary ✓ ✗ ✗ ✗ ✗ ✗ Sign tables ✗ ✓ ✗ ✗ ✗ ✗ Multiple format precision comparison ✗ ✗ ✓ ✗ ✗ ✗ Lookup tables ✗ ✗ ✗ ✓ ✗ ✗ Visual representation ✗ ✗ ✗ ✗ ✓ ✗ Multiple languages ✗ ✓ ✗ ✗ ✗ ✗ Table 6.1: Feature comparison of the proposed application with state of the art e-learning resources 6.5. Evaluation with users In order to improve the quality of the application it would be ideal to perform a evaluation of the application with users in order to receive direct feedback. In this section an usability evaluation of the application is presented as a future work possibility. 6.5.1. Investigation objectives and questions As the evaluation of the whole application would result either in very long evaluation sessions or shorter ones with lower quality of the answers the evaluation would have to be separated in multiple tests or sessions. The goal of a first evaluation is to know the following points of the application: 1. Ease of understanding of the binary posit structure. 2. Ease of use of the application itself. In order to evaluate those goals the following investigation questions are set: 1. Is the user capable to convert a decimal number to posit after a session of use? 2. Is the user capable to convert a posit binary number to decimal after a session of use? 3. Does the user get stuck in a specific part of the application? 4. Does the user open the glossary? 5. Does the user manage to go through the explanation of the conversion? 38 Chapter 6. Evaluation of the Application 6.5.2. Environment and duration of the test The tester profile for the application could be Computer Science students that have recently learnt about floating point and without any previous knowledge about posits. The testers would have 20 minutes with the application followed with 5 minute questionnaire and a 5 minute interview after. The environment could be a computer room in a university or school or it could be made remotely (through meet, discord, etc.) 6.5.3. Description of the tester’s tasks The testers have to open the application and experiment with the conversion and the information options of the application without any help from the investigators. 6.5.4. Initial instructions The investigators should welcome the testers and tell them the procedure of the test: they would have 20 minutes to experiment with the application and understand the posit format and then answer a short questionnaire and a short interview. The investigator has to tell the tester that their screen would be recorded as well as the audio of the interview. 6.5.5. Behavior of the investigator Passive observation of the user. The investigator must not intervene in any way. Before starting they would remind the user that the objective of the evaluation is to test the application, not their capacities. Every investigator would be watching a single user and would be taking notes on the observation points. 6.5.6. Evaluation design Observation: the session would be recorded for posterior analysis. The points to observe are if the user opens the glossary and if they go through the conversion explanation at least once. Questionnaire: the questionnaire would have four exercises of conversion that the player has to solve in the given time. The exercises would be comprised of two conversions from decimal to posit and two vice versa. Each one would have an easier and a harder exercise. Interview: the interview would be mainly about the answering the investigation question "Does the user get stuck in a specific part of the application?". 6.5. Evaluation with users 39 It would as well include a question asking about general feedback from the tester about the application in general. Another possible evaluation would be with university professors that could bring in feedback of its educative values from the point of view of a teacher and evaluate where it could be incorporated in university lessons.