Visualizing Classification Results: Confusion Star and Confusion Gear

Received November 25, 2021, accepted December 18, 2021, date of publication December 22, 2021, date of current version January 6, 2022. Digital Object Identifier 10.1109/ACCESS.2021.3137630 Visualizing Classification Results: Confusion Star and Confusion Gear AMALIA LUQUE 1, (Member, IEEE), MIRKO MAZZOLENI 2, (Member, IEEE), ALEJANDRO CARRASCO 3, (Member, IEEE), AND ANTONIO FERRAMOSCA 2, (Member, IEEE) 1Departamento de Ingeniería del Diseño, Escuela Politécnica Superior, Universidad de Sevilla, 41011 Sevilla, Spain 2Department of Management, Information and Production Engineering, University of Bergamo, 24044 Dalmine, Italy 3Departamento de Tecnología Electrónica, School of Computer Engineering, Universidad de Sevilla, 41012 Seville, Spain Corresponding author: Amalia Luque ([email protected]) ABSTRACT Recent developments in machine learning applications are deeply concerned with the poor interpretability of most of these techniques. To gain some insights in the process of designing data-based models it is common to graphically represent the algorithm’s results, either in their final or intermediate stage. Specially challenging is the task of plotting multiclass classification results as they involve categorical variables (classes) rather than numeric results. Using the well-known MNIST dataset and a simple neural network as an example, this paper reviews the existing techniques to visualize classification results, from those centered on a particular instance or set of instances, to those representing an overall performance metric. As classification results are commonly summarized in the form of a confusion matrix, special attention is paid to its graphical representation. From this analysis, a new visualization tool is derived, which is presented in two forms: confusion star and confusion gear. The confusion star is centered on the classification errors, while the confusion gear focuses on the classification hits. The proposed visualization tools are also evaluated when facing: (i) balanced and imbalanced classifiers issues; (ii) the problem of representing errors with different orders of magnitude. By using shapes instead of colors to represent the value of each matrix cell, the new tools significantly improve the readability of the confusion matrices. Furthermore, we show how the area enclosed by the confusion stars and gears are directly related to standard classification metrics. The new graphic tools can be also usefully employed to visualize the performances of a sequence of classifiers. INDEX TERMS Machine learning, classification performance, confusion matrix, data visualization, confusion star, confusion gear. I. INTRODUCTION Machine learning models in general, and deep learning algorithms in particular, are powerful algorithms able to provide very good results when there is a pattern to be learnt from available data, but at the cost of operating as a black-box. On the other hand, having some insights about how they work is a key issue for several reasons: improving the interpretability and explainability of the models [1], debugging and improving architectures and algorithms [2], comparing and selecting results [3], and even for pedagogical purposes [4]. Therefore, a common approach to unveil their functioning relies on some kind of visualization of their inner operation and final results e.g., in the computer vision domain [5]. The associate editor coordinating the review of this manuscript and approving it for publication was Weiping Ding . The main target audience of these tools is the model developer community [6], but also technically skilled model users [7] and even non-experts [8] can benefit of a visual description. These users may be interested in the visual representation of different types of models’ information, such as model architecture [9], neural network’s weights [10], convolutional filters’ values [11], neurons’ activation outputs [12] or edges’ backpropagation gradients [13]. However, by far the most represented information is the model’s predictions either for a particular instance [14], for a group of instances [15] or for the overall dataset [16]. Many methods have been described with the aim of visualizing the prediction process. An up-to-date comprehensive survey of them, structured using the Five W’s and How questions (Why, Who, What, How, When, and Where), can be found in [17]. Also a perspective of visual analytics for VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 1659 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear understanding, diagnosing, and refining models is reviewed in [18]. Additionally, different visualizing tools integrating several approaches have been developed [19]–[23]. Focusing on how to visualize the results predicted by machine learning algorithms, different approaches should be considered depending on the type of problem addressed. The information to be represented (prediction results) is qualitatively different for tasks such as regression, classification, clustering, reinforcement learning, etc. This paper addresses the issue of visualizing the results obtained by multiclass classification algorithms, since this is one of the most frequent tasks in machine learning applications (for instance, around 75% of the datasets in the well-known University of California Irvine Machine Learning Repository [24] contain classification problems). In most cases, the performance of a classifier is summarized by a single metric (accuracy, precision, etc.), but ‘‘it is important to understand both what a classification metric expresses and what it hides’’ [25]. For this reason, a classification metric can also be disaggregated as a set of values with the purpose of gaining better insight into the classifier’s results. As for the level of disaggregation to be used in visualizing classification results, three approaches are considered in the paper: •Low-detailed results, using a single-valued metric for the classification of the whole dataset. •Medium-detailed results, where the classification of the whole dataset is summarized by a small set of values. •High-detailed results, representing classification scores for a single instance or a set of instances in the dataset. Although the paper briefly examines how to represent low and high-detailed classification results, its main focus is on how to visualize them at a medium level of detail, which is commonly described by its multiclass confusion matrix [26]. The main contributions of this research can be summarized as follows: •Two new approaches to visualize the results of a multiclass classifier are proposed, namely the confusion star and confusion gear graphics. •Their use as an intuitive guideline to understand the classification behavior is explored. •Their application to imbalanced datasets is considered. •Their role to compare different classifiers is highlighted, as well as to understand the influence of classifier’s hyperparameters. •The relationships between the shape of these graphs and common classification metrics are derived. The paper is organized as follows. Section II describes the structure of the dataset used in the research, defines the classification scoring procedure and formalizes the concept of confusion matrix. Then, in section III, several techniques to visualize classification scores and multiclass confusion matrices are reviewed. The extension of these ideas is addressed in section IV, where the confusion star and confusion gear concepts are presented. Later, in section V these new tools are discussed, tackling issues such as the impact of imbalanced datasets, the inner and outer areas of the graphics, the use of logarithmic scale and the visualization of evolving classifiers by means of a sequence of the new graphics. Finally, the main findings of the research are presented in the conclusion section. II. METHODOLOGY A. DATASET Throughout this research the MNIST (Modified National Institute of Standards and Technology) dataset [27] has been used as the primary dataset. It contains 70,000 images, each of them representing a handwritten digit (0 to 9). The dataset is split into a 60,000 images subset that is used to train the classifier (training dataset) and a 10,000 images subset employed for generalization purposes (testing dataset). In this case there are 10 classes, one for each digit. This dataset has been widely used as a reference to analyze different classification algorithms. Our goal in this paper is not to obtain a better classifier but, given the results of any of them, to explore how to represent its confusion matrix. As a first example, a classifier implemented as a very simple neural network has been considered, with only an 8-neurons hidden layer and a sigmoid as activation function. The output layer contains 10 nodes (one for each class) with a softmax activation function. Such a network is trained during just 5 epochs, and its generalization results are evaluated on the testing dataset. These test results are used in the following to show different visualization methods. This classifier is advisedly simple for the purpose of obtaining low performance: in this case, differences among the considered visualization techniques can be more easily appreciated. By increasing the number of hidden layers, the number of nodes per layer, and the number of training epochs, much better classification results can be obtained. As instance, using convolutional neural networks, excellent results (99.8% accuracy) have been reported [28]. In the final part of the paper, it is discussed the evolution of the classification performance as a function of the number of training instances. In this case the MNIST dataset has also been used, now raising the number of neurons in the hidden layer up to 128-neurons and training the neural networks during 100 epochs. To show the ability of confusion stars and gears to visualize classification results in problems with a high number of classes, a second dataset, the CIFAR-100, has also been considered [29]. This dataset consists of 60000, 32×32 color images in 100 classes, with 600 images per class, where 50000 images (83%) are used for training and 10000 for test (17%). A 6-layer Convolutional Neural Network (CNN) classifier has been employed, according to the code in [30]. This is not a very powerful classifier as it shows an accuracy of about 40%, while the state of the art classifiers for this problem reach figures over 96% [31]. However this moderate 1660 VOLUME 10, 2022 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear accuracy is quite convenient to depict confusion stars and gears with many classes. Finally, to show the impact of imbalanced datasets on confusion stars and gears, a reduced version of the Abalone dataset is employed. This dataset, available in [24], derives from a non-machine-learning study [32] and contains physical measurements (height, several lengths, diameter, sex) of the abalone mollusk exemplars, along with the number of ‘‘rings’’ present in the shell. The number of rings is proportional to the age of the mollusk. The purpose is to classify each observation in its age class. Certain classes in the original dataset contains very few instances (some classes with one or no elements) making unaffordable any prediction. To overcome this problem a reduced dataset has been obtained by selecting only 10 classes, from class (age) 4 to 13, containing 3670 instances which represents the 88% of the total population. The resulting dataset contains the same number of classes (10) than the MNIST problem but they are highly imbalanced, which is quite convenient for the sake of comparison. A simple multiclass logistic regression has been used as classifier. B. CLASSIFICATION SCORE MATRIX Let us consider a statistical population Pthat contains a set of elements, usually in a large and potentially infinite number. In this population nelements are randomly sampled, obtaining a dataset D={d1,d2,...,dn}, where direpresents the i-th element. Let also be a set of classes 2={θ1, θ2, . . . , θC} where Cis the number of classes, and θjrepresents the j-th class. A certain element d∈Dis defined by a pair h8, θi formed by a vector 8=[ϕ1, ϕ2, . . . , ϕF]that contains the F features that define the element, and the class θto which the element belongs to. Let us call P8= {81,82,...,8n}the set containing the feature vectors 8of the population P. A classifying algorithm Ais defined as a function from the population Pto RC(the set of real numbers of dimension C), which can be expressed as A:P→RC. Therefore, each element belonging to Pis associated to a scoring vector 9= [ψ1, ψ2, . . . , ψC], that is, a score for each class in 2. If the scores can be interpreted as probabilities, the algorithm is a probabilistic classifier. Otherwise, if scores are binary values (0,1) the algorithm is a hard classifier. A decision rule Ris defined as a function which associates a scoring vector 9, defined in RC, in an estimation of the class ˆ θ∈2. Finally, a classifier Cis defined as an ordered pair of functions hA,Riindicating that it first applies the classification algorithm A, and then the decision rule R. So, C:P8 A −→ RCR −→ 2. Considering not the whole dataset but each single element, a classifier can be described as two sequential transformations, 8A −→ 9R −→ ˆ θ, where ˆ θis the class estimate. Therefore, the result obtained applying the classifier Cto an element in the dataset Dis a scoring vector [ψ1, ψ2, . . . , ψC], and a class estimation ˆ θ. To measure the classifier performance, the actual class of the element must also be included. Then, the performance of a classifier, operating on a dataset with ninstances, can be expressed by the score matrix (SM), with SM ∈Rn×(C+2), given by SM ≡       ψ11 ψ12 . . . ψ1Cˆ θ1θ1 ψ21 ψ22 . . . ψ2Cˆ θ2θ2 . . .. . ..... . .. . .. . . ψn1ψn2. . . ψnC ˆ θnθn        .(1) This matrix contains the information about the performance of the classifier at its maximum level of disaggregation. C. CONFUSION MATRIX In many situations, the classifier performance is analyzed not considering the scores associated to each class, but just comparing the estimated and the actual class for each instance in the dataset. So, by discarding the first Ccolumns of the score matrix, the more compact estimation matrix (EM) is obtained EM ≡       ˆ θ1θ1 ˆ θ2θ2 . . .. . . ˆ θnθn        .(2) The estimation matrix EM is has a smaller dimension (less columns) than the score matrix SM, but it still has a high level of disaggregation, since it contains information for each instance in the dataset. Therefore, it is common to summarize it using the confusion matrix (CM) defined as CM ≡      m11 m12 . . . m1C m21 m22 . . . m2C . . .. . ..... . . mC1mC2. . . mCC       ,(3) where mij represents the number of instances of class θi estimated by the classifier as belonging to the class ˆ θj. The results obtained classifying the MNIST dataset with the neural network previously described can be summarized in the confusion matrix shown in TABLE 1. Last column also shows that there are a similar number of instances in each class, which means that classes are quite balanced. In the definition of the confusion matrix, is usual to describe mij as a fraction of the total number of instances mi belonging to the class θi. By calling this ratio λij ≡mij/mi, then mij can be expressed as mij =λij ·mi, and the confusion matrix can be rewritten as CM =      λ11m1λ12m1. . . λ1Cm1 λ21m2λ22m2. . . λ2Cm2 . . .. . ..... . . λC1mCλC2mC. . . λCC mC       =3◦M. (4) VOLUME 10, 2022 1661 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear TABLE 1. Confusion matrix obtained classifying the MNIST dataset with a very simple neural network. TABLE 2. Confusion matrix for a multiclass classification. The symbol ◦represents the element-wise multiplication (also called Hadamard product), 3is the unit confusion matrix expressed by, 3≡      λ11 λ12 . . . λ1C λ21 λ22 . . . λ2C . . .. . ..... . . λC1λC2. . . λCC       ,(5) and Mis the matrix defined by M≡      m1m1. . . m1 m2m2. . . m2 . . .. . ..... . . mCmC. . . mC       .(6) TABLE 2 summarizes the main elements considered in the definition of the confusion matrix, where gjis the number of instances estimated as belonging to the j-th class. III. VISUALIZING CLASSIFICATION RESULTS A. CLASSIFICATION SCORES OF INSTANCES Fully detailed classification results regarding the i-th instance correspond to the i-th row of the score matrix (1) defined by SMi≡ψi1ψi2. . . ψiC ˆ θiθi.(7) The classification scores for the first three instances in the MNIST testing dataset can be depicted as in Fig. 1 (in colors blue, orange and green respectively). FIGURE 1. Classification scores for the first three instances in the MNIST testing dataset. (Filled dots) correctly classified instances. (Empty dots) not correctly classified instances. In this classifier the scores are generated by the softmax activation function of the 10-neurons output layer, so they are in the range [0,1], sum up to 1 and, therefore, they can be interpreted as probabilities. For example, the first instance (represented by a blue line) has a (0.05,0.07,0.12, . . .)probability of belonging to the class (0,1,2, . . .). Belonging to class 7 obtains the highest probability (0.34), so this is the class estimated by the classifier. In this case, the instance is classified correctly, which is indicated using filled dots. For the second instance (represented by an orange line), belonging to class 0 obtains the highest probability (0.26). In this case, this is an error as the actual class is 2 which is indicated using empty dots. 1662 VOLUME 10, 2022 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear This type of representation only has meaning for a single instance or for a very reduced number of them. In order to depict classification scores for many instances a scatter polar plot has been proposed [33] as it is shown in Fig. 2 for every instance in the MNIST testing dataset. FIGURE 2. Polar representation of the classification scores for the MNIST testing dataset. Each class is represented by a certain angle ϕj, which for the j-th class is defined by ϕj=2π(j−1) C.(8) The classification result for the i-th instance is depicted as a dot in a position defined by its vector Eri= C X j=1 Erij = C X j=1 ψij6ϕj,(9) where Erij is a vector of module ψij and phase ϕj. B. CLASSIFICATION SCORES OF CLASSES A partial perspective of the score matrix may consider not every instance in the dataset, but only those belonging to a certain class. In this case, the scoring results of the instances belonging to the j-th class are defined by a slice of the score matrix SM(j)≡{SMi},∀i|θi=j. As the level of disaggregation of this matrix is still very high, it is commonly summarized using some statistics for each column (mean score value, standard deviation, density function, etc.). Fig. 3 depicts one of these summaries, here in the form of a boxplot. The i-th subplot considers the miinstances belonging to the i-th class and the j-th box indicates the distribution of the values ψij,∀i|di∈θj, that is the scores of elements of the i-th class that are being estimated as belonging to the j-th class. The instances belonging to class 1, for example, are estimated as belonging to class 1 with a probability distributed as it is shown in the second box of the second plot, clearly outperforming the remaining probability distributions. Then, very good classification results should be expected for instances belonging to class 1. Conversely, instances belonging to class 2 (third plot) have a probability of being correctly classified as it is shown in the third box. This distribution is only slightly better than the ones corresponding to the estimated classes 1 and 7, so many classification errors should be expected for instances belonging to class 2. C. REPRESENTATION OF THE CONFUSION MATRIX Let us now focus on how to represent the classification results using a medium level of detail, that is, based on its confusion matrix. The most common way to depict a certain multiclass confusion matrix is straightforwardly drawing it as a C×C colored grid where each cell has a color scaled according to its value. Sometimes the cell also contains a text with its numeric value, as it shown in Fig. 4. In case of an imbalanced dataset, it is better to represent the unit confusion matrix, commonly expressed by the percentage values, as it is depicted in Fig. 5. The confusion matrix or the unit confusion matrix can be alternatively represented as in Fig. 6 where, for each actual class, a set of Cstacked bars are drawn. The height of each bar in a certain stack (actual class) is proportional to the number (or ratio) of instances estimated as belonging to each class, that is, corresponding to the values of a row in the confusion matrix. A similar stacked bar approach is used in [34]. D. REPRESENTATION OF BINARY CONFUSION MATRICES Sometimes it is worth to assess the classification results of one class versus all the remaining ones (OvA binary classification). So, let us consider the instances belonging to the i-th class which will be denoted as the ‘‘positive’’ (P) class. The remaining instances belong to different classes which will be collectively denoted as the ‘‘negative’’ (N) class. In this way, the number of instances correctly classified as positives (TP: True Positives) is TP =mii. Similarly, the number of instances erroneously classified as positives (FN: False Negatives) is FN =mi−mii =    C X k=1 k6=i mik     −mii.(10) The number of elements not belonging to the i-th class (that is, belonging to the negative class) which are erroneously classified (FP: False Positives) is FP =    C X k=1 k6=i mki    −mii.(11) Finally, the number of elements not belonging to the i-th class (that is, belonging to the negative class) which are VOLUME 10, 2022 1663 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear FIGURE 3. Statistical distribution of the classification scores (probabilities) for the instances in the MNIST testing dataset belonging to a certain class. FIGURE 4. Straightforward colored grid representation of the confusion matrix corresponding to the classification of the MNIST testing dataset. correctly classified (TN: True Negatives) is TN =mN−FP =m− C X k=1 k6=i mki.(12) Considering these results, the binary matrices corresponding to every class can be represented as it is shown in Fig. 7. Alternatively, they can be represented using stacked bar plots, as it is depicted in Fig. 8. Binary classification results can also be analyzed using the receiver operating characteristic (ROC) curve [35]. Converting the classification scores for an instance into its estimated class requires a decision rule Rwhich, in the binary case, is usually a threshold τ. If the score of belonging to the positive class ψiP is greater than the threshold, the instance is estimated as positive; otherwise as negative. So, the elements of the binary confusion matrix depend on τ, and also their related metrics. Specifically, the True Positive Rate (TPR) and FIGURE 5. Straightforward colored grid representation of the unit confusion matrix corresponding to the classification of the MNIST testing dataset. FIGURE 6. Stacked bar representation of the unit confusion matrix corresponding to the classification of the MNIST testing dataset. the False Positive Rate (FPR) are defined as TPR (τ)≡TP (τ) mP ;FPR (τ)≡1−TN (τ) mN .(13) 1664 VOLUME 10, 2022 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear FIGURE 7. Straightforward representation of the unit binary confusion matrices corresponding to the classification of the MNIST testing dataset. FIGURE 8. Stacked bar representation of the unit binary confusion matrices corresponding to the classification of the MNIST testing dataset. FIGURE 9. ROC curves corresponding to the OvA binary classification of the MNIST testing dataset. The ROC is built as a parametric curve in τ, with FPR (τ) in the horizontal and TPR (τ)in the vertical axis. The resulting ROC curves for the 10 binary classifiers are depicted in Fig. 9. E. ALTERNATIVE REPRESENTIONS OF THE CONFUSION MATRIX Some authors have proposed alternative representation for the confusion matrix such as, for instance, in [14] where a chord FIGURE 10. Chord diagram representation of the confusion matrix corresponding to the classification of the MNIST testing dataset. diagram, called by the authors confusion wheel, is used. This plot is depicted in Fig. 10 where each class corresponds to a circular sector with a size proportional to the number of instances belonging to that class. Later a chord is drawn starting at the actual class sector and ending at the estimated class sector. The width of the chord at each side is proportional to the number of instances belonging to that class classified as belonging to the other side’s class. The color of the chord is that of its widest side. VOLUME 10, 2022 1665 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear FIGURE 11. Sankey diagram representation of the confusion matrix corresponding to the classification of the MNIST testing dataset. FIGURE 12. MDS transformation of the confusion matrix corresponding to the classification of the MNIST testing dataset. Also in [36] it is proposed to represent the confusion matrix using a Sankey diagram as in Fig. 11. In the upper part each class (origin) is represented by a rectangle with a width proportional to the number of instances belonging to that class. In the lower part, the estimated classes (destinations) are drawn with a width proportional to the number of instances predicted as belonging to that class. The ribbons drawn in the middle represent the instance belonging to the upper side class but classified as belonging to the lower side class. In [37] the confusion matrix is conceived as a similarity matrix between classes. Then, it is transformed in its opposite, that is, a dissimilarity or distance matrix. Finally this matrix is represented in a two-dimensional plane using the multidimensional scaling (MDS) technique. The result is shown in Fig. 12 where each class is represented by a point in the new 2D plane. The closer a pair of classes, the more similar they are and, therefore, the more difficult is to separate them. For example, classes 4 and 9 are very close in the MDS plane, which means that it is very difficult to separate them and so a high number of classification errors should be expected. Along with the individual representations described above, it is also common to find visual representations of the classifier results that combine several of the preceding graphs. FIGURE 13. Linear representation of the confusion matrix corresponding to the classification of the MNIST testing dataset. Although these more sophisticated graphics may seem visually very appealing, they do not necessarily provide additional information compared to the more conventional representations. Therefore, in the following section, a new graphical representation is proposed. IV. BEYOND CONFUSION MATRIX A. LINEAR REPRESENTATION OF THE CONFUSION MATRIX Let us consider the miinstances belonging to the i-th class. The results of their classification are summarized in the i-th row of the confusion matrix, CMi≡mi1mi2. . . miC , where mij represents the number of instances belonging to the i-th class, estimated as belonging to the j-th class. Then, it is possible to represent the confusion matrix as a sequence of Clines, each of them corresponding to a row CMi. Every line is defined by Cvalues, corresponding to each mij elements. The result is depicted in Fig. 13. A similar approach is used in [38]. In a good classifier most instances are correctly estimated as belonging to its actual class, so mii ≈mi;mij ≈0,∀j6= i. That is, a single very high value escorted by the remaining very low values. This important imbalance in the values of each row is clearly seen in the plot and it makes difficult its 1666 VOLUME 10, 2022 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear interpretation. This is also the reason why such graphic is not commonly used to represent the confusion matrix. To overcome the issues raised in the previous representation, the CMicontaining the classification results corresponding to the i-th class is transformed into a new vector EMi≡ei1ei2. . . eiC , where its elements are defined as eii =mi−mii and eij =mij,∀j6= i. Then, for a perfect classification, eij =0,∀j. The matrix EM = {eij} is denominated the error matrix. The i-th row of this matrix can also be formulated in terms of the ratio over the total number of instances belonging to the i-th class, EMi=i1mii2mi. . . iC mi, where the ratio ij =eij/mi. The matrix E= {ij}is denominated the unit error matrix and can be represented as a sequence of Clines, as it is shown in Fig. 14. FIGURE 14. Linear representation of the unit error matrix corresponding to the classification of the MNIST testing dataset. In the previous linear representation (Fig. 14) an abnormally high value is observed in each line, corresponding to the element eii, that is, the number of instances belonging to the i-th class erroneously classified as belonging to any other class. To explain these peaks let us first remind that the Celements of the i-th row in the confusion matrix are mutually dependent having C−1 degrees of freedom, that is, they obey the equation C X j=1 mij =mii +X j6=i mij =mi.(14) Then, the number of hits (correct classifications) for the i-th class is mii =mi−X j6=i mij.(15) Recalling the definition of the elements of the error matrix, its diagonal elements can be written as eii =mi−mii =X j6=i mij =X j6=i eij (16) FIGURE 15. Boxplot containing the distribution of the values for eij ,∀j6= i. The mean value ¯ eij (in blue) and the eii (in green) are also depicted. All the error values are expressed in %. The term eij counts the number of instances belonging to the i-th class, erroneously classified as belonging to the j-th class. Calling ¯eij its mean value ,∀j6= i, it can be written that eii =(C−1)· ¯eij. Then, in the MNIST example (with C=10), the value of eii will be 9 times higher than the mean of the remaining eij. This is the reason why a peak appears in the linear representation of Fig. 14. The distribution of the classification errors for each class is depicted in Fig. 15, where it is clearly shown that the value of eii (in green) is much higher (9 times) than the value of ¯eij (in blue). Considering the C−1 degrees of freedom in the rows of the error matrix, any of them can be omitted without losing information. Then, removing the element eii is a convenient decision as it eliminates the peaks in the plot, as it is depicted in Fig. 16. FIGURE 16. Linear representation without redundancies of the unit error matrix corresponding to the classification of the MNIST testing dataset. It must be noted that the horizontal axis does not indicate the estimated class but an index to this class once the redundant element has been removed, that is, the value VOLUME 10, 2022 1667 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear FIGURE 31. Confusion stars (in logarithmic scale) corresponding to a pair of points before and after the first jump in the learning curve: 502 instances (left; accuracy of 38.13%) and 610 instances (right; accuracy of 67.23%). FIGURE 32. Sequence of logarithmic confusion stars corresponding to several point of the learning curve. The application of the confusion stars to compare two points of the learning curve can be extended to a sequence of points, drawing a grid of stars as it is shown in Fig. 32. In that graphic, which resembles the concept of as small multiple [42], can be seen that, for example, the problems classifying instances of classes 4, 8 and 9 that shows the classifiers trained with up to 1000 training instances, are mostly solved once the 3000 instances barrier is overcome. From this point on, a smooth and continuous improvement of the classification results is obtained. The same information can be obtained analyzing the corresponding confusion gears. Representing a sequence of confusion matrices by a grid of stars has an obvious limitation of space: the more matrices to be represented, the smaller is the size of each star. To tackle this problem, the sequence of confusion matrices can be represented generating a movie where each frame corresponds to a single confusion matrix. An example of this video can be seen in the online version of the paper (see also appendix). In Fig. 33, an example of a frame of the movie is shown. F. SUMMARY OF VISUALIZATION METHODS Through the paper, up to 13 methods for visualizing classification performance have been described. Some of them focus on classification scores of single instances while others are interested on how the classifiers behave for the instances of certain classes. On the other hand, some visualization 1674 VOLUME 10, 2022 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear TABLE 4. Summary of visualization methods. FIGURE 33. Frame of the movie generated to visualize the learning process: learning curve drawn up to 1000 training instances (right), and its corresponding confusion star in logarithmic scale (left). methods are designed primarily for two classes (binary classification) while others can represent multiple classes. Visualization methods can be featured by how they represent the different classes (actual or estimated) and the classification performance. Some of them use color to convey the required information while others use geometric elements for this purpose: X and/or Y axis position in rectangular plots, radial and/or angular position in polar plots, length and/or width of graphical elements, etc. A summary of the visualization methods described in the paper is shown in TABLE 4. VI. CONCLUSION This paper has reviewed several methods to visualize classification results at different levels of detail: from those centered on how a particular instance or set of instances are classified, to those that summarize the classification performance in a single metric. A particular interest has been devoted to classification results which are summarized in the form of a confusion matrix, presenting the main procedures to visualize it from the straightforward row-column matrix representation, with colors indicating the value of each matrix cell, to more complex and sophisticated graphics. From this analysis, a new way of representing the information conveyed by confusion matrices is proposed in the form of a confusion star (focusing on the errors) or a confusion gear (centered on the hits). The new visualization tool can be employed to represent the original and possibly imbalanced confusion matrix, or the balanced unit version of that matrix. VOLUME 10, 2022 1675 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear The new tool successfully represents multiclass classification results in the form of a radial plot. The traditional way to represent confusion matrix uses colors (and eventually texts) to indicate the number of instances belonging to an actual class that are classified to an estimated class. Instead, confusion stars and gears use shapes to convey that information. Changing colors by shapes significantly improves the readability of the proposed graphics. An additional property of the confusion stars and gears is that the enclosed area provides information about the overall classification performance. The relation of these areas to standard classification metrics has also been derived. Finally, it has also been shown that the new graphic tools can usefully be employed to visualize the performances of a sequence of classifiers. APPENDIX Supplementary materials can be found in the on line version of the paper or they and can also be downloaded from https://github.com/amalialuque/confusionstar. They contain: 1) Three Excel files with the confusion matrices described in Section II.A. 2) An Excel file with the sequence of confusion matrices described in Section V.E. 3) A video file (in Graphics Interchange Format, GIF, format) visualizing the learning process described in Section V.E. 4) A Jupyter notebook, providing an implementation of the functions required to plot a confusion matrix as a confusion star (or confusion gears); and to generate a video file visualizing a sequence of confusion matrices in the form of confusion stars (or confusion gears). Additionally, the algorithm that converts a confusion matrix into a confusion star plot can be found as supplementary material to the paper. REFERENCES [1] F.-Y. Tzeng and K.-L. Ma, ‘‘Opening the black box–data driven visualization of neural networks,’’ in Proc. IEEE Vis., Jan. 2005, pp. 383–390, doi: 10.1109/visual.2005.1532820. [2] K. Wongsuphasawat, D. Smilkov, J. Wexler, J. Wilson, D. Mané, D. Fritz, D. Krishnan, F. B. Viégas, and M. Wattenberg, ‘‘Visualizing dataflow graphs of deep learning models in tensorflow,’’ IEEE Trans. Vis. Comput. Graphics, vol. 24, no. 1, pp. 1–12, Jan. 2018, doi: 10.1109/TVCG.2017.2744878. [3] Y. Ming, S. Cao, R. Zhang, Z. Li, Y. Chen, Y. Song, and H. Qu, ‘‘Understanding hidden memories of recurrent neural networks,’’ in Proc. IEEE Conf. Vis. Anal. Sci. Technol. (VAST), Oct. 2017, pp. 13–24, doi: 10.1109/VAST.2017.8585721. [4] A. W. Harley, ‘‘An interactive node-link visualization of convolutional neural networks,’’ in Proc. Int. Symp. Vis. Comput., vol. 9474, 2015, pp. 867–877, doi: 10.1007/978-3-319-27857-5_77. [5] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, ‘‘Grad-CAM: Visual explanations from deep networks via gradient-based localization,’’ in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 618–626, doi: 10.1109/ICCV.2017.74. [6] J. Wang, L. Gou, H. Yang, and H.-W. Shen, ‘‘GANViz: A visual analytics approach to understand the adversarial game,’’ IEEE Trans. Vis. Comput. Graph., vol. 24, no. 6, pp. 1905–1917, Jun. 2018, doi: 10.1109/TVCG.2018.2816223. [7] D. Smilkov, N. Thorat, C. Nicholson, E. Reif, and M. Wattenberg, ‘‘Embedding projector: Interactive visualization and interpretation of embeddings,’’ in Proc. Neural Inf. Process. Syst., 2016, pp. 1–4. [8] D. Chen, R. K. E. Bellamy, P. K. Malkin, and T. Erickson, ‘‘Diagnostic visualization for non-expert machine learning practitioners: A design study,’’ in Proc. IEEE Symp. Vis. Lang. Hum.-Centric Comput., Dec. 2016, pp. 87–95, doi: 10.1109/VLHCC.2016.7739669. [9] A. Bauerle, C. van Onzenoodt, and T. Ropinski, ‘‘Net2Vis—A visual grammar for automatically generating publication-tailored CNN architecture visualizations,’’ IEEE Trans. Vis. Comput. Graphics, vol. 27, no. 6, pp. 2980–2991, Jun. 2021, doi: 10.1109/TVCG.2021.3057483. [10] D. Smilkov, S. Carter, D. Sculley, and M. Wattenberg, ‘‘Directmanipulation visualization of deep networks,’’ in Proc. ICML, 2016, pp. 1–5. [11] M. D. Zeiler and R. Fergus, ‘‘Visualizing and understanding convolutional networks,’’ in Proc. Eur. Conf. Comput. Vis., vol. 8689, 2014, pp. 818–833, doi: 10.1007/978-3-319-10590-1_53. [12] J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson, ‘‘Understanding neural networks through deep visualization,’’ in Proc. ICML, 2015, pp. 1–12. [13] D. Cashman, G. Patterson, A. Mosca, N. Watts, S. Robinson, and R. Chang, ‘‘RNNbow: Visualizing learning via backpropagation gradients in RNNs,’’ IEEE Comput. Graph. Appl., vol. 38, no. 6, pp. 39–50, Nov. 2018, doi: 10.1109/MCG.2018.2878902. [14] B. Alsallakh, A. Hanbury, H. Hauser, S. Miksch, and A. Rauber, ‘‘Visual methods for analyzing probabilistic classification data,’’ IEEE Trans. Vis. Comput. Graphics, vol. 20, no. 12, pp. 1703–1712, Dec. 2014, doi: 10.1109/TVCG.2014.2346660. [15] H. Strobelt, S. Gehrmann, H. Pfister, and A. M. Rush, ‘‘LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks,’’ IEEE Trans. Vis. Comput. Graphics, vol. 24, no. 1, pp. 667–676, Jan. 2018, doi: 10.1109/TVCG.2017.2744158. [16] N. Pezzotti, T. Höllt, J. Van Gemert, B. P. Lelieveldt, E. Eisemann, and A. Vilanova, ‘‘DeepEyes: Progressive visual analytics for designing deep neural networks,’’ IEEE Trans. Vis. Comput. Graphics, vol. 24, no. 1, pp. 98–108, Jan. 2018, doi: 10.1109/TVCG.2017.2744358. [17] F. Hohman, M. Kahng, R. Pienta, and D. H. Chau, ‘‘Visual analytics in deep learning: An interrogative survey for the next frontiers,’’ IEEE Trans. Vis. Comput. Graphics, vol. 25, no. 8, pp. 2674–2693, Aug. 2019, doi: 10.1109/TVCG.2018.2843369. [18] S. Liu, X. Wang, M. Liu, and J. Zhu, ‘‘Towards better analysis of machine learning models: A visual analytics perspective,’’ Vis. Inf., vol. 1, no. 1, pp. 48–56, Mar. 2017, doi: 10.1016/j.visinf.2017.01.006. [19] M. Kahng, P. Y. Andrews, A. Kalro, and D. H. Chau, ‘‘ActiVis: Visual exploration of industry-scale deep neural network models,’’ IEEE Trans. Vis. Comput. Graphics, vol. 24, no. 1, pp. 88–97, Jan. 2018, doi: 10.1109/TVCG.2017.2744718. [20] D. Ren, S. Amershi, B. Lee, J. Suh, and J. D. Williams, ‘‘Squares: Supporting interactive performance analysis for multiclass classifiers,’’ IEEE Trans. Vis. Comput. Graphics, vol. 23, no. 1, pp. 61–70, Jan. 2017, doi: 10.1109/TVCG.2016.2598828. [21] A. P. Norton and Y. Qi, ‘‘Adversarial-playground: A visualization suite showing how adversarial examples fool deep learning,’’ in Proc. IEEE Symp. Vis. Cyber. Secur., May 2017, pp. 1–4, doi: 10.1109/VIZSEC.2017.8062202. [22] R. E. Curtis, A. Yuen, L. Song, A. Goyal, and E. P. Xing, ‘‘TVNViewer: An interactive visualization tool for exploring networks that change over time or space,’’ Bioinformatics, vol. 27, no. 13, pp. 1880–1881, Jul. 2011, doi: 10.1093/bioinformatics/btr273. [23] J. Talbot, B. Lee, A. Kapoor, and D. S. Tan, ‘‘EnsembleMatrix: Interactive visualization to support machine learning with multiple classifiers,’’ in Proc. Conf. Hum. Factors Comput. Syst., 2009, pp. 1283–1292, doi: 10.1145/1518701.1518895. [24] D. Dua and C. Graff. (2019). UCI Machine Learning Repository. Accessed: May 31, 2021. [Online]. Available: https://archive.ics.uci.edu/ml [25] J. Lever, M. Krzywinski, and N. Altman, ‘‘Classification evaluation,’’ Nature Methods, vol. 13, no. 8, pp. 603–604, 2016, doi: 10.1038/nmeth.3945. [26] A. Luque, A. Carrasco, A. Martín, and A. de las Heras, ‘‘The impact of class imbalance in classification performance metrics based on the binary confusion matrix,’’ Pattern Recognit., vol. 91, pp. 216–231, Oct. 2019, doi: 10.1016/j.patcog.2019.02.023. 1676 VOLUME 10, 2022 A. Luque et al.: Visualizing Classification Results: Confusion Star and Confusion Gear [27] L. Deng, ‘‘The MNIST database of handwritten digit images for machine learning research,’’ IEEE Signal Process. Mag., vol. 29, no. 6, pp. 141–142, Nov. 2012, doi: 10.1109/MSP.2012.2211477. [28] J. Hu, L. Shen, S. Albanie, G. Sun, and E. Wu, ‘‘Squeeze-and-excitation networks,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 8, pp. 2011–2023, Apr. 2020, doi: 10.1109/TPAMI.2019.2913372. [29] A. Krizhevsky and G. Hinton, ‘‘Learning multiple layers of features from tiny images,’’ Univ. Toronto, Toronto, ON, Canada, Tech. Rep., 2009, pp. 1–58. [30] P. Madhusudhan. (2018). CIFAR100 Small Image Classification Keras Dataset. Accessed: Nov. 18, 2021. [Online]. Available: https://phani1995. github.io/artificial/neural/network/2018/10/31/CIFAR100_small_imae _classification_keras_dataset.html [31] P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur, ‘‘Sharpness-aware minimization for efficiently improving generalization,’’ in Proc. Int. Conf. Learn. Represent., 2020, pp. 1–20. [32] W. J. Nash, T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford, ‘‘The population biology of abalone (Haliotis species) in Tasmania. I. Blacklip abalone (H. rubra) from the north coast and the Furneaux group of islands of bass strait,’’ Sea Fish. Div., Chennai, India, Tech. Rep. 48, 1994. [33] C. Seifert and E. Lex, ‘‘A novel visualization approach for data-mining- related classification,’’ in Proc. Int. Conf. Inf. Vis., 2009, pp. 490–495, doi: 10.1109/IV.2009.45. [34] E. Beauxis-Aussalet and L. Hardman, ‘‘Visualization of confusion matrix for non-expert users,’’ in Proc. Conf. Visual Anal. Sci. Technol. (VAST), 2014, pp. 1–7. [35] T. Fawcett, ‘‘An introduction to ROC analysis,’’ Pattern Recognit. Lett., vol. 27, no. 8, pp. 861–874, Jun. 2006, doi: 10.1016/j.patrec.2005.10.010. [36] M. Páhringer, A. Hinterreiter, and M. Streit, ‘‘InstanceFlow: Visualizing the evolution of classifier confusion on the instance level,’’ in Proc. IEEE Visualizat. Conf., May 2020, 2020, pp. 291–295, doi: 10.1109/VIS47514.2020.00065. [37] R. Susmaga, ‘‘Confusion matrix visualization,’’ in Intelligent Information Processing and Web Mining. Berlin, Germany: Springer, 2004, pp. 107–116. [38] Y. Zhou, T. Wischgoll, L. M. Blaha, R. Smith, and R. J. Vickery, ‘‘Visualizing confusion matrices for multidimensional signal detection correlational methods,’’ Proc. SPIE Vis. Data Anal., vol. 9017, no. 3, May 2014, Art. no. 901709, doi: 10.1117/12.2042610. [39] B. Diri and S. Albayrak, ‘‘Visualization and analysis of classifiers performance in multi-class medical data,’’ Expert Syst. Appl., vol. 34, no. 1, pp. 628–634, Jan. 2008, doi: 10.1016/j.eswa.2006.10.016. [40] S. Dutta, T. L. Turton, and J. P. Ahrens, ‘‘A confidence-guided technique for tracking time-varying features,’’ Comput. Sci. Eng., vol. 23, no. 2, pp. 84–92, Mar. 2021, doi: 10.1109/MCSE.2020.3047979. [41] A. Hinterreiter, P. Ruch, H. Stitz, M. Ennemoser, J. Bernard, H. Strobelt, and M. Streit, ‘‘ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion,’’ IEEE Trans. Vis. Comput. Graph., early access, Jul. 27, 2020, doi: 10.1109/tvcg.2020.3012063. [42] E. Tufte, N. Goeler, and R. Benson, Envisioning Information, vol. 2. Cheshire, CT, USA: Graphics Press, 1990. AMALIA LUQUE (Member, IEEE) received the bachelor’s degree in industrial engineering, in 2007, the master’s degree in automation, robotics and telematics, in 2010, and the Ph.D. degree in industrial engineering, in 2014. Since 2015, she has been involved in teaching related to project engineering at the University of Seville, where she is currently a Professor. Her main research interests include control, business intelligence, data mining, feature extraction, and artificial intelligence. MIRKO MAZZOLENI (Member, IEEE) was born in Ponte San Pietro, Italy, in 1989. He received the M.Sc. degree (summa cum laude) in computer engineering and the Ph.D. degree in engineering and applied sciences (control systems) from the University of Bergamo, Italy, in 2014 and 2018, respectively. He is currently an Assistant Professor with the University of Bergamo. His main research interests include system identification, machine learning, and fault diagnosis. ALEJANDRO CARRASCO (Member, IEEE) received the master’s and Ph.D. degrees in computer engineering from the University of Seville, Spain, in 1998 and 2003, respectively. Since 1997, he has worked for several companies in the area of software engineering and computer networks. He is currently a Lecturer and a Researcher at the Department of Electronic Technology, University of Seville. He has founded a new technology-based company (NTBF) and actively participated and directed several research and development projects. His research interests include data mining, distributed services and artificial intelligence applied to industrial computing, and cybersecurity. ANTONIO FERRAMOSCA (Member, IEEE) was born in Maglie, Italy, in 1982. He received the bachelor’s degree in computer science engineering from the University of Pavia, Pavia, Italy, in 2006, and the Ph.D. degree (summa cum laude) in engineering from the University of Seville, Seville, Spain, in 2011. He was a Visiting Researcher with the University of Wisconsin–Madison, Madison, WI, USA, in 2009, and the Institute of Technological Development for the Chemical Industry, Santa Fe, Argentina, in 2010, and the University of Seville, in 2015. He is currently an Assistant Professor with the Department of Management, Information and Production Engineering, University of Bergamo, Italy. He has authored and coauthored more than 90 publications, including book chapters, journal articles, conference papers, and industrial deliverables. His research interests include model predictive control, nonlinear systems, impulsive systems, distributed control, stability, control of biological systems, and robust control. VOLUME 10, 2022 1677