IVTURS: a linguistic fuzzy rule-based classification system based on a new Interval-Valued fuzzy reasoning method with TUning and Rule Selection

1 IVTURS: a linguistic fuzzy rule-based classification system based on a new Interval-Valued fuzzy reasoning method with TUning and Rule Selection Jos´ e Antonio Sanz, Alberto Fern´ andez, Humberto Bustince, IEEE Member, and Francisco Herrera, IEEE Member Abstract—Interval-valued fuzzy sets have been shown to be a useful tool for dealing with the ignorance related to the definition of the linguistic labels. Specifically, they have been successfully applied to solve classification problems, performing simple modifications on the fuzzy reasoning method to work with this representation and making the classification based on a single number. In this paper we present IVTURS, a new linguistic fuzzy rulebased classification method based on a new completely intervalvalued fuzzy reasoning method. This inference process uses interval-valued restricted equivalence functions to increase the relevance of the rules in which the equivalence of the interval membership degrees of the patterns and the ideal membership degrees is greater, which is a desirable behaviour. Furthermore, their parametrized construction allows the computation of the optimal function for each variable to be performed, which could involve a potential improvement in the system’s behaviour. Additionally, we combine this tuning of the equivalence with rule selection in order to decrease the complexity of the system. In this paper we name our method IVTURS-FARC, since we use the FARC-HD method [1] to accomplish the fuzzy rule learning process. The experimental study is developed in three steps in order to ascertain the quality of our new proposal. First, we determine both the essential role that interval-valued fuzzy sets play in the method and the need for the rule selection process. Next, we show the improvements achieved by IVTURS-FARC with respect to the tuning of the degree of ignorance when it is applied in both an isolated way and when combined with the tuning of the equivalence. Finally, the significance of IVTURS-FARC is further depicted by means of a comparison by which it is proved to outperform the results of FARC-HD and FURIA [2], which are two high performing fuzzy classification algorithms. Index Terms—Linguistic Fuzzy Rule-Based Classification Systems, Interval-Valued Fuzzy Sets, Fuzzy Reasoning Method, Interval-Valued Restricted Equivalence Functions, Tuning, Rule Selection. I. INTRODUCTION FUzzy Rule-Based Classification Systems (FRBCSs) have been widely employed in the field of pattern recognition This work was supported in part by the Spanish Ministry of Science and Technology under projects TIN2011-28488 and TIN2010-15055 and the Andalusian Research Plan P10-TIC-6858 and P11-TIC-7765. J. Sanz and H. Bustince are with the Departmento of Autom´ atica y Computaci´ on, Universidad Publica de Navarra, Navarra, 31006 Spain e-mails: {joseantonio.sanz,bustince}@unavarra.es A. Fern´ andez is with the Department of Computer Science, University of Ja´ en, Ja´ en, 23071 Spain e-mail: [email protected] F. Herrera is with the Department of Computer Science and Artificial Intelligence, CITIC-UGR (Research Center on Information and Communications Technology), University of Granada, Granada, 18071 Spain e-mail: [email protected].es and classification problems [1]–[8]. Aside from their good performance, FRBCSs are adequate since they also provide a linguistic model interpretable to the users because they are composed of a set of rules composed of linguistic terms [9], [10]. One of the key points in the subsequent success of fuzzy systems (like FRBCSs) is the choice of the membership functions [11]. This is a complex problem due to the uncertainty related to their definition, whose source can be both the intrapersonal and the interpersonal uncertainty associated with the linguistic terms [12], [13]. Interval-Valued Fuzzy Sets (IVFSs) [14] have proven to be an appropriate tool to model the system uncertainties and the ignorance in the definition of the fuzzy terms [15]. An IVFS provides an interval, instead of a single number, as the membership degree of each element to this set. The length of the interval can be seen as a representation of the ignorance related to the assignment of a single number as membership degree [16]. IVFSs have been successfully applied in computing with words [17], mobile robots [18] and image processing [19], [20], among others. In this paper we present IVTURS, which is short for linguistic FRBCS based on an Interval-Valued fuzzy reasoning method (IV-FRM) with TUning and Rule Selection. The main contribution of IVTURS is a novel IV-FRM in which the ignorance represented by the IVFSs is taken into account throughout the reasoning process. To do so, we completely extend the classical fuzzy reasoning method [21] including the computation of the matching degree using Interval-Valued Restricted Equivalence Functions (IV-REFs) [22], [23]. The goal is to show how equivalent are the interval membership degrees of the antecedent of the rules to the ideal interval membership degree ([1,1]). In a nutshell, the higher the equivalence between the example and the antecedent the greater the significance of the rule in the decision process is. IV-REFs are constructed using parametrized functions. It is therefore easy to construct different functions by modifying the values of their parameters. This fact makes it possible to compute the most suitable set of IV-REFs for each specific problem by defining a genetic tuning to accomplish this optimization problem. Additionally, we combine it with a fuzzy rule selection process as it is a well known synergy in this field to improve both the interpretability and the accuracy of the final fuzzy system [10]. IVTURS is composed of three stages: 1) The generation of an initial Interval-Valued Fuzzy Rule-Based Classification 2 System (IV-FRBCS). To do this, we firstly learn the rule base using the recent fuzzy rule learning algorithm known as FARC-HD [1] (Fuzzy Association Rule-based Classification model for High Dimensional problems). Then, we model its linguistic labels with IVFSs and we initialize the IV-REF for each variable of the problem; 2) The application of the new IVFRM which makes use of IV-REFs and 3) The optimization step using the proposed synergy between the tuning of the equivalence and rule selection. In this paper, our methodology is built-up over the FARC-HD algorithm in order to learn the initial FRBCS, hence denoting the whole model as IVTURSFARC. We show the goodness and high potential of the use of IVTURS-FARC firstly by determining the suitability of the synergy produced when combining IVFSs, tuning and rule selection. Next, by studying whether the new IV-FRBCS enhances the results achieved by the tuning of the weak ignorance [24] when it is performed both individually and combined with the new tuning of the equivalence. Furthermore, we analyze the significance of the results obtained with IVTURSFARC versus the ones achieved by two of the best performing fuzzy methods published in the specialized literature, i.e., the original FARC-HD model [1] and the FURIA algorithm [2]. The performance of the proposals will be evaluated according to the accuracy rate and will be tested over a wide collection of data-sets selected from the KEEL data-set repository1[25], [26]. We will use some non-parametric tests [27]–[29] for the purpose of showing the significance of the performance improvements achieved by our proposal. This paper is arranged as follows: in Section II we recall some preliminary concepts in both FRBCSs and IVFSs theory together with the description of the construction method of IV-REFs used in this paper. Next, we introduce in detail the components of IVTURS-FARC in Section III, which involves the description of the initialization of the IV-FRBCS’ components, the definition of the new IV-FRM and the proposal to combine the genetic tuning of the system’ parameters with rule selection. The experimental framework and the results obtained by the application of our approaches together with the corresponding analysis are presented in Sections IV and V respectively. We finish the paper with the main concluding remarks in Section VI. II. PRELIMINARIES In this section, for the sake of completeness, we first review several preliminary concepts in both FRBCSs and IVFSs. Next, we recall the concept of IV-REFs [22], [23] and the construction method of these functions used in this paper. A. Fuzzy Rule-Based Classification Systems There are a lot of techniques used to deal with classification problems in the Data Mining field. Among them, FRBCSs are widely employed as they provide an interpretable model by means of the use of linguistic labels in their rules. The two main components of FRBCSs are: 1(http://www.keel.es/dataset.php) •Knowledge Base: it is composed of both the Rule Base (RB) and the Data Base, where the rules and the membership functions are stored respectively. •Fuzzy reasoning method: it is the mechanism used to classify objects using the information stored in the knowledge base. In order to generate the knowledge base, a fuzzy rule learning algorithm is applied that uses a set of Plabeled patterns xp= (xp1, . . . , xpn),p={1,2,...,P}where xpi is the ith attribute value (i={1,2,...,n}). Each of the n attributes is described by a set of linguistic terms together with their corresponding membership functions. In this work, we consider the use of fuzzy rules in the following form: Rule Rj:If x1is Aj1and ... and xnis Ajn then Class = Cjwith RWj(1) where Rjis the label of the jth rule, x= (x1,...,xn)is an n-dimensional pattern vector, Aji is an antecedent fuzzy set representing a linguistic term, Cjis the class label, and RWj is the rule weight [30]. Specifically, in this paper we consider the computation of the rule weight using the most common specification, that is, the fuzzy confidence value or certainty factor defined in [31] as: RWj=CFj=X xp∈ClassCj µAj(xp) P X p=1 µAj(xp) (2) where µAj(xp)is the matching degree of the pattern xpwith the antecedent part of the fuzzy rule Rj. Let xp= (xp1,...,xpn)be a new pattern to be classified, L denote the number of rules in the RB and M the number of classes of the problem; then, the steps of the fuzzy reasoning method [21] are as follows: 1) Matching degree, that is, the strength of activation of the if-part for all rules in the RB with the pattern xp.A conjunction operator (t-norm), T, is applied in order to carry out this computation. µAj(xp) = T(µAj1(xp1),...,µAjn (xpn)), j = 1,...,L. (3) 2) Association degree. To compute the association degree of the pattern xpwith the M classes according to each rule in the RB. To this aim, a combination operator, h, is applied to combine the matching degree with the rule weight. When using rules in the form shown in (1) this association degree only refers to the consequent class of the rule (i.e. k=Class(Rj)). bk j=h(µAj(xp), RW k j), k = 1, . . . , M, j = 1,...,L. (4) 3) Pattern classification soundness degree for all classes. We use an aggregation function, f, which combines the positive degrees of association calculated in the previous step. 3 Yk=f(bk j, j = 1,...,L and bk j>0), k = 1,...,M. (5) 4) Classification. We apply a decision function Fover the soundness degree of the system for the pattern classification for all classes. This function will determine the class label lcorresponding to the maximum value. F(Y1,...,YM) = arg max(Yk) k=1,...,M (6) B. Interval-Valued Fuzzy Sets In this section we introduce the IVFSs’ theoretical concepts which are necessary to understand the paper. First, we recall the definition of IVFSs with our interpretation of their length. Then, we remind both the intersection operation on IVFSs, which will be used to carry out the conjunction among the antecedents of the rules, and the complement operation on IVFS, which is used in the definition of IV-REFs. Next, we present the interval arithmetic that will be used to compute the rule weight as an element of L([0,1]). Finally, we introduce the total order relationship for intervals, which will be used in the classification step of the IV-FRM. Let us denote by L([0,1]) the set of all closed subintervals in [0,1], that is, L([0,1]) = {x= [x, x]|(x, x)∈[0,1]2and x≤x}. Definition 1: [32], [33] An interval-valued fuzzy set (or interval type 2 fuzzy set) Aon the universe U6=∅is a mapping AIV :U→L([0,1]), so that AIV (ui) = [A(ui), A(ui)] ∈L([0,1]),for all ui∈U. We denote by Lthe length of the interval under consideration, that is L(AIV (ui)) = A(ui)−A(ui). The length of the IVFSs can be seen as a representation of the ignorance related to the definition of the membership functions [16]. Independently of the source of the ignorance, it can be quantified by means of weak ignorance functions, as introduced in our previous work [34]. In this paper, we will use two basic operations with IVFSs, namely intersection and complement. On the one hand, we will apply t-norms [35], [36] to model the conjunction among the linguistic variables composing the antecedent of the rules. Therefore, we recall its extension on IVFSs. Definition 2: [37] A function T: (L([0,1]))2→L([0,1]) is said to be an interval-valued t-norm if it is commutative, associative, increasing in both arguments (with respect to the order x≤Lyif and only if x≤yand x≤y), and has the neutral element 1L. Definition 3: [37] An interval-valued t-norm is said to be t-representable if there are two t-norms Taand Tbin [0,1], being Ta≤Tb, so that T(x,y) = [Ta(x, y), Tb(x, y)] for all x, y ∈L([0,1]). All interval-valued t-norm without zero divisors verify that T(x,y) = 0Lif and only if x= 0Lor y= 0L. In this paper, we model the intersection by means of trepresentable interval-valued t-norms without zero divisors that will be denoted TTa,Tb, since they can be represented by Taand Tbas defined above. On the other hand, we will use interval-valued fuzzy negations, which are the extension of fuzzy negations on IVFSs, because IV-REFs must fulfill a condition based on them. In fuzzy set theory a strictly decreasing, continuous function c: [0,1] →[0,1] so that c(0) = 1 and c(1) = 0 is called a strict negation [38]. If cis also involutive, then it is called a strong negation. On this basis, we recall the definition for an interval-valued fuzzy negation. Definition 4: [37] An interval-valued fuzzy negation is a function N:L([0,1]) →L([0,1]) that is strictly decreasing (with respect to ≤L) so that N(1L) = 0Land N(0L) = 1L. If for all x∈L([0,1]),N(N(x)) = x, N is said to be involutive. Next, we present the interval arithmetic that we will use to compute the rule weight as an element of L([0,1]). This fact allow us to extend the IV-FRM in such a way that the ignorance represented by the IVFSs is taken into account throughout the inference process. A deep study of interval arithmetic can be found in [39]. Let [x, x],[y, y]be two intervals in R+so that x≤Ly, the rules of interval arithmetic are as follows: •Addition: [x, x] + [y, y] = [x+y, x +y]. •Subtraction: [x, x]−[y, y] = [|y−x|, y −x]. •Multiplication: [x, x]∗[y, y] = [x∗y, x ∗y]. •Division: [min(min(x y,x y),1),min(max(x y,x y),1)]. When we will need to use a total order relationship for intervals, i.e. when the largest interval membership needs to be determined in the last step of the IV-FRM, we will use the one defined by Xu and Yager in [40]: let [x, x],[y, y]∈ L([0,1]), and let s([x, x]) = x+x−1be the score of [x, x] and h([x, x]) = 1 −(x−x)be the accuracy degree of [x, x]. Then •If s([x, x]) < s([y, y]), then [x, x]<[y, y]; •If s([x, x]) = s([y, y]), then a) If h([x, x]) = h([y, y]), then [x, x] = [y, y]; b) if h([x, x]) < h([y, y]), then [x, x]<[y, y], Observe that any two intervals are comparable with this order relation. Moreover, it follows easily that 0Lis the smallest element in L([0,1]) and 1Lis the largest. We must remark that in the case of working with intervals in R+ 0, the above described total order relationship works but the domain of the result for both the score and the accuracy degrees changes (from [−1,1] to [−1,∞]for the score degree and from [0,1] to [−∞,1] for the accuracy degree). C. Construction Method of Interval-Valued Restricted Equivalence Functions This section is aimed at providing an appropriate background about the concept of IV-REFs [22], [23], which is one of the main tools used in our new IV-FRM. IV-REFs are used to quantify the equivalence degree between two intervals. They are the extension on IVFSs of the restricted equivalence functions [41], since they allow to quantify how equivalent two values are. 4 Definition 5: [41] A function REF : [0,1]2→[0,1] is called a restricted equivalence function associated with a strong negation c, if it satisfies the following conditions: (R1) REF(x, y) = REF(y, x)for all x, y ∈[0,1]; (R2) REF(x, y) = 1 if and only if x=y; (R3) REF(x, y) = 0 if and only if x= 1 and y= 0 or x= 0 and y= 1; (R4) REF(x, y) = REF(c(x), c(y)) for all x, y ∈[0,1], cbeing a strong negation; (R5) For all x, y, z ∈[0,1], if x≤y≤z, then REF(x, y)≥REF(x, z)and REF(y, z)≥ REF(x, z). We must point out that in this work we use the standard negation, that is, c(x) = 1 −x. Among the methods developed to construct restricted equivalence functions [42] we use the one based on automorphisms, which are defined below. Definition 6: An automorphism of the unit interval is any continuous and strictly increasing function φ: [0,1] →[0,1] so that φ(0) = 0 and φ(1) = 1. Example 1: The equation φ(x) = xa, being a∈(0,∞), generates a family of automorphisms. •If a= 1 →φ(x) = x •If a= 2 →φ(x) = x2 •If a= 0.5→φ(x) = x(1 2) •If a= 100 →φ(x) = x100 •If a= 0.01 →φ(x) = x(1 100 ) Fig. 1 depicts the behavior of the these five automorphisms. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x φ(x) φ(x) = x0.01 φ(x) = x0.5 φ(x) = x φ(x) = x2 φ(x) = x100 Fig. 1: Example of different automorphisms generated by varying the value of the parameter aas shown in Example 1. Specifically, the construction method of restricted equivalence functions based on automorphism is the one introduced in Proposition 1. Proposition 1: [42] If φ1, φ2are two automorphisms of the unit interval, then REF(x, y) = φ−1 1(1 − |φ2(x)−φ2(y)|) with c(x) = φ−1 2(1 −φ2(x)) is a restricted equivalence function. Example 2: Taking φ1(x) = x,φ2(x) = xwe obtain the following restricted equivalence function REF(x, y) = 1 − |x−y|, which satisfies conditions (R1)-(R5) with c(x) = 1 −xfor all x∈[0,1]. As mentioned previously, IV-REFs are the extension of restricted equivalence functions on IVFSs. Their definition is as follows. Definition 7: [22], [23] An Interval-Valued Restricted Equivalence Function (IV-REF) associated with a intervalvalued negation Nis a function IV -REF :L([0,1])2→L([0,1]) so that: (IR1) IV -REF(x,y) = IV -REF(y,x)for all x,y∈ L([0,1]); (IR2) IV -REF(x,y) = 1Lif and only if x=y; (IR3) IV -REF(x,y) = 0Lif and only if x= 1Land y= 0Lor x= 0Land y= 1L; (IR4) IV -REF(x,y) = IV -REF(N(x), N(y)) with N an involutive interval-valued negation; (IR5) For all x,y,z∈L([0,1]), if x≤Ly≤L z, then IV -REF(x,y)≥LIV -REF(x,z)and IV -REF(y,z)≥LIV -REF(x,z). In [22], [23], the authors show several construction methods of IV-REFs. Among them, we make use of the IV-REF construction method given in the following corollary. Corollary 1: [22], [23] Let REF be a restricted equivalence function and let T and S be any t-norm and any t-conorm in [0,1], IV -REF(x,y) = [T(REF(x, y), REF(x, y)), S(REF(x, y), REF(x, y))] is an IV-REF. Using the construction method of IV-REFs given in Corollary 1 applying the construction method of restricted equivalence functions recalled in Proposition 1 we obtain the IVREFs construction method used in this paper (Eq.(7)): IV -REF(x,y) = [T(φ−1 1(1 − |φ2(x)−φ2(y)|), φ−1 1(1 − |φ2(x)−φ2(y)|)), S(φ−1 1(1 − |φ2(x)−φ2(y)|), φ−1 1(1 − |φ2(x)−φ2(y)|))] (7) Example 3: Taking φ1(x) = x,φ2(x) = xwe obtain the following IV-REF IV -REF(x,y) = [T(1 − |x−y|,1− |x−y|), S(1 − |x−y|,1− |x−y|)] III. A LINGUISTIC FUZZY RULE-BASED CLASSIFICATION SYSTEM BASED ON AN INTERVAL-VALUED FUZZY REASONING METHOD WITH TUNING AND RULE SELECTION In our new approach we propose the introduction of the concept of minimum distance classifiers in the IV-FRM of the IV-FRBCSs. To do so, we compute the matching degree 5 between the patterns and the antecedent of the rules using IVREFs in order to quantify the equivalence degree between the interval membership degree and the ideal interval membership degree (1L= [1,1]) for each linguistic label composing the antecedent of the rule. The motivation is to strengthen the relevance of the rules with a higher equivalence degree with respect to the new pattern to be classified. The parametrized construction of the IV-REFs allows an easy generation of many of these functions to be performed. In this manner, we face the problem of choosing a suitable similarity function by applying a genetic tuning, which can lead to an improvement of the behaviour of the system in a general framework by looking for the most appropriate set of IV-REFs to solve each specific problem we deal with. In the remainder of this section, we first present a general outline of our new method (Section III-A). Then, we describe in detail the initialization of the system’ parameters (Section III-B) and the novel IV-FRM making use of IV-REFs (Section III-C). Finally, we introduce the tuning approach used to choose the most appropriate IV-REF for each variable together with the rule selection method (Section III-D). A. Overviewing IVTURS This section is aimed at showing a general overview of IVTURS. As depicted in Fig. 2, it is composed of three steps: 1) Initialization of the IV-FRBCS. This step involves the following tasks: •The generation of the initial FRBCS by means of the FARC-HD method by Alcal´ a-Fdez et al. [1]. •Modelling the linguistic labels of the FRBCS by means of IVFSs. •The generation of the initial IV-REF for each variable of the problem. 2) The extension of the fuzzy reasoning method on IVFSs. 3) The application of the optimization approach, which is composed of: •The genetic tuning in which we look for the best values of the IV-REFs’ parameters. •The rule selection process in order to decrease the system’s complexity. As we have mentioned, in this paper our approach combines IVTURS with the FARC-HD method to carry out the fuzzy rule learning process. Therefore, we denote our new proposal as IVTURS-FARC. In the remainder of this section, we describe in detail each step composing our new method. B. Initialization of the Interval-Valued Fuzzy Rule-Based Classification System It is well known that there are two possibilities in the generation of a type-2 model [43]: 1) a partial dependent one, where an initial type-1 fuzzy model is learnt and then used as a smart initialization of the parameters of the type-2 fuzzy model [44]–[46]; 2) a total independent method, where the type-2 fuzzy model is learnt without the help of any base type-1 fuzzy model [47]. In this paper, we use the first option, that is, we generate a base FRBCS using the FARC-HD algorithm, which is based on three stages: 1) Extracting the fuzzy association rules for classification by applying a search tree, whose depth of the branches is limited. 2) Preselecting the most interesting rules using subgroup discovery in order to decrease the computational cost of the system. 3) Optimizing the knowledge base by means of a combination between the well known tuning of the lateral position of the membership functions and a rule selection process. We make use of the two first stages in order to learn the initial FRBCS, which is the basis of our IV-FRBCS. We must point out that for the learning step we consider triangular membership functions, which are obtained by performing a linear partitioning of the input domain of each variable. After having the base FRBCS, we model its linguistic labels by means of IVFSs. To this aim we apply the following process: •We take as the lower bound of each IVFS the initial membership function (the one used in the learning step). •We generate the upper bound of each linguistic label. For their construction, the amplitude of the support of the upper bounds is determined by the value of the parameter W, which is initially set to 0.25 to achieve an amplitude 50% larger than that of their lower bound counterpart. An example of these sets is depicted in Fig. 3. aWbb+c 2cWd 1 0 U u Aj(u) Aj(u) Fig. 3: Example of an initial constructed IVFS as defined in [24], [34]. The solid line is the initial fuzzy set and therefore, it is the lower bound of the IVFS. The dashed line is the upper bound of the IVFS. Finally, we also have to generate the initial IV-REF associated with each variable of the problem. To this end, we apply the construction method recalled in Section II-C using the identity function as automorphism (φ(x) = x) in every case. In this manner, the initial IV-REF for every variable of the problem is the one that was shown in Example 3. C. Interval-Valued Fuzzy Reasoning Method This section is aimed at describing the new IV-FRM. To do so, we modify all the steps of the fuzzy reasoning method [21] recalled in Section II-A. In this way, we develop a method that intrinsically manages the ignorance that the IVFSs represent. 6 Initialization of the system + Generation of the initial IV-REFs Generation of the IVFSs Generation of the initial FRBCS Generation of the initial IV-FRBCS + Extension of the fuzzy reasoning method on IVFS Optimization Rule selection + Tuning of the equivalence + Fig. 2: Flow chart of our new IV-FRBCS. Let L be the number of rules in the RB and M the number of classes of the problem; If xp= (xp1, . . . , xpn)is a new pattern to be classified, the steps of the IV-FRM are the following: 1) Interval matching degree: we use IV-REFs to compute the similarity between the interval membership degrees (of each variable of the pattern to the corresponding IVFS) and the ideal membership degree, 1L, and then we apply a t-representable interval-valued t-norm (TTa,Tbas introduced in Section II-B) to these results: [Aj(xp), Aj(xp)] = TTa,Tb(IV -REF([Aj1(xp1), Aj1(xp1)],[1,1]),..., IV -REF([Ajn(xpn), Ajn(xpn)],[1,1])), j= 1, . . . , L. (8) We must point out that the result of the initial IV-REF for each variable is the interval membership degree, since the equation shown in Example 3 is applied. The result of each IV-REF changes according to the values of its parameters. These situations are shown in Example 4, where the result of the initially constructed IV-REF is shown in the first item whereas the two last items show results of IV-REFs when the initial values of their parameters have been modified. Example 4: Let [A(u), A(u)] = [0.6,0.7]. The results provided when using IV-REFs (Eq.(7)) constructed from different automorphisms are: •φ1(x) = xand φ2(x) = x: IV -REF([0.6,0.7],[1,1]) = [0.6,0.7] •φ1(x) = x2and φ2(x) = x: IV -REF([0.6,0.7],[1,1]) = [0.77,0.84] •φ1(x) = x0.5and φ2(x) = x: IV -REF([0.6,0.7],[1,1]) = [0.36,0.49] 2) Interval association degree: we apply a combination operator, h, to the interval matching degree computed previously and the rule weight: [bk j, bk j] = h([Aj(xp), Aj(xp)],[RWk j, RWk j]) k= 1, . . . , M, j = 1,...,L. (9) We must point out that the rule weight is an element of L([0,1]). To compute it, we apply the certainty factor (see Eq. (2)) making use of the interval arithmetic introduced in Section II-B. The resulting equation is shown in Eq. (10). [RWj, RWj] = X xp∈ClassCj [Aj(xp), Aj(xp)] P X p=1 [Aj(xp), Aj(xp)] (10) 3) Interval pattern classification soundness degree for all classes. We aggregate the positive interval association degrees of each class by applying an aggregation function f. [Yk, Yk] = f([bk j, bk j], j = 1,...,L and [bk j, bk j]>0L), k= 1,...,M. (11) 4) Classification. We apply a decision function Fover the interval soundness degree of the system for the pattern classification for all classes: F([Y1, Y1], ..., [YM, YM]) = arg max([Yk, Yk]) k=1,...,M (12) The last step of the IV-FRM consists of selecting the maximum interval soundness degree. Therefore, in order to to be able to make this decision, we use the total order relationship for intervals [40] presented in Section II-B. D. Tuning of the Equivalence and Rule Selection In this proposal, we make use of genetic algorithms with a double aim: 1) to tune the values of the parameters used in the construction of the IV-REFs in order to increase the reasoning capabilities of the IV-FRM and 2) to perform a rule selection process in which we obtain a compact and cooperative fuzzy rule set. According to Eq. (7), each IV-REF is constructed using the automorphisms φ1and φ2. In this paper, we take φ1(x) = xa and φ2(x) = xb, where a, b ∈(0,∞). Therefore, we can generate a huge number of IV-REFs by taking different values for the parameters aand b. The selection of a suitable IVREF to measure the equivalence degree in each variable could involve an improvement in the behaviour of the system to solve specific problems. To face this optimization problem, we propose the tuning of the values of the parameters aand bused to construct the IV-REFs associated with each variable of the problem. In order to cover as much search space as posible, we suggest varying these values in the interval [0.01,100]. This range is due to the fact that when using φ(x) = x0.01 the shape of the function is close to a crisp one and we use the inverse function (φ(x) = x100) in order to provide the same flexibility in both 7 sides of the identity function. The shadow surface of Fig. 1 (Section II-C) depicts the search space covered when using the proposed variation interval, which is almost the whole search space. On the other hand, regarding the size of the output model, fuzzy rule learning methods usually generate a large number of fuzzy rules so as to achieve a highly accurate system. However, in the fuzzy rule set created we can find irrelevant, redundant, erroneous or conflicting rules, which may perturb the performance of the system [48]. Therefore, a rule reduction process is often applied in order to improve the system’s accuracy by removing useless rules and, in turn, easing the readability of the system. There are a lot of methods to deal with the rule reduction process [10], [49]; from among them, we will use a rule selection approach developed using a simple binary codification in order to express whether or not the fuzzy rules belong to the rule set. In order to accomplish this genetic process we consider the use of the CHC evolutionary model [50] due to both its good properties to deal with complex search spaces [51], [52] and the good results provided in this topic [53], [54]. In the following, we describe the specific features of our evolutionary model: •Coding scheme. Each chromosome is composed of two well differentiated parts implying a double codification scheme: real codification for the tuning of the equivalence (CE) part and binary coding for the rule selection process (CR). 1) Tuning of the equivalence. Let nbe the number of attributes, the part of the chromosome to carry out the tuning of the IV-REFs is a vector of size 2× n:CE={a1, b1, a2, b2, ..., an, bn}, where ai, bi∈ [0.01,1.99] with i= 1,2, ..., n. Each pair of genes (ai, bi)represent the values of the parameters aand bto construct the IV-REF associated with the ith attribute. To construct the corresponding IV-REF, we have to adapt the gene values to the interval in which the automorphisms can vary ([0.01,100]). To do so, we adapt the value of each parameter ausing the following equation (we consider the same adaptation for parameter b): a=(a, if 0< a ≤1 1 2−a,if 1< a < 2(13) 2) Rule selection. Let Lbe the number of fuzzy rules in the RB, the part of the chromosome to perform the rule selection is a vector of size L, CR={r1, ..., rL}where ri∈ {0,1}with i= {1,2, ..., L}, determining the subset of fuzzy rules which compose the final RB as follows: if ri= 1then Ri∈RB else Ri/∈RB Therefore, the whole chromosome scheme is as follows: CE+R={CE, CR} •Initial Gene Pool. To include the initial FRBCS in the population, we initialize an individual with all genes with value 1. In this manner, we construct the initial IV-REFs using the identity functions as automorphism for each variable and we include all the fuzzy rules in the RB. •Chromosome Evaluation. We use the most common metric for classification, i.e. the accuracy rate. •Crossover Operator. Due to the double coding scheme of the chromosome, we apply a different crossover operator for each part of the chromosome: the Parent Centrix BLX operator [55] (which is based on the BLX-α) is used for the real coding part and the half uniform crossover scheme [56] is considered for the remainder. •Restarting Approach. To get away from local optima, we consider a restarting approach. To do so, as in the elitist scheme, we include the best global solution found until this moment in the next population and we generate the remaining individuals at random. For more details about the evolutionary algorithm, please refer to [1]. IV. EXPERIMENTAL FRAMEWORK In this section, we first present the real world classification data-sets selected for the experimental study. Next, we introduce the parameter set-up considered throughout this study. Finally, we introduce the statistical tests which are necessary to compare the results achieved throughout the experimental study. A. Data-sets We have selected a wide benchmark of twenty-seven real world data-sets selected from the KEEL data-set repository [25], [26], which are publicly available on the corresponding web page 2including general information about them, partitions for the validation of the experimental results and so on. Table I summarizes the properties of the selected data-sets, showing for each data-set the number of examples (#Ex.), the number of attributes (#Atts.) and the number of classes (#Class.). We must point out that the magic, pageblocks, penbased, ring, satimage and shuttle data-sets have been stratified sampled at 10% in order to reduce their size for training. In the case of missing values (crx, dermatology and wisconsin), those instances have been removed from the data-set. A5-fold cross-validation model was considered in order to carry out the different experiments. That is, we split the dataset into 5 random partitions of data, each one with 20% of the patterns, and we employed a combination of 4 of them (80%) to train the system and the remaining one to test it. B. Methods set-up This section is aimed at introducing the configurations that have been considered for the different methods used along the experimental study, namely, the FARC-HD method [1], our different versions of this method using IVFSs and the FURIA 2http://www.keel.es/dataset.php 8 TABLE I: Summary Description for the employed data-sets. Id. Data-set #Ex. #Atts. #Class. aus Australian 690 14 2 bal Balance 625 4 3 cle Cleveland 297 13 5 con Contraceptive 1,473 9 3 crx Crx 653 15 2 der Dermatology 358 34 6 eco Ecoli 336 7 8 ger German 1,000 20 2 hab Haberman 306 3 2 hay Hayes-Roth 160 4 3 hea Heart 270 13 2 ion Ionosphere 351 33 10 iri Iris 150 4 3 mag Magic 1,902 10 2 new New-Thyroid 215 5 3 pag Page-blocks 548 10 5 pen Penbased 1,992 16 10 pim Pima 768 8 2 sah Saheart 462 9 2 spe Spectfheart 267 44 2 tae Tae 151 5 3 tit Titanic 2,201 3 2 two Twonorm 740 20 2 veh Vehicle 846 18 4 win Wine 178 13 3 wiR Winequality-Red 1,599 11 11 wis Wisconsin 683 9 2 algorithm [2], which is briefly describe below (please refer to [2] for details). FURIA [2] builds upon the RIPPER interval rule induction algorithm [57]. The model built by FURIA uses fuzzy rules of the form given in Eq. (1) using fuzzy sets with trapezoidal membership functions. Specifically, FURIA builds the fuzzy rule base by means of these two steps: 1) Learn a rule set for every single class using a one-versusall decomposition. To this aim, a modified version of RIPPER is applied, which involves a building and an optimization phase. 2) Obtain the fuzzy rules by means of fuzzifying the final rules from the modified RIPPER algorithm in a greedy way. At classification time, the class predicted by FURIA is the one with maximal support. In case the query is not covered by any rule, a rule stretching method is proposed based on modifying the rules in a local way so as to make them applicable to the query. Regarding the configurations, for the FARC-HD algorithm we will apply the following one: •Conjunction operator: product t-norm. •Combination operator: product t-norm. •Rule weight: certainty factor. •Fuzzy reasoning method: additive combination [21]. •Number of linguistic labels per variable: 5 labels. •Minsup: 0.05. •Maxconf: 0.8. •Depthmax: 3. •kt: 2. For the IVTURS-FARC, we have considered the following configuration: •Conjunction operator: product interval-valued t-norm. •Combination operator: product interval-valued t-norm. •IVFSs construction: –Shape: Triangular membership functions. –Upper bound: 50% greater than the lower bound (W= 0.25). •Configuration of the initial IV-REFs: –T-norm: minimum. –T-conorm: maximum. –First automorphism: φ1(x) = x1(a= 1). –Second automorphism: φ2(x) = x1(b= 1). Regarding the genetic tuning with rule selection process, we have used the values suggested in [1], which are: •Population Size: 50 individuals. •Number of evaluations: 20,000. •Bits per gene for the Gray codification (for incest prevention): 30 bits. Finally, for the parameters of the FURIA algorithm, namely the number of folds and optimizations, we have set their values to 3 and 2 respectively, as recommended by the authors. C. Statistical Tests for Performance Comparison In this paper, we use some hypothesis validation techniques in order to give statistical support to the analysis of the results [58], [59]. We will use non-parametric tests because the initial conditions that guarantee the reliability of the parametric tests cannot be fulfilled, which implies that the statistical analysis loses credibility with these parametric tests [27]. Specifically, we use the Friedman aligned ranks test [60] to detect statistical differences among a group of results and the Holm post-hoc test [61] to find the algorithms that reject the equality hypothesis with respect to a selected control method. The post-hoc procedure allows us to know whether a hypothesis of comparison could be rejected at a specified level of significance α. Furthermore, we compute the adjusted pvalue (APV) in order to take into account the fact that multiple tests are conducted. In this manner, we can directly compare the APV with respect to the level of significance αin order to be able to reject the null hypothesis. Furthermore, we consider the method of aligned ranks of the algorithms in order to show graphically how good a method is with respect to its partners. The first step to compute this ranking is to obtain the average performance of the algorithms in each data set. Next, we compute the subtractions between the accuracy of each algorithm minus the average value for each data-set. Then, we rank all these differences in descending order and, finally, we average the rankings obtained by each algorithm. In this manner, the algorithm which achieves the lowest average ranking is the best one. These tests are suggested in the studies presented in [27]– [29], [58], where its shown that their use in the field of machine learning is highly recommended. A complete description of the test and software for its use can be found on the website: http://sci2s.ugr.es/sicidm/. V. ANALYSIS OF THE USEFULNESS OF IVTURS-FARC In this section, we analyse the behaviour of IVTURS-FARC. To do so, we develop an experimental study composed of three steps: 9 1) We determine the importance of both the rule selection process and, foremost, the IVFSs by comparing IVTURS-FARC versus its fuzzy counterpart with and also without rule selection (Section V-A). 2) We analyse the improvements achieved with respect to our previous proposal in the topic (Section V-B). 3) We study whether IVTURS-FARC improves the results obtained by two state-of-the-art fuzzy classifiers (Section V-C). The description of the methods used to carry out the two first steps of the experimental study are introduced in Table II. Methods using the prefix FS use the fuzzy system learnt applying the first two stages of the FARC-HD algorithm as defined in [1] and they apply REFs3for computing the matching degree. Methods using the suffix T WI apply our previous proposal for tuning the ignorance degree that each IVFS represents [24]. TABLE II: Description of the methods used in Sections V-A and V-B of the experimental study. Notation Linguistic labels Matching degree Tuning of the Tuning of the Rule selection equivalence ignorance degree FS T E Fuzzy sets REFs Yes No No FS T E+R Fuzzy sets REFs Yes No Yes IVFS T E IVFSs IV-REFs Yes No No IVFS T WI IVFSs IV-REFs No Yes No IVFS T E+WI IVFSs IV-REFs Yes Yes No IVTURS FARC IVFSs IV-REFs Yes No Yes Table III shows the classification accuracy of the different approaches used along the experimental study. Results are grouped in pairs for training and test, where the best global result for each data-set is emphasised in bold-face. Vertical lines group the methods involved in each scenario. In the remainder of this section, we develop the analysis carried out in the three aforementioned scenarios. A. Determining the suitability of IVTURS-FARC This section is aimed at showing the goodness of IVTURSFARC, that is, the combination of IVFSs with the tuning of the equivalence and a rule selection process. To do so, we have to analyse whether the rule selection step strengthens the quality of the results and, most importantly, we must determine whether the use of IVFSs is the main cause of the enhancement achieved. For this reason, we compare IVTURSFARC with respect to its version without the rule selection process (IVFS T E) and also with the fuzzy counterparts (FS T E+R and FS T E) in order to support the key role that IVFSs play in the system. Analysing the results achieved by these four approaches, we find that IVTURS-FARC reaches the best performance in twelve out of the twenty-seven data-sets implying a global performance improvement. This fact is confirmed in Fig. 4, where it is clearly shown that our new IV-FRBCS is the best ranking method. When applying the Friedman aligned ranks test we find a p-value of 5.70E-5, which confirms the existence of statistical 3These functions allow to quantify the equivalence between two numbers. They are the non interval-valued fuzzy version of the IV-REFs as explained in Section II-C. 65,76 53,94 60,85 37,44 0 10 20 30 40 50 60 70 FS_T_E FS_T_E+R IVFS_T_E IVTURS-FARC Fig. 4: Rankings of the four versions of IVTURS-FARC. differences among these four approaches. For this reason, we perform the Holm post-hoc test, the results of which are shown in Table IV, selecting IVTURS-FARC as the control method since it is the best ranking one. These results clearly determine the quality of IVTURS-FARC since: 1) the use of IVFSs leads to the outperforming of the results of the non interval-valued fuzzy versions of IVTURS-FARC with and without the rule selection stage and 2) the rule selection process allows us to enhance the results of the tuning of the equivalence. TABLE IV: Holm test to compare IVTURS-FARC with respect to its different versions. iAlgorithm Hypothesis APV 1 FS T E Rejected for IVTURS-FARC 0.003 2 IVFS T E Rejected for IVTURS-FARC 0.018 3 FS T E+R Rejected for IVTURS-FARC 0.053 B. Analysing the performance improvement with respect to the tuning of the weak ignorance degree Once the suitability of the IVTURS-FARC is confirmed, we study whether our new proposal enhances the results of our previous approach to the topic. To do so, we consider the use of the following proposals: •IVFS T WI: the initial IV-FRBCS4with a previous tuning approach for modifying the ignorance degree that each IVFS represents [24], that is, we modify the value of the parameter Wused in the construction of the IVFSs. •IVFS T E+WI: the initial IV-FRBCS with a tuning approach that simultaneously performs both the tuning of the ignorance degree [24] and the tuning of the equivalence, that is, we modify the values of the parameters W,aand bused in the construction of the IVFSs and the IV-REFs respectively. •IVTURS-FARC: the proposed method as explained in Section III-A. From the results in Table III, we can observe that IVTURSFARC achieves a notable enhancement of the global performance, which is based on the achievement of the best result 4The initial IV-FRBCS is the system obtained after the modeling of the linguistic labels of the base fuzzy classifier by means of IVFSs. It uses the new IV-FRM introduced in Section III-C with the identity function as the automorphism for the construction of each IV-REF.