Soft Computing: Contents, Techniques and Application

Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 DOI:10.5121/cseij.2023.13301 1 SOFT COMPUTING: CONTENTS, TECHNIQUES AND APPLICATION Michael Gr. Voskoglou Department of Mathematical Sciences, University of Peloponnese, Patras, Greece ABSTRACT Soft Computing is a relatively new branch of Computer Science that deals with approximate reasoning. The techniques of Soft Computing are used successfully nowadays in many domestic, commercial and industrial applications becoming a major research object in automatic control engineering. The present paper reviews the contents of Soft Computing, which include probabilistic and in particular Bayesian reasoning, fuzzy logic, artificial neural networks and genetic algorithms. These topics are complementary to each other and can be used simultaneously for solving complex real-life problems, which cannot or it is too difficult be modelled mathematically. The paper also explores the main techniques used in Soft Computing and discusses their advantages with respect to the traditional techniques of hard computing. KEYWORDS Soft Computing (SC), Probability, Bayesian Reasoning (BR), Fuzzy Logic (FL), Artificial Neural Networks (ANNs), Genetic Algorithms (GAs) 1. INTRODUCTION The theory and techniques of Soft Computing (SC) were introduced during the 1980‟s by the pioneer of FL Lofti Zadeh, who defined it as “an emerging approach to computing, which parallels the remarkable ability of the human mind to reason and learn in an environment of uncertainty and imprecision” [1]. In contrast to the conventional methods of hard computing, which are based on symbolic logical reasoning and numerical modelling, the methods of SC deal with approximate reasoning and processes that give solutions to complex real-life problems, which are too difficult to be modelled mathematically [1-4]. Its implementations in intelligent control, decision making support, nonlinear programming, etc., have made SC popular for people of a wide variety of disciplines and scientific backgrounds, like engineers, computer scientists and mathematicians, other natural and positive scientists, etc. SC is a synthesis of a series of computing and mathematical topics including Probabilistic Reasoning (PR), Fuzzy Logic (FL), Artificial Neural Networks (ANNs) and Genetic Algorithms (GAs). The techniques connected to the previous topics are not competitive, but complementary to each other and can be used together for solving a given problem. For example, FL can be used for knowledge representation via fuzzy IF – THEN rules, ANNs for learning and adaptation, GAs for evolutionary computation, etc. The techniques of SC are nowadays used successfully in many domestic, commercial and industrial applications becoming a major research object in automatic control engineering and having the potential to expand further in the forthcoming era of the fourth industrial revolution and the advanced Internet of Things (IoT) [5]. Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 2 The present paper reviews the main SC techniques and their applications, and points out their advantages and disadvantages with respect to the traditional techniques of hard computing. The rest of the paper is organized as follows: Section 2 presents basic information about fuzzy sets (FSs), the main principles of FL generated by them and their applications. Section 3 is devoted to probabilistic reasoning and in particular to Bayesian reasoning (BR), which appears today as a link between FL and the traditional bivalent logic (BL) of Aristotle. Sections 4 and 5 give a brief description of AANs and of GAs respectively and of their applications. The article closes with Section 6 including the final conclusions and some recommendations for future research. 2. FUZZY SETS, SYSTEMS AND LOGIC Zadeh defined in 1965 the notion of fuzzy set (FS) [6] for tackling mathematically the existing in everyday life partial truths and the definitions having no clear boundaries. A FS is defined as follows: Definition 1. A FS A in the set of the discourse U with membership function m: U → [0, 1] is the set of the ordered pairs A = {(x, m(x): x ∈ U}. The closer m(x) - called the membership degree of x in A – to 1, the better x satisfies the characteristic property of A. For reasons of simplicity, many authors identify a FS with its membership function. A crisp subset A of U is a FS in U with its characteristic function being its membership function. A disadvantage of FSs is that the definition of the membership function is not unique, depending on the “signals” that each observer receives from the environment. For the FS of “high mountains” (definition with no clear boundaries), for example, an observer may consider all the mountains with heights greater than 2000 meters as high and another one all those with heights greater than 2500 meters. Thus, the first observer will assign membership degrees 1 to all mountains with heights between 2000 and 2500 meters, whereas the second one will assign to them membership degrees less than 1. In general, there is not any exact rule for defining the membership function of a FS and the methods used for this are usually empirical or statistical. The only restriction is that the given definition must be compatible with common sense, otherwise the constructed FS does not give a creditable description of the corresponding real situation. This could happen, for example, if in the previous FS of “high mountains”, mountains with heights less than 500 meters have membership degrees greater than 0.5. It must be emphasized that probabilities and membership degrees, although they both act on the same interval [0, 1], are concepts different to each other. The expression “The probability that John is a good player is 0.7”, for example, based on the principles of BL, means that John is either a good player or not, but, according to the description of his qualifications by his coach to a third person, the probability to be a good player is 70%. On the contrary, the FL‟s expression “The membership degree of John in the FS of good players is 0.7” means that John is a rather good player (partial truth). When membership functions were interpreted as possibility distributions for handling incomplete information, FSs were used also to manage the existing in real world uncertainty. Zadeh [7] clarified the distinction between possibility and probability stating that “what is probable must preliminarily be possible”. Most concepts and operations on crisp sets have been extended in a natural way to FSs. For more details about FSs and the connected to them uncertainty we refer to the book [8]. The development of science and civilization owes a lot to Aristotle‟s (384-322 BC) bivalent logic (BL), which is based on the “principle of the excluded middle “(any proposition is either true or false) and remained for centuries at the center of human reasoning. Although opposite views Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 3 reported also early in human history supporting the existence of a third area between true and false, integrated propositions for multi-valued logics appeared only in the early 1900s mainly by Lukasiewicz and Tarski [9, 10]. Zadeh introduced with the help of FSs the infinite-valued FL [11], where the truth values are modelled by numbers in the unit interval [0, 1]. FL satisfies Lukasiewicz‟s “principle of valence”, according to which propositions are not only either true or false, but they can also have intermediate truth values. FL extends and completes the traditional BL by examining what happens in the boundaries between true and false. The basic arithmetic operations have been extended to FL with the help of fuzzy numbers, which are FSs in the set R of real numbers satisfying some special conditions [12]. The process of solving a problem with the FL‟s approach includes the following steps:  Fuzzification of the problem‟s data by defining properly the corresponding FSs.  Elaboration of the fuzzy data with the FL‟s IF-THEN rules and operators on the purpose of obtaining the solution of the problem in the form of a FS.  Defuzzification of the problem‟s solution in order to express it in the natural language. This process is graphically represented in Figure 1 [3]. Figure 1. Graphical representation of the solution of a problem with the FL‟s approach Defuzzification is understood to be the representation of the fuzzy outcomes of a problem‟s solution by a suitable real value. Among the more than 30 defuzzification methods reported in the literature [8], the most frequently used is the center of gravity (COG) technique. Applying this technique the representative real value is obtained by calculating the coordinates of the COQ of the level‟s section between the graph of the corresponding membership function and the OX axis [13]. A characteristic application of the COG technique is the Rectangular Fuzzy Assessment model (RFAM), created for the assessment of the quality performance of a group of objects participating in a certain activity; e.g. see [14, Section 4]. A fuzzy system is understood to be a group of FSs related and connected to each other. A fuzzy system “learns” from the experience, i.e. it is not only able to use its own knowledge to represent and explain the real world, but can also increase it with the help of new data. In other words, a fuzzy system mimics the way in which humans think. Fuzzy systems have been proved very successful in practical applications to almost all sectors of human activity. Such applications started to appear in industry during the 1970‟s. The first one was in the area of cement kiln control [15], followed by Mambani‟s fuzzy systems for controlling a steam engine and the operation of traffic lights [16]. Another important type of fuzzy control system (FCS) was designed later in Japan by Takagi, Sugeno and Kang [17]. Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 4 The operation of a FCS is based on the Fuzzy Approximation Theorem (FAT) [18]. In simple words the FAT, represented graphically in Figure 2, states that any system can be approximated by a fuzzy system having a finite number of IF-THEN rules. Figure 2. Graphical representation of the Fuzzy Approximation Theorem The curve in Figure 2 represents the original system and each of the patches covering it represents an IF-THEN fuzzy rule. If an IF-THEN rule is accurate (not fuzzy), the corresponding patch degenerates to a point on the curve. Fuzzy systems are able nowadays to control the function of various domestic devices like electric washing-machines, fridges and air-conditions [19], a building‟s central heating system [19] , the movement of an autonomous vehicle [20], or even the purchase of shares from the stock exchange market [19] , etc. Following Zadeh‟s FSs, several extensions, generalizations and alternatives, accompanied by important applications, have been reported during the last years, aiming to treat more effectively all the types of the existing in real word uncertainty. The most popular among them are reviewed briefly in [21]. Note that, the already mentioned difficulty to define properly the membership function of a FS, exists also for all generalizations of FSs involving membership functions, such as type-2 FSs, intuitionistic FSs, neutrosophic sets, etc. A number of theories, however, have been also proposed to overcome this difficulty. Interval-valued FSs, for example, replace the membership degrees of the elements of a FS with closed subintervals of [0, 1], rough sets use a pair of crisp sets that give the lower and upper approximation of the original set, soft sets tackle the existing uncertainty with the help of a suitable set of parameters, in the theory of grey systems/numbers the definition of a membership function is not necessary, etc. [22]. Although none of the previous theories has been proved sufficient for treating all the types of the existing uncertainty alone, their combination forms an adequate framework for this purpose. 3. PROBABILITIES AND BAYESIAN REASONING According to E. Jaynes [22] probability theory can be considered as an extension of BL reducing to it when something is absolutely true or absolutely false, and many eminent scientists supported this argument (e.g. see [23). Probability, however, the development of which was based on the principles of BL, has been proved suitable for tackling only the cases of uncertainty which are due to randomness [24]. As a result, the Jaynes‟ probabilistic logic is subordinate to FL. Bayes’ rule, which is a straightforward consequence of the well-known formula P(B/A) = P(A B) P(A)  calculating the conditional probability P(B/A), is expressed by the equation P(A/B) = P(B/A)P(A) P(B) (1) Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 5 Bayesian reasoning (BR) appears today as a link between BL and FL. In fact, the Bayes‟ rule calculates the conditional probability P(A/B) in terms of the conditional probability P(B/A) of the inverse process, the prior probability P(A) and the posterior probability P(B). The value of P(A) is fixed before the experiment, whereas the values of P(B/A) and P(B) are calculated with the help of the experiment‟s data. In many cases, however, the value of P(A) does not remain constant. As a result, different values of P(A/B) are obtained by the Bayes‟ rule for each possible value of P(A). In this way, the Bayes‟ rule introduces a kind of multi-valued logic treating the existing uncertainty about the value of P(A) in a way analogous to FL [25, Section 5]. This argument is illustrated by the following example: Example 1: A man is subject to a diagnostic test tracing a virus. The statistical accuracy of the test is 95%. The man has no symptoms of the corresponding disease, but the test is positive. What is the probability for him to be a carrier of the virus, if it is known that i) 2% or ii) 5% respectively of the inhabitants of the city, where he lives, have been infected by the virus? Solution: Consider the events: A = the man is a carrier of the virus and B = the test is positive. We need to calculate the conditional probability P(A/B), given that the problem‟s data show that in both cases P(B/A) = 0.95. i) In this case P(A) = 0.02. Further, among 100 inhabitants of the city, 2 on average are carriers and 98 are non-carriers of the virus. Assuming that all these people make the test, we should have on average 2x95% = 1.9 positive tests from the carriers and 98x5% =4.9 from the non-carriers of the virus, i.e. 6.8 in total positive tests. Therefore, P(B) = 0.068 and the Bayes‟ rule gives that 0.95 x 0.02 P(A/B) 0.279 0.068  . This means that the probability for the man to be a carrier of the virus is only 27.9%. ii) In this case P(A) = 0.05 and among 100 inhabitants of the city who made the test we have on average 5x95% = 4.75 positive tests from the carriers and 95x5% = 4.75 from the noncarriers of the virus. Therefore, P(B) = 0.095% and by the Bayes‟ rule we obtain that the probability for the man to be a carrier of the virus is now 50%. BR is involved in many everyday life and science situations [26]. In particular, according to the scientific method [27, Section 3], the development of a scientific theory starts with a series of observations, say a1, a2, …, an, on the corresponding phenomena that leads by induction (intuitively) to a theory T1 for their explanation. Then, after T1 has been verified deductively and a series of additional deductive inferences K1, K2,…, Ks have been obtained, a new series of observations follows, say b1, b2, …., bm. If some of the new observations are not compatible with the premises (axioms, principles, etc.) of T1, a new theoryT2 is developed to extend or replace T1, and so on (Figure 3) [27]. Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 6 Figure 3. A graphical representation of the scientific method A deductive inference (I), therefore, obtained on the basis of the premises (A) of theory, is true, if these premises are true. Consequently, the conditional probability P(I/A), which can be calculated by the Bayes‟ rule, expresses the degree of truth of the inference I. Recent research provided serious indications that most of the mechanisms of the human mind‟s function are Bayesian [28]. This marks out BR as a very useful tool for Artificial Intelligence (AI), which designs and constructs “clever” machines mimicking human behavior. The Nobel Prize winner J. Mather has already expressed his fears that the Bayesian machines of AI could become too smart in future, so that to leave for humans a second role only [29]! Sir H. Jeffrey‟s characterization, therefore, of the BR as the “Pythagorean theorem of probability theory” [30] is well justified. 4. ARTIFICIAL NEURAL NETWORKS Human brain consists of more than a billion neural cells each of them working as a simple processor of information. The interaction between all cells and their parallel processing makes the brain's function and abilities possible. A biological neural network (BNN) is a group of biological neurons connected together by axons and dendrites, their connections called synapses. The cell-body of a BNN processes the information, the axon enables the signal conducting and the synapses control the signals. Whereas the axon is the output part of a neuron, the dendrites are input elements, which receive synaptic signals from other neurons. The transmission of signals is achieved by diffusion of chemicals called neurotransmitters [2]. ANNs are inspired and based on the function of BNNs trying to simulate the learning process of the human brain [2, 3]. The term "artificial" means that ANNs are implemented in computer programs being able to handle the large number of necessary calculations during the learning process (neural computing, [31]). An ANN is a group of artificial neurons or nodes connected together in a way analogous to BNNs. The connections of the biological neurons are modeled in ANNs as weights between nodes. Each artificial neuron performs a particular little operation and Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 7 the overall ANN‟s operation is the weighted sum of all these operations. The basic components of an ANN include the input, the hidden layer and the output layer (Figure 4, [32]). Figure 4. A simplified view of an ANN An ANN must be trained to make each set of inputs to produce the desired outputs. This is done by feeding teaching patterns to the ANN and letting it change its weighting function according to some already imposed learning rules. An ANN is usually designed for specific operations and works best if the relations between the inputs and the outputs are non-linear. ANNs are highly efficient for tackling problems in which there are no algorithms or specific rules to be followed for their solution. The applications of ANNs include analysis, classification and recognition of data, pattern recognition, control, associative memory, image processing and compression, forecasting applications, weather and stock market prediction, security and loan applications, etc. [2, 3, 33]. The ANNs make extensive use of the traditional and modern learning theories [34]. A disadvantage of ANNs is that there is not any general methodology for training and verifying them, therefore they cannot be used as universal tools for solving problems. Also, excessive training may be required in complex ANNs. The quality of the outcomes of an ANN depends on the accuracy of the given data. Fuzzy systems and ANNs simulate the operations of the human mind. The ANNs, having the ability to learn and to process the information rapidly, concentrate on the “hardware” of the human brain, whereas fuzzy systems concentrate on the “software” emulating human reasoning. Figure 5 represents graphically the relationships among ANNs, FL and probabilistic reasoning within the wider class of SC [2]. The intersections in Figure 5 include probabilistic approaches to ANNs and FL systems, BR and neuro-fuzzy systems, which are hybrid systems using a learning algorithm from an ANN to determine their fuzzy parameters. Characteristic examples of such systems are the adaptive neuro-fuzzy inference systems (ANFIS) providing accelerated learning capacity and adaptive interpretation capabilities to model complex patterns [33]. Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 8 Figure 5. FL, ANNs and probabilistic reasoning within the wider class of SC 5. GENETIC ALGORITHMS GAs are models of AI inspired by the process of evolution in nature, which are included in the wider class of evolutionary computing [3]. Their basic idea is to emulate the natural selection for finding the best solution to optimization problems appearing in real-life situations. A GA, for example, can search through several electronic designs to find the best combination resulting in the better or cheaper design of an electronic device. Alan Turin proposed in 1950 a “learning machine” which would parallel the principles of evolution, but computer simulation of evolution started in 1954 with the work of the Norwegian – Italian mathematician N. A. Barricelli who used the computer of the Institute for Advanced Study in Princeton, USA [34]. The evolution in a GA, being an iterative process, starts with a randomly created population - called generation - of possible solutions of the corresponding optimization problem, which are usually referred to as individuals. The initial generation‟s size varies according to the form of the corresponding problem, but it typically contains several hundreds or even thousands of individuals. Often the initial generation includes the entire range of the possible solutions, referred as the feasible region or search space. In the next step each individual is evaluated, with the help of a properly defined fitness function, on how well fits the desired requirements. The best fit individuals of the first generation are selected for breeding the next generation, along with a small proportion of less fit individuals, which ensures the genetic diversity of the subsequent generation (mutation). A new generation is created then by combining the best aspects of the selected individuals (crossover). The same circle is repeated until a termination condition is reached; e.g. a solution satisfying a minimum criterion, or the completion of a pre-fixed number of generations, or the end of the existing budget, or, in the best case, when successive iterations no longer produce better results, or even a combination of the previous conditions. GAs have found successful applications in many sectors, such as biomedical and control engineering, code-breaking, games‟ theory, automated manufacturing and design, climatology, etc. On the contrary, due to the high possibility of an exponential increase of the search space, GAs cannot be used easily in problems where the number of individuals exposed to mutation is large (e.g. design of a house, of a complex engine, of a plane, etc.). In such cases GAs can be applied only if the corresponding problem is broken down into the simplest possible parts (e.g. airfoils instead of whole aircraft designs). Also GAs cannot effectively solve problems in which Computer Science & Engineering: An International Journal (CSEIJ), Vol 13, No 3, June 2023 9 the fitness criterion is simply a right/wrong statement, e.g. decision making problems. The suitability of a GA depends on the amount of knowledge about the problem for solution; for wellknown problems better, more specialized, approaches often exist for their solution. Another limitation in the use of GAs is that the search for finding the optimal solution to complex problems often requires a very expensive fitness function evaluation. Also, the termination condition is not always clear. Further, in certain problems the GAs converge to local optima rather, than to the global one not knowing how to sacrifice the local in favor of the global fitness, etc. 6. DISCUSSION AND CONCLUSIONS In the present work we reviewed the main techniques applied to SC including FL, probabilistic methods and in particular Bayesian reasoning, AANs and GAs. All these techniques are tolerant to uncertainty, imprecision, partial truths and approximation and are used for solving complex problems which cannot, or it is very difficult to be described by mathematical models. In machinery using traditional control systems. The techniques of SC, however, also have some disadvantages with respect to the traditional and stable methods of hard computing. The definition of the membership function of a FS, for example, is not unique depending upon the “signals” that each observer receives from the environment. Probabilities have been proved suitable to tackle successfully only the cases of uncertainty due to randomness, although Bayesian reasoning appears today as a link between BL and FL. The ANNs cannot be used as universal tools for solving related problems, because there is not any general methodology for training and verifying them. Further, excessive training may be required in complex ANNs. The use of GAs, which are usually designed for special applications only, has also a number of serious limitations, such as the frequently very expensive evaluation needed for their fitness function, the existing in certain cases vagueness about the termination process, their tendency to converge to local optima rather than to the global one, etc. In the current era of the fourth industrial revolution and the growth of the advanced IoT, however, as the processing by computer devices is increasing and their cost is reduced, the need of using SC methods has become too important and has the potential to expand further. The attempt, therefore, to improve and expand the SC techniques (e.g. see [35-37]), is an interesting area for further research. REFERENCES [1] Zadeh, L.A ((1994), “Fuzzy logic, neural networks and soft computing‟, Communications of the ACM, 37(3), 77-84. [2] Paplinski, A.P. (20050, “Basic concepts of Neural Networks and Fuzzy Logic Systems”, available online: https://www.cic.ipn.mx/~pescamilla/ContInt/Paplinski2005.pdf, [3] Gupta, P., Kulkarni, N. (2013), “An Introduction of Soft Computing Approach over Hard Computing”, Int. Journal of Latest Trends in Engineering and Technology, 3(1), 254-258. [4] Ibrahim, D. (2016), “An overview of soft computing”, Procedia Computer Science, 102, 34-38. [5] Schwab, K. (2016), The Fourth Industrial Revolution, Crown Publishing Group, New York, USA. [6] Zadeh, L.A. (1965), “Fuzzy Sets”, Inf. Control, 8, 338–353. [7] Zadeh, L.A. (1978), “Fuzzy Sets as a Basis for a Theory of Possibility”, Fuzzy Sets and Systems, 1, 3–28,). [8] Klir, G.J., Folger, T.A. (1988), Fuzzy Sets, Uncertainty and Information, Prentice-Hall: London, UK. [9] Lejewski, C. (1967), “Jan Lukasiewicz”, Encycl. Philos., 5, 104–107. [10] “Tarski, A.” (2018), Encyclopedia Brittanica,, available online: www.britannica.com/biography/Alfred-Tarski, last accessed 2018/12/20.