Computational approaches to Explainable Artificial Intelligence: Advances in theory, applications and trends

Information Fusion 100 (2023) 101945 Available online 29 July 2023 1566-2535/© 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/bync/4.0/). Contents lists available at ScienceDirect Information Fusion journal homepage: www.elsevier.com/locate/inffus Full Length Article Computational approaches to Explainable Artificial Intelligence: Advances in theory, applications and trends J.M. Górriz1,2,3,∗, I. Álvarez-Illán1,2, A. Álvarez-Marquina25, J.E. Arco1,2,4, M. Atzmueller6,7, F. Ballarini15, E. Barakova16, G. Bologna41, P. Bonomini14,15,8,31, G. Castellanos-Dominguez40, D. Castillo-Barnes1,4,2, S.B. Cho47, R. Contreras19, J.M. Cuadra22, E. Domínguez17, F. Domínguez-Mateos24, R.J. Duro12, D. Elizondo18, A. Fernández-Caballero20,26, E. Fernandez-Jover28, M.A. Formoso4,2, N.J. Gallego-Molina4,2, J. Gamazo22, J. García González17, J. Garcia-Rodriguez29, C. Garre24, J. Garrigós8, A. Gómez-Rodellar25,27, P. Gómez-Vilda24,25, M. Graña42, B. Guerrero-Rodriguez30, S.C.F. Hendrikse48, C. Jimenez-Mesa1,2, M. Jodra-Chuan33,34, V. Julian9, G. Kotz21, K. Kutt10, M. Leming46, J. de Lope43, B. Macas14,8, V. Marrero-Aguiar36, J.J. Martinez8, F.J. Martinez-Murcia1,2, R. Martínez-Tomás22, J. Mekyska32, G.J. Nalepa10, P. Novais35, D. Orellana37, A. Ortiz4,2, D. Palacios-Alonso24,25, J. Palma11, A. Pereira38, P. Pinacho-Davidson19, M.A. Pinninghoff19, M. Ponticorvo23, A. Psarrou39, J. Ramírez1,2, M. Rincón22, V. Rodellar-Biarge25, I. Rodríguez-Rodríguez4,2, P.H.M.P. Roelofsma50, J. Santos12, D. Salas-Gonzalez1,2, P. Salcedo-Lagos21, F. Segovia1,2, A. Shoeibi1,2, M. Silva44, D. Simic45, J. Suckling3, J. Treur49, A. Tsanas27, R. Varela13, S.H. Wang5, W. Wang5, Y.D. Zhang5, H. Zhu5, Z. Zhu5, J.M. Ferrández-Vicente8,25 1SiPBA at Data Science and Computational Intelligence Institute, University of Granada, Granada, Spain 2Data Science and Computational Intelligence Institute (DaSCII), University of Granada, Granada, Spain 3Department of Psychiatry, University of Cambridge, Cambridge, UK 4Department of Communications Engineering, University of Malaga, Malaga, Spain 5School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK 6Semantic Information Systems Group, Osnabrück University, Osnabrück, Germany 7German Research Center for AI (DFKI), Osnabrück, Germany 8Department of Electronics, Computer Technology and Projects, Universidad Politécnica de Cartagena, Cartagena, Spain 9Valencian Research Institute for AI (VRAIN), Universitat Politècnica de València, València, Spain 10 Jagiellonian Human-Centered AI Laboratory (JAHCAI) and Institute of Applied Computer Science, Jagiellonian University, Kraków, Poland 11 Department of Information and Communication Engineering, University of Murcia, Murcia, Spain 12 Centre for Information and Communications Technology Research (CITIC), Department of Computer Science and Information Technologies, University of A Coruña, A Coruña, Spain 13 Department of Computer Science, University of Oviedo, Oviedo, Spain 14 Instituto Argentino de Matemática Alberto Calderón y Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Buenos Aires, Argentina 15 Instituto Tecnológico de Buenos Aires (ITBA), Buenos Aires, Argentina 16 Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands 17 Department of Computer Science, University of Malaga, Malaga, Spain 18 De Montfort University, Leicester, UK 19 Department of Computer Science, Faculty of Engineering, Universidad de Concepción, Concepción, Chile 20 Neurocognition and Emotion Research Unit, Albacete Research Institute of Informatics, University of Castilla-La Mancha, Albacete, Spain 21 Department of Educational Informatics, Universidad de Concepción, Concepción, Chile 22 Department of AI, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain 23 Department of Humanistic Studies, University of Naples Federico II, Napoli, Italy 24 Escuela Técnica Superior de Ingeniería Informática, Universidad Rey Juan Carlos, Campus de Móstoles, 28933 Móstoles, Madrid, Spain 25 Neuromorphic Speech Processing Lab, Center for Biomedical Technology, Universidad Politécnica de Madrid, Campus de Montegancedo, 28223 Pozuelo de Alarcón, Madrid, Spain ∗Corresponding author at: SiPBA at Data Science and Computational Intelligence Institute, University of Granada, Granada, Spain. E-mail addresses: [email protected],[email protected] (J.M. Górriz). https://doi.org/10.1016/j.inffus.2023.101945 Received 14 June 2023; Accepted 22 July 2023 Information Fusion 100 (2023) 101945 2 J.M. Górriz et al. 26 Biomedical Research Networking Center in Mental Health, Instituto de Salud Carlos III, 28016 Madrid, Spain 27 Usher Institute, Faculty of Medicine, University of Edinburgh, Edinburgh, UK 28 Instituto de Bioingeniería, Universidad Miguel Hernández, Elche, Alicante, Spain 29 Department of Computers Technology, University of Alicante, Alicante, Spain 30 Central University of Ecuador, Quito, Ecuador 31 Instituto de Ingeniería Biomédica, Fac. de Ingeniería, Universidad de Buenos Aires, Buenos Aires, Argentina 32 Department of Telecommunications, Brno University of Technology, 61600 Brno, Czech Republic 33 Department of Personality, Assessment and Clinical Psychology, Faculty of Education, Complutense University of Madrid, 28040 Madrid, Spain 34 Asociación Nuevo Horizonte, 28231 Las Rozas de Madrid, Spain 35 ALGORITMI Research Centre/LASI, University of Minho, Braga, Portugal 36 Facultad de Filología, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain 37 Facultad de Energía, Universidad Nacional de Loja, Loja, Ecuador 38 Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic Institute of Leiria, Leiria, Portugal 39 University of Westminster, London, UK 40 Signal Processing and Recognition Group, Universidad Nacional de Colombia, 170003 Manizales, Colombia 41 University of Applied Sciences and Arts of Western Switzerland, Geneva, Switzerland 42 Department of CCIA, University of the Basque Country (UPV/EHU), Spain 43 Department of AI, Universidad Politécnica de Madrid, Campus de Montegancedo, 28660 Boadilla del Monte, Madrid, Spain 44 Universidad Mayor de San Andrés, La Paz, Bolivia 45 University of Novi Sad, 21102 Novi Sad, Serbia 46 Center for Systems Biology, Massachusetts General Hospital, Boston, MA, United States of America 47 Department of Computer Science, Yonsei University, Seoul, Republic of Korea 48 Vrije Universiteit Amsterdam, Department of Clinical Psychology, Amsterdam, Netherlands 49 Vrije Universiteit Amsterdam, Department of Computer Science, Amsterdam, Netherlands 50 Erasmus MC, Rotterdam, Netherlands ARTICLE INFO Keywords: Explainable Artificial Intelligence Data science Computational approaches Machine learning Deep learning Neuroscience Robotics Biomedical applications Computer-aided diagnosis systems ABSTRACT Deep Learning (DL), a groundbreaking branch of Machine Learning (ML), has emerged as a driving force in both theoretical and applied Artificial Intelligence (AI). DL algorithms, rooted in complex and non-linear artificial neural systems, excel at extracting high-level features from data. DL has demonstrated humanlevel performance in real-world tasks, including clinical diagnostics, and has unlocked solutions to previously intractable problems in virtual agent design, robotics, genomics, neuroimaging, computer vision, and industrial automation. In this paper, the most relevant advances from the last few years in Artificial Intelligence (AI) and several applications to neuroscience, neuroimaging, computer vision, and robotics are presented, reviewed and discussed. In this way, we summarize the state-of-the-art in AI methods, models and applications within a collection of works presented at the 9th International Conference on the Interplay between Natural and Artificial Computation (IWINAC). The works presented in this paper are excellent examples of new scientific discoveries made in laboratories that have successfully transitioned to real-life applications. 1. Introduction Current research in Artificial Intelligence (AI) is predominantly focused on addressing the challenge of explainability in developed models and algorithms, particularly artificial neural networks. This emerging trend, referred to as Explainable Artificial Intelligence (XAI), offers several advantages such as enhanced confidence in the decision-making process, improved error analysis capabilities, result verification, and potential model refinement. XAI instills safety and trust among users by elucidating the ‘‘how’’ and ‘‘why’’ of automated decision-making in diverse applications such as bio-inspired systems, virtual agents, emotion and affective analysis, robotics, and medical diagnosis. These advantages will be further explored in this manuscript. A novel approach within XAI involves interpreting the predictions of recently developed Deep Learning (DL) models using various visualization techniques [1]. One notable application is in the medical field, where XAI methods contribute significantly to the analysis and classification of mammography images [2], yielding valuable insights. DL is a generic name that covers an ever-expanding constellation of computational approaches that have in common some kind of biological inspiration and the use of gradient descent-based learning methods [3]. The DL revolution started quietly in the 1990s with the first proposed Convolutional Neural Networks (CNN) [4], but its adoption exploded around 2010, growing exponentially afterwards into a myriad of architectures and applications [5–9]. In essence, DL approaches are data-driven and therefore conditioned to the available data. Generative approaches [10] try to overcome this limitation by producing synthetic samples by exciting a generative model with noise. Bio-inspired computing methods have continued to see a steady expansion in recent years. Apart from the rapid growth of DL based architectures in Machine Learning (ML), bio-inspired solutions for search and optimization algorithms are still a rapidly growing field of research. New methods continuously appear in the scientific literature that are inspired by the behavior of animals, plants, social phenomena, and physical systems. A simple search in Scopus with the words ‘‘bioinspired’’ (title, keyword, abstract) returns more than 21,000 research papers, with continuous progress since the beginning of the century. Fig. 1 shows the percentages of these returned articles, classified by subject area, indicating a wide variety of applications of bio-inspired methods, especially in engineering. These computational approaches have fostered new areas of interdisciplinary research. For example, Affective Computing (AfC) is an emerging research field aimed at developing methods and tools for emotion recognition, processing, and simulation in computer systems [11]. One method that can focus research on affective computing is its intersection with ambient intelligence (AmI) and context-aware systems (CAS) leading to the development of Affective Computing and Context Awareness in Ambient Intelligence (AfCAI) [12]. We assume that this goal-oriented yet multidisciplinary research approach, encompassing AI, computer science, biomedical engineering and experimental science, will offer more comprehensive solutions in fundamental and applied research. Moreover, the use of virtual agents supporting human tasks has resulted in more evidence that the development of social interactions between them can be automated using computing principles inspired by natural processes. For a long time, technology has been insufficient in developing systems that relate to human beings in a natural human Information Fusion 100 (2023) 101945 3 J.M. Górriz et al. Fig. 1. Scopus found articles related to ‘‘bio-inspired’’ computing methods and classified by subject area. way [13]. However, the current prospects indicate that through biologically grounded computing principles and AI the automation of long term social and emotional relations between human and virtual agents can occur. For example, this involves computational modeling of social sowing [14], emotion recognition [15–18], sentiment analysis [19] and human attention and performance monitoring [15,20–22]. Needless to say, the application of AI to the field of robotics is currently very open and wide ranging. Research in this area is carried out from different perspectives, ranging from the more hardwarerelated aspects of sensing and actuation, which are necessary to provide the robot with appropriate sensing data in the correct representation, or to calibrate and adequately prepare the actuation mechanisms, to addressing higher levels of cognition that aim to make robots fully autonomous through the construction of specific applications of robotic systems. Sensing focus more on the application of various developments in AI in terms of specific algorithms – often based on deep learners and other modern approaches – to specific sensing or actuation tasks within traditional robotic architectures [23]. That is, from the sensor viewpoint, it seeks to facilitate the detection of specific features using a single sensor, as in vision, or contemplating a multimodal approach as in the integration of different sensory modalities [24]. On the other hand, from an actuation perspective, it deals with calibration and actuation representation issues. Finally, biomedical and health applications are a key area in contemporary AI research, where new devices and AI approaches, techniques or toolkits are being developed. The main feature of this field is the degree of interdisciplinary between diverse professionals. For instance, the application of medical principles joins to design and develop new approaches or tools that require the conjunction of engineers, physicians, mathematicians and speech therapists, among others. Bioinformatics, biomechanics, biomaterials, medical devices, and rehabilitation engineering are other different fields that strongly interact with AI within biomedical applications. These applications allow advancing in health care diagnosis,monitoring,treatment or even therapy. The evaluation of brain functions using digital biomarkers, from imaging technologies, physiological fluids, genomics, and AI-based data analytics, is attracting considerable research interest. These methods provide powerful decision support tools towards the functional assessment of treatment and even possible rehabilitation in neurological disorders [25]. For example, Neuroimaging (NI) creates a large amount of information that can be used to diagnose a wide range of brain diseases. Despite the high quality of these images, selecting the appropriate treatment is not a straightforward task because patients suffering from different pathologies may present similar structural or functional features and experience similar symptoms. The emergence of AI permits the development of powerful tools to address this issue, leading to Computer Aided Diagnosis (CAD) systems that can assist clinicians in their decision-making. The application of techniques to model brain connections as matrices (connectomics) is a promising avenue for understanding and analyzing how our brain works, but their medical application to assist in disease and disorder detection is a field that still needs development. One of the missing elements for this development is the establishment of a standard method for calculating connectomes from MRI data. In the absence of a standard, the analysis of how different connectome calculation processes, in combination with computational learning methods for the diagnosis of, for example, Autism Spectrum Disorders, is of particular interest to allow the possibility of clinical use of these systems [26]. The use of different connectome calculation methods and several computational learning methods on the ABIDE dataset [27] are studied separately. Additionally, the combination of AI and ML methods with new biomarkers offers more accurate models to diagnose and predict the evolution of neurological diseases. ML has also proven its efficiency and effectiveness in analyzing different types of medical imaging technologies, including Magnetic Resonance Imaging (MRI) [28,29], Single Photon Emission Computerised Tomography (SPECT) [28], X-rays, CT [30], Electroencephalography (EEG) [31], Cardiac magnetic resonance (CMR) [32], and so on. In the speed and accuracy of pattern recognition in other fields, ML is close or already has exceeded human performance, and thus this indicates the great potential of ML’s widespread application in healthcare and biomedicine. To the same end, we are also interested in AI tools for diagnosing and monitoring subjects with subtle patterns, such as Mild Cognitive Impairment (MCI), based on inexpensive, minimally invasive and easyto-acquire biomarkers. Thus, we summarize in the following sections a number of AI systems that automatically analyze cognitive abilities (memory, planning, constructional praxis, and semantic production) and biological signals, either in neuropsychological tests or activities of daily life. With the increased computational power and connectivity provided by modern devices and facilitated by the internet, smart technologies have pervaded daily life, especially in areas related to health and wellbeing. This allows for large amounts of data collection and processing. These novel devices usually come to the market as entertainment tools, such as virtual reality goggles, trackers, cell phones, and tablets. However, they can be used not only for gaming but also for rehabilitative functions in a clinical setting. Likewise, the number of virtual reality devices sold in the last five years has increased considerably (statistics available at: https://www.statista.com/). This is a favorable point for the development of new longitudinal monitoring applications based on these new devices. Applications using virtual reality and several trackers have begun to stand out in recent years [33,34]. This presents many opportunities to revolutionize not only healthcare but also the way it is delivered. 1.1. A summary of the paper We provide a detailed overview (see Fig. 2) of some conceptual sessions that have been published in the aforementioned areas within the 9th International Conference on the Interplay between Natural and Artificial Computation (IWINAC). Due to the relevance of the topic, DL models and applications are summarized in the first section of this paper. In particular, Section 2contains some applied contributions in DL, encompassing signal processing; image interpretation in medical, pictorial, and quality control domains; emotion recognition; and some AI contributions to the foundations of Deep Reinforcement Learning (DRL) and DL systems explainability. We mainly focus on three different aspects: stacked autoencoders with Multi-Layer Perceptrons (MLPs) [35], DRL [36], and the explainability of CNNs by the extraction of propositional rules [37]. In the following section, Section 3, we present a paradigm for devising new models and theories in AI as the mere observation of the behavior of biological systems. Bio-inspired models and systems are among the most successful methods for tackling hard combinatorial problems. In certain settings, effective solutions can only be achieved in an acceptable amount of time using this approach. Information Fusion 100 (2023) 101945 4 J.M. Górriz et al. Fig. 2. Taxonomy and overview of the main areas covered in this paper and their relationships, with emphasis on the topics covered within. This section includes original applications of these methods in a broad array of challenging problems in the fields of scheduling, routing, quantum computing and protein structure prediction, etc. showcasing the potential of the field. In this sense, biological inspiration has reached a stage of maturity that allows exploring issues as diverse as those related to image processing, group formation under efficiency criteria and emotion expressed through natural language. The studies available in the area of affective computing (AfC) cover a broad spectrum of research problems: from the development of appropriate methods for collecting emotion information from subjects, e.g. methods of data visualization and preprocessing, the evaluation of existing and development of new ML models, to practical applications, including the behavior of social agents in social networks, and the operation of desktop robots in hand disease rehabilitation (Section 4). These studies demonstrated distinct and valuable approaches to AfC AIrelated research. Emotions are essential in human communication and interaction. However, automatic systems for emotion recognition are still an unachieved objective in AfC. This section also introduces some applications focusing on (i) EEG for detecting emotions in the brain and (ii) virtual reality (VR) for eliciting and helping to recognize emotions in healthy and mentally impaired people. Section 5explores various applications of AI to the field of robotics, including hardware-related sensing and actuation aspects and higher levels of cognition that aim to make robots completely autonomous. There are two main research perspectives explored: approaches that provide specific algorithms for particular modules within a robotic architecture, and approaches that contemplate the architecture as a whole and seek the integrated operation of architectures that can lead robots to be able to address open-ended learning situations. A special focus is dedicated to computer vision, where artificial neural networks have been used extensively to process images and have a wide range of applications. However, there are still challenges to be overcome, such as reliability issues and lack of adaptability once training is completed. Robotic applications are also explored, particularly in terms of autonomy and natural interaction with humans. Section 6deals with new applications, devices or approximations to neurodegenerative, sensorial, cardiac, or emotional disorders. The section summarizes new neuroprosthetic approaches and models using EEG for understanding the brain, controlling exoskeletons, or detecting stress. Moreover, several ML applications in this field are assessed for retinal analysis, breast cancer identification and electrocardiographic (ECG) signal analysis for identifying different cardiac pathologies. Finally, Section 7gives additional details and insight on one specific (and relevant) biomedical application field: neuroscience. This field covers several aspects of signal processing and fusion techniques, image and bio-electrical modalities and biomarkers within signal analysis, computer-aided diagnosis and neurorehabilitation systems, precision medicine, and so on. The discussion in Section 8contains the results and outcomes of the present review paper, while conclusions are drawn in Section 9. 2. Explainable artificial intelligence in deep learning XAI is a hot research topic that aims to make AI systems transparent and trustworthy. Without explainability, developed methods are incapable of devising new theories and leading to incremental science [2]. For instance, a recent review [38] pointing to pitfalls and misconducts in the proposals of new DL approaches may represent this state of affairs. 2.1. Recent methods The most commonly used DL architecture is various types of CNNs, such as the noisy autoencoders [35], the hybrid with LSTM networks [39], Seq2Seq architectures [40], and ad-hoc vanilla CNNs [41]. The application of transfer learning based on publicly available and Information Fusion 100 (2023) 101945 5 J.M. Górriz et al. well-known pre-trained networks has also become a common firsthand approach to tackling diverse problems, as well as hybrid systems composing classical ML (namely Gaussian Mixture Models) and transfer learning of CNN approaches [42]. Another salient feature worth noting is the use of public data for the numerical experiments and demonstrations, which is a definitive step forward to open science [43]. In an autoencoder (AE), the input layer is the same dimensionality as the output layer. Between these two layers, an arbitrary number of hidden layers acts as an encoder and a decoder. Generally, the encoder achieves a transformation of the input to a higher or lower dimensional space. Subsequently, the decoder recreates the input data from the encoder’s output. Typical AE applications are data denoising [44,45], dimensionality reduction [46,47] and generative models [48,49]. Here, the authors of [35] aimed at testing whether a modified version of Stacked Denoising Autoencoders (SDAE) could perform better than the regular model. Unlike supervised learning, reinforcement learning does not require labeled input/output pairs. Typically, with this paradigm, an agent interacts in an unexplored environment to maximize its reward. During learning, a crucial question is the exploration∕exploitation dilemma. Specifically, the former is about acquiring more information in the unexplored territory, while the latter is about making the best decision given current knowledge in order to maximize cumulative rewards. Here, the authors of [36] presented an application based on the ‘‘Montezuma’s Revenge’’ game [50] in which the probability of determining a very long sequence of particular actions using random exploration is extremely small; thus, requiring methods with more directed exploration. The underlying model in reinforcement learning is a Markov Decision Process (MDP), whose objective is to maximize the future cumulative reward. With Atari games, each observation was an RGB image of size (210,160,3) where every action was chosen again for several frames since they are very similar. In addition, images were converted to grayscale with a smaller size (84,84). The last 𝑘images represented a single observation, so that an agent in the game could better understand crucial parameters, such as the direction or speed of objects in the game [51]. With the use of heuristic data, the authors reached good generalization. Essentially, they focused on whether the features of the state were rewarding, instead of focusing on whether the state was rewarding. Finally, the environments used were episodic; their end was triggered by the loss of a life or after winning the game. The deep neural networks (CNN) were implemented with the reinforcement learning library called ‘‘Coach’’. Before the advent of CNNs, a natural way to explain MLP classifications was to use propositional rules [52]. Andrews et al. introduced a taxonomy describing the general characteristics of all rule extraction methods [53]. Guidotti et al. presented a survey on black-box models with its ‘‘explanators’’ [54]. Moreover, Vilone et al. review the XAI domain by clustering the various methods using a hierarchical classification [55]. Many recent techniques involved learning interpretable patterns in the local region near a sample [56,57]. However, the main drawback of local algorithms is their difficulty in characterizing a phenomenon in its entirety. Moreover, many other techniques used in image classification visualize areas are mainly relevant for the outcome [58]. Explainability is a crucial concern that can be imputed to any trained neural network architecture. For example, with stacked AEs, propositional rules were generated in [59]. In [37], a technique for the rule extraction problem applied to a CNN architecture was proposed. The advantage of using deep autoencoders rather than MLPs with many hidden layers is that the former can produce more efficient feature representations than the latter. Nevertheless, the vanishing gradient problem affects the training through multiple layers. Therefore a possible approach to avoid this problem is to stack individually trained layers, i.e. deep SDAE [60]. Specifically, a small amount of noise was added to the input vectors; thus, the weight values of the autoassociative layers were determined by MSE minimization. In addition, a two-layer stacked DAE was proposed instead of a single-layer DAE. This approach was applied to four regression problems and three time-series datasets. 2.2. Applications of DL with XAI The key idea behind the rule extraction technique proposed in [37] is to transfer the feature maps learned by a CNN to the Discretized Interpretable Multi-Layer Perceptron (DIMLP) [61]. DIMLPs are specific MLPs from which propositional rules are generated, thanks to the precise localization of discriminative hyperplanes [62]. CNN networks were trained with the MNIST benchmark dataset of digits with two convolutional layers. Then, all the feature maps were transferred to a DIMLP network that was trained after compression of the maps by the Discrete Cosine Transform (DCT). In order to execute the rule extraction algorithm in a reasonable time, the DCT was only applied to a small number of low spatial frequencies. Finally, propositional rules were extracted from the DIMLPs, with rules showing in the antecedents the amplitudes of spatial frequencies in the images represented by the feature maps. Fig. 3 represents at the left the centroid of the samples activating a generated rule after applying the inverse DCT (belonging to class ‘‘0’’). In the middle and on the right are shown two centroids of two different feature maps linked to the same rule. It is worth noting that the feature maps detect some characteristic parts of the number ‘‘0’’. The predictive accuracy of the extracted rules was similar to the original CNN, when the MLP classifications agreed with the rule classifications (in about 97% of the testing samples). Thus, replacing a CNN network with many DIMLPs trained on their feature maps was an appropriate approach. Besides, it was possible to identify the relevant locations that contributed to the classification and reasoning behind the model. Nowadays, DL covers many tasks can be represented in a way amenable to a computation that may emulate human reasoning or perception. The ability to discern among pictorial styles is a subtle skill [41] that can be mimicked to some degree by DL architectures. Interestingly, the visualization of the confusion incurred by the trained DL classifier, as shown in Fig. 4, results in an excellent map of the relations among pictorial styles. This observation opens the door to new ways to exploit DL results. For instance, this new approach would allow us to visualize the relation among diverse neurodegenerative diseases on the basis of the confusion matrices of weak diagnostic tools. Another subtle perception task is the detection of emotions in speech, i.e. speech emotion recognition (SER) [39], whose importance will increase as the interaction of humans and cyber-systems becomes more and more natural in our lives. Besides attempting to model the brain, ML, especially deep artificial neural networks, have been inspired by the functioning of biological elements to mimic their properties. Following this principle, [63] proposes a different approach to continuous learning, a desirable property in neural network models that are not entirely developed nowadays. The proposal explores the stability-plasticity dilemma to avoid losing already learned information when dealing with non-stationary distributions. It is done by altering the learning algorithm with a new learning rate function in a competitive learning paradigm. Although the experiments are performed only on 2-dimensional binary images (as depicted in Fig. 5), they are promising and set a research direction for improving the system. 2.3. Application to image and video processing Although object detection and image segmentation are among two of the most successful DL areas of application, their limitations are far from being solved. The performance of the methods makes them Information Fusion 100 (2023) 101945 6 J.M. Górriz et al. Fig. 3. A centroid of samples activating a rule after applying the inverse DCT (left) and two centroids of two different feature maps linked to the same rule (middle and right). Fig. 4. The graph of relations between art styles, induced by the confusion matrix of the best DL architecture found, mimics the experts opinions and historical records. Fig. 5. Example of neurons adapting to different figures shapes. suitable to work with objects of considerable size in the images. Even though in many circumstances this limitation is not a problem, there are many cases in which small objects should be detected or segmented. Related to this problem, [64] proposes a test-time augmentation metamethod in which a pre-trained semantic segmentation model was used to generate high-resolution sub-images in which the different areas are identified. The final results are significantly improved although the execution increases given that the semantic segmentation method has to be applied several times. Fig. 6 shows an example of image segmentation. While the small object detection problem is inherent to image and video processing, there are others created by humans to take advantage of neural network-based systems. Adversarial attacks are input manipulations designed to cause false predictions in image classification models by adding imperceptible perturbations to an image. To defend against such attacks, [65] proposes a GAN-based pre-processing methodology. Instead of allowing direct processing of the image 𝑖, the Fig. 6. Example of image segmentation using test-time augmentation method as described in [64]. proposed method encodes the image into a latent vector 𝑧using a previously trained autoencoder and a GAN to generate from 𝑧another image similar to 𝑖. If 𝑖contains a malicious perturbation, the pre-process removes it. Once we can assume that the system is working correctly, some problems are difficult to define and, thus, particularly hard to solve. Anomaly detection is one of them because of its dependence on context. In [66], an object detection method is used to identify vehicles, track them, and obtain their trajectories and velocity vectors. The trajectories and velocities are compared with those of the nearest neighbors to obtain a context for defining the usual behavior and distance anomalies to that behavior. DL is used to solve timeand resource-intensive problems, and light versions of typical architectures can help solve real problems in real time. In [67], a simple yet effective approach is used to measure the Information Fusion 100 (2023) 101945 7 J.M. Górriz et al. Fig. 7. Architecture approach for live streaming latency estimation. Fig. 8. Sample frame of sperm imaging and schematic zoom of the parts of a normal spermatozoon: Head (a), middle-piece (b) and tail (c). end-to-end (e2e) latency in the live video streaming pipeline, from when the signal is generated in the production studios until it is played on the client device. The method is based on user-centric behavior by looking at the time the content is produced in the source context and comparing it to the current clock time at the playback device. Given a clock timestamp introduced in the signal at the production stage, we rely on an intelligent streaming latency measurement agent that first detects with YOLO that mark at the playout device, then uses optic character recognition (OCR) to convert the bitmap text in the clock to a string text, and finally, compares it with the real-time clock in the machine, providing real-time end to end streaming latency (see Fig. 7). The method, albeit simple, allows us to measure the latency of any playout device, as it does not rely on any in-band signaling but a human-centric behavior simulated by an intelligent measurement agent. On the other hand, the lack of labeled data is problematic when applying deep architectures in many fields. Here, solutions that provide synthetic data are very useful. One of these fields is sperm analysis, which has a central role in diagnosing and treating infertility (see Fig. 8). Traditionally, the assessment of sperm health was performed by an expert by viewing the sample through a microscope. To simplify this task and assist the expert, CASA (Computer-Assisted Sperm Analysis) systems were developed. These systems rely on low-level computer vision tasks such as classification, detection and tracking to analyze sperm health and mobility. These tasks have been widely addressed in the literature, with some supervised approaches surpassing the human capacity to solve them. However, the accuracy of these models have not been directly translated into CASA systems. The generation of synthetic semen samples to tackle the absence of labeled data is necessary. In [68], a parametric modeling of spermatozoa is proposed demonstrating how models trained on synthetic data can be used on real images with no need for the further fine-tuning stage. 2.4. Novel applications with miscellaneous technologies Of course, DL systems are becoming pervasive in the most diverse technological chores, from the basic signal denoising process [35] to the segmentation of images [69] to the interpretation of radiological images for the identification of specific diseases [70] (see also Section 7). Fig. 9. Susceptibility map selected for 15 day cumulative precipitation using Jenks method. A challenging application is the recognition of hand-made signatures [40] in historic documents, which are very noisy due to document conservation and the diverse conditions of the scanning process. Historical postcards also constitute noisy visual data and are very heterogeneous in structure. Deep image segmentation using already well-established U-net approaches [69] allows the extraction of handwritten data for subsequent analysis. In lane detection for automated driving tasks, the extensive use of temporal information embedded in encoder–decoder networks allows for increased robustness and accuracy [71]. Critical industries are also increasingly adopting DL approaches for quality control. In microelectronics [42], image data augmentation allows training a robust hybrid system including GMM and transfer learning of ResNet50 system for feature extraction. In aeronautics manufacturing, where thousands of fixation elements must be precisely detected, single-shot detectors have shown great performance [72]. Another field in which ML can be successfully applied is the prediction of catastrophic events like landslides. This is a problem traditionally tackled with conventional methods, of a deterministic nature, with a limited number of variables and a static treatment of these variables. In the first one, Landslide prediction with ML and time windows has proven to be a successful alternative for dealing with geoenvironmental problems. A feature engineering process allowed us to determine the most influential geological, geomorphological and meteorological factors in the occurrence of landslides. These variables, together with the landslide inventory, form a dataset to train different ML models, whose evaluation and comparison showed the best performance of the multilayer perceptron with an accuracy of 99.6%. The main contribution consisted of treating precipitation dynamically using time windows for different periods and determining the ranges of values of the conditioning factors that, combined, would trigger a landslide for each time window [73]. Furthermore, the use of ML models for the identification of high landslide-risk areas yielded probability values that can be represented as multi-temporal landslide susceptibility maps. The distribution of the values in the different susceptibility classes is done by comparing equal interval, quantile, and Jenks methods, which allowed us to select the most appropriate map for each accumulated precipitation (Fig. 9). In this way, areas of maximum risk are identified, as well as specific locations with the highest probability of landslides. These products are valuable tools for risk management and prevention [67]. 3. Bio-inspired applications, in general Bio-inspired Computation (BIC) is a branch of AI based on behaviors and characteristics of living beings, particularly the inheritance and behavior of swarms. Genetic Algorithms (GA) may be considered the flagship of the kind of algorithms relying on inheritance and adaptation to the environment, but other approaches of this type, such as Information Fusion 100 (2023) 101945 8 J.M. Górriz et al. Differential Evolution (DE) or Genetic Programming (GP), have a long track of success as well. Furthermore, swarm algorithms such as Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC) or Ant Colony Optimization (ACO), among others, introduced new features borrowed from the emergent behavior of swarms without central control, which makes them more suited to some problems and able to adapt to both discrete and combinatorial, single and multiobjective, or unimodal and multimodal problems. The boom in BIC continues to occur in many cases without a thorough analysis of what is really new in each new bio-inspired metaheuristic approach and in comparison to the well-established and widely used methods of evolutionary computation and swarm intelligence optimization methods. Many papers are tailored to show that the new method performs better than others on a set of benchmarks or in a particular application by adjusting the defining parameters to those benchmarks or that application, while the other methods used in the comparison are adjusted to their standard values or to values reported by authors in related applications, as noted in [74]. Thus, researchers in this field must be self-critical of the rise of these new solutions, contrasting what is really new and what is included in other traditional search algorithms or what novelty a new bio-inspired metaheuristic brings. BIC algorithms are considered weak methods since the only knowledge needed in the problem domain is embedded into the fitness function. However, their flexibility allows the designers to introduce domain knowledge, usually by means of local searchers or greedy algorithms, but also with specific recombination or variation operators, or even coding schemes that are specific to the problem. The resulting approaches, termed Memetic Algorithms (MA), are among the most outstanding methods for many complex problems. Nowadays, biological inspiration has reached a stage of maturity that allows exploring issues as diverse as those related to image processing, group formation under efficiency criteria, and emotion expressed through natural language. In the following subsections, we summarize several main contributions in the field and, as pointed out, they include original applications of bio-inspired algorithms such as GA, MA, DE, GP or ACO, to a number of industrial and scientific problems of current interest, such as Quantum Computing, Protein Structure Prediction, Complex Scheduling and Learning Heuristics, and so on. 3.1. Quantum computing Quantum Computing (QC) is an emergent technology with that promises to solve many problems intractable with classic computational methods. However, the development and execution of Quantum Algorithms raises a number of specific challenges. One of these problems is distributing the quantum operations over a given quantum hardware to minimize the risk of decoherence, and satisfying a number of constraints imposed by the hardware structure, which is termed the Quantum Circuit Compilation Problem (QCCP). This is one of the main issues in QC. This problem was addressed in [75], where the authors exploit GA to solve the QCCP derived from the so-called Quantum Approximation Optimization Algorithm (QAOA) applied to the MaxCut constraint satisfaction problem, obtaining competitive results with the state-of-the-art. Fig. 10 illustrates the main steps of this procedure. 3.2. Complex scheduling Companies in any modern industry need sophisticated scheduling systems to meet their production on time, subject to the best use of human and energy resources. This fact poses harder and harder scheduling problems that require innovative solving methods to reach satisfying solutions. Given the extreme difficulty of most of the new scheduling problems of industrial interest, bio-inspired approaches such as swarm and local search algorithms are, in many cases, the most reasonable choice. A number of papers from the BICA session deal with Fig. 10. Example of MaxCut instance (a) and one possible solution over the quantum hardware with 4 qubits (b left) represented by a quantum circuit (b right). Each binary gate must be executed on adjacent qubits, for this reason some 𝑠𝑤𝑎𝑝 gate (that represented by ×in the extremes) must be inserted. Fig. 11. AlphaFold2 structure prediction of protein Q31R69 (Synechococcus elongatus, 116 amino acids) with two helices and several beta sheets. The more blue, the greater the confidence in the prediction. scheduling problems arising in different industrial environments. For example, in [76] the authors propose an accurate model for virtual resources scheduling in a cloud, which is based on the quality of service requirements and pay-per-use basis and solved by GAs. In many real-life problems, the duration of the tasks is uncertain a priori. Therefore a robust schedule must remain feasible for any actual processing time. This fact was considered in [77,78]; in the first case, uncertainties are modeled by intervals, while in the second the authors propose the use of fuzzy numbers. In [77], the authors tackle the classic Job Shop Scheduling (JSP) with makespan minimization by an ABC algorithm, and in [78], the confronted problem is Flexible JSP with energy optimization by means of MAs. 3.3. Protein structure prediction Protein structure prediction (PSP) is a challenge in bioinformatics, since structure determines protein function. The authors in [79] analyze the advantages and drawbacks of a number of PSP strategies, considering the recent DL-based methods of RoseTTAFold and DeepMind’s AlphaFold2, as well as energy minimization methods. The latter alternative includes an MA based on DE and Rosetta’s fragment replacement technique for PSP [80,81]. Fig. 11 shows an example of structure prediction using AlphaFold2. 3.4. Learning heuristics Heuristics for problem-solving are normally defined by humans exploiting the knowledge from experts in the problem at hand. This is the case, for example, of scheduling priority rules that are often applied when the time available to build a schedule is limited (realtime) or when the problem is dynamic and tasks must be scheduled online. However, the automatic construction of such rules may be the best option. This approach is followed in [82,83], where the authors exploit GP to evolve rules for the Travelling Salesman Problem (TSP) and the Unrelated Parallel Machine Scheduling Problem, respectively. This is a suitable approach because scheduling rules are arithmetical expressions that can be naturally evolved by GP. Moreover, GP provides Information Fusion 100 (2023) 101945 9 J.M. Górriz et al. Table 1 Fitness for GA and Bacteria strategies. Iterations GA Bacteria 1 0.524 𝟎.𝟓𝟐𝟏 200 0.425 𝟎.𝟒𝟐𝟒 400 𝟎.𝟒𝟏𝟔 0.418 600 0.416 𝟎.𝟒𝟏𝟓 800 0.416 𝟎.𝟒𝟏𝟒 1000 0.415 𝟎.𝟒𝟏𝟑 1200 0.415 𝟎.𝟒𝟏𝟏 1400 0.415 𝟎.𝟒𝟏𝟎 1600 0.414 𝟎.𝟒𝟏𝟎 1800 0.413 𝟎.𝟒𝟏𝟎 2000 0.413 𝟎.𝟒𝟏𝟎 a variety of rules, which may be further used to build ensembles, an approach considered in [83], where the authors show that ensembles may produce much better solutions than single rules at a reasonable cost. 3.5. Educational and social applications Group formation is an interesting challenge for several reasons. First, different criteria must be met according to the group objectives. In the specific case of group formation of students to improve the results of the learning process, the accepted general condition is that the composition in every group is as heterogeneous as possible. This means that the greater the difference between individuals in the group, the greater their learning capacity. On the other hand, different groups should be as similar as possible, which means that the smaller the differences between the different groups, the more overall learning capacity improves. One possible approach has been the use of lexical availability techniques, to consider the level of knowledge of students in different specific topics. An interesting alternative is to consider the metaphor of the behavior of bacteria. These organisms perform as a population that is always searching for an optimum condition for survival. In the particular case of students, the success criterion, which represents the achievement of academic goals, is similar to the survival criteria of a population [84]. Table 1 shows how fitness evolves when genetic algorithm (GA) and bacteria strategies are used. It can be seen that with bacteria, solutions are better than when we use genetic algorithms, and the stationary state of the best value is obtained with a smaller number of iterations. In recent years, emotions (see Section 4) have emerged as a relevant topic in the field of social sciences, particularly when emotions can be recovered from the lexical availability of speakers. The key reference can be found in [85], explaining the adaptive characteristic of emotions and identifying the eight primary ones. Fig. 12 shows the emotions wheel in the structural model. Each emotion is associated with a color. According to the intensity of an emotion, the corresponding color intensifies. Emotions are more intense when they approach the center of the wheel, and they may evolve from a particular state to a different one. In the figure, we can see that, if trust intensifies, then it can turn into admiration. On the other hand, if trust diminishes, it may turn into acceptance. The lexical availability methodology allows us to recover the most used vocabulary by a population sharing a particular context. This proposal is aimed to predict emotions, grouped in interest centers. Data collected for the experiments considered eight different countries and a total population of 13,918 individuals. Once again, the combination of a classic approach (the lexical availability methodology) and neural networks allows us to detect emotions in a specific context or historical reality. The training of neural networks with these data has permitted us to predict how emotions can evolve depending on particular geographical and social parameters. The central idea is to collect data to describe emotional states, which is a similar approach to that considered above, related to students’ learning processes [86]. Fig. 12. Plutchik wheel. 4. Interdisciplinary research in affective computing The use of virtual agents that support human tasks has increased rapidly in recent years. This stream of research has evidenced that computing principles inspired by natural processes can automatize social interactions between virtual agents and humans. For a long time, technology has been insufficient in developing systems that relate to human beings in a natural human way [13], but nowadays the relationships between human and virtual agents are feasible. To do so, this involves computational modeling of social sowing [14], emotion recognition [15–18], sentiment analysis [19] and human attention and performance monitoring [15,20–22]. Research in the field of affective computing (AfC) that is aimed at the development of systems that recognize, interpret, process or simulate human effects [87,88], addresses a number of research questions: •How can emotions be classified? •Which data can be used as a source for inferring emotions? •How can emotion-related data be collected to ensure that the prepared dataset covers a variety of emotions yet has ecological soundness? •How to prepare emotion prediction models using ML and statistical methods? •In which ways and to what extent can emotion-related information be practically used in computer systems? Different studies in the area of AfC address only selected questions, and the answers they offer vary depending on the multidisciplinary composition of the research teams, as well as intended specific applications. Emotions can be defined as positive or negative experiences associated with a particular pattern of physiological activity. Much work has studied emotions on different physiological variables such as electroencephalographic (EEG) recordings, since the brain is considered to be the source of all reactions to any external stimulus [89]. People infer other people’s basic emotions primarily from facial expressions and tone of voice, whereas a deficiency in this ability would lead to a misinterpretation of social cues [90]. Dynamic onscreen stimuli do not evoke in subjects the feeling of ‘‘being there’’, Information Fusion 100 (2023) 101945 16 J.M. Górriz et al. Finally, uniform embedding techniques have limitations for the reconstruction of the phase space of nonlinear time series whose dynamics is not completely known, so new embedding techniques based on non-uniform methodologies can help in this problem. This can be applied to electrocardiography databases. For the uniform reconstruction, Average Mutual Information can be used to find the time delay while False Nearest Neighbor and Average False Neighbor can be used to find the attractor dimension. Non-uniform embedding provides a better quality in the reconstruction of the phase space. 6.5. Bio signal analysis in neuromotor disorders Neuromotor disorders might have their causes on either pre-motor or primary neurons, on bulbar midbrain areas, on motor units, or in the muscular fibers [25]. These disorders, such as PD, Amyotrophic Lateral Sclerosis (ALS), Huntington’s Chorea, or Myasthenia Gravis (MG), name the aggregate of symptoms that are the result of the neuromotor system affected structures. Most of them do not have a clear etiology or effective treatment yet, but some treatments might successfully improve the functional motor capacities and living conditions of patients. PD is the most prevalent neuromotor disease among all, quantify its incidence in 15 cases per 100,000, with a prevalence ranging from 100 to 200 cases per 100,000 [175]. PD has a major impact on the daily activity behavior of patients, resulting in difficulty in walking, handling objects, resting tremor, facial rigidity, etc. as well as non-motor symptoms (e.g. cognitive decline, depression, etc.) which are also challenging PD patients’ ability to lead an independent life [25]. Having this panorama in mind, it is convenient to evaluate the neuromotor function of PD participants. Specifically, the interest lies in measures of potential changes in the functional behavior of patients after being submitted to non-invasive stimulation in order to induce more stability in their neuromuscular activity in the shortcoming period after. One type of stimulation consists in the application of auditory stimuli which might compensate the lack of endogenous oscillations at the basal ganglia due to neurodegeneration [176]. The oscillations in the basal ganglia of PDPs typically shift down to frequencies in the beta band, 14–30 Hz, characteristic of hypokinetic states or dopamine deficiency, as well as to <10 Hz frequencies, associated with tremor, dystonia and sleep. Neuroacoustical Stimulation (NAS) consisted in the application of binaural beats following the protocol described in [177], from the beatings of a pure tone applied to each ear corresponding to the two-tone frequency differences. In the two first cases discussed in the section, NAS used a sinusoidal signal of 154 Hz through the left ear, and another sinusoid of 168 Hz through the right ear, which induce a binaural perceived tone of 14 Hz. To analyze the effects of NAS on PDPs in longitudinal studies, two approaches can be taken. The first approach [178] concentrates on the assessment of the effects of NAS on the motor activity of PD patients with a smart watch while carrying on movement tests consisting of exercises such as walking a short distance, raising from an armchair, extending and flexing arms and wrists, and so on. Triaxial accelerometer signals captured tremor magnitudes in the 3.5–7.5 Hz band, tremor endurance within the resting periods between exercises, and bradykinesia in the 0.5–3.5 Hz band during pronation and supination exercises. The results from two PDPs (male and female) and five controls (two males, three females) presented different statistical distributions between PDPs and controls regarding tremor and bradykinesia during an eight-week period. The distributions of PDPs produced higher medians and wider dispersion than their control counterparts. Although these were preliminary results and cannot be attributed any statistical significance due to the sample size, they constitute promising advances to be extended in future studies, as it was put forth during the debate after the presentation. The second approach analyzes the results of NAS on the phonation of the same participants [179]. In this case, the vowel sequence [𝑎∶→ 𝑒∶→𝑖∶→𝑜∶→𝑢∶] was used as the benchmarking test. Their Fig. 22. Timely evolution of 𝜆: sham rTMS case. performance was evaluated on a set of four recording sessions after NAS, separated by a week between recordings. The features analyzed were the logarithm of the Vowel Space Area, the Formant Centralization Ration, the Vowel Articulation Index, the Second Formant Span, the normalized First and Second Formant Spans, the modulus of the Normalized Formant Spans, and the Absolute Kinematic Velocity of the jaw-tongue joint. The male participant tests manifested positive evolution on the Second Formant Span, whereas the female participant showed positive evolution in all the features, except in the Normalized First Formant Span. The male participant showed improvements in the Cepstral Peak Prominence and in the tremor on the EEG-related 𝜗band (4 − 8 Hz). The female participant showed improvements in the Energy Profile distribution. Another research topic is to explore the effect of active/sham repetitive Transcranial Magnetic Stimulation (rTMS) [180] on hypokinetic dysarthria in PDPs [181]. In this case, the phonation features used in the comparative analysis were the Jitter, Shimmer, Cepstral Peak Prominence, and the amplitude distributions of the EEG-related 𝛿 (0–4 Hz), 𝜗(4–8 Hz), 𝛼(8–16 Hz), 𝛽(16–32 Hz), 𝛾(>32 Hz), and 𝜇(8–12 Hz) tremor bands, extracted from a sustained vowel [a:]. The resulting features’ densities were compared using the normalized Jensen–Shannon distances with respect to a set of 16 normative controls of both genders. The data were extracted from a recording previous to stimulation and four recordings after stimulation, spaced in time covering a three-month period. The results showed a corrective effect in the active stimulated participant across the feature set except for Shimmer, and positive effects also, although not so clearly distinguishable in the sham-stimulated participant (see Fig. 22). A potential interpretation pointed to the possible benefits of speech exercises having also a possible rehabilitative effect on the sham case. A fourth study, [182], reveals the differential behavior of the amplitude distributions of the {𝛿, 𝜗, 𝛼, 𝛽, 𝛾, 𝜇}bands from the vocal fold strain tremor, extracted from a sustained vowel [a:] by comparing their entropy contents on two PDPs (one male, and one female) with respect to two normative controls (one male, one female) [183]. This preliminary investigation provides some useful early insights regarding apparent differences between PD and control participants, in the sense that entropy showed to be much larger in PDPs with respect to controls for all the EEG-related bands studied. To summarize, the session showed a compact structure on a neat connecting narrative, with four contributions analyzing the issue of phonation instability in PD under different but related scopes, including NAS and rTMS looking forward to rehabilitation. This was put forward in the discussion, together with the need of benchmarking databases specifically designed to accomplish this specific kind of study at a statistical significance level. The perspective of studying phonation instability in relationship to EEG-related band activity could open interesting new research lines offering insights on the indirect estimation of neuromotor activity in upper motor areas by means of speech and phonation (see Fig. 23). Information Fusion 100 (2023) 101945 17 J.M. Górriz et al. Fig. 23. Timely evolution of 𝜆: active rTMS case. 7. Artificial intelligence in neuroscience Neuroscience has been one of the most benefited areas from the advances in AI [184]. The use of different machine learning algorithms to explore and discover patterns related to specific neurological conditions, disorders, or diseases constitutes an important added value to traditional methods. Among the applications being clearly benefited from ML and AI algorithms are those related to NI and neurophysiological or speech signals. In the case of NI, AI allows the automatic identification of patterns linked to a specific disorder and useful in a differential diagnosis task [185]. In the same way, these techniques may provide relevant exploratory information regarding the development of the disease and thus, for the personalized treatment towards the paradigm of precision medicine [186]. A large number of conditions can be monitored through NI techniques in conjunction with ML approaches. This includes ailments such as Developmental Dyslexia, Autism, or Schizophrenia, or other degenerative conditions such as PD or AD, which cause cognitive function to decline and never recover. Moreover, neuronal damage derived from other circumstances, such as respiratory disorders that can produce hypoxia, can also be examined with similar techniques. The primary relevance of these studies is the worldwide increase in the prevalence of neurological disorders, and an early diagnosis is crucial to slow the progression of these diseases [28]. 7.1. AI supports NI analysis Different NI modalities play crucial roles in the study of neurological disorders. In fact, these noninvasive techniques provide highly relevant information that assists clinicians in diagnostic decisions. This information is extracted and analyzed in Computer Aided Diagnosis (CAD) systems [187], which include AI methods in the different stages of the NI processing pipeline. Registration methods constitute a critical step that may determine further analyses. These methods are also benefited from ML methods. For example, the spatial registration of brain scans to a common reference space [188,189] does not only allow direct comparisons voxelwise but also increases interclass separation. With this and similar processing applied, CAD systems based on ML can more easily identify patterns that explain how the human brain works and how it deviates from typical aging trajectories towards degenerative disease [190]. In the evaluation of patients with dementia, their brain scans may be significantly altered in terms of morphology as a result of neurodegeneration and thus undergo greater changes to their shape during the warping process to a normative template or atlas. Moreover, in the case of PD, FP-CIT SPECT scans depict dopamine transport concentrations that are localized almost exclusively to the striatum with relatively little activity elsewhere in the cortex or cerebellum [191]. On the other hand, the changes in FP-CIT SPECT scans with a spatial registration Fig. 24. Schema showing the results of a spatial deformation when applying either intensity preservation of the concentration or the intensity preservation of the amount. Fig. 25. Architecture of the siamese network used to compute the asymmetry between brain regions. that adopted an intensity preservation strategy are assessed with a novel dimensionless factor that uses the differences between affine and non-linear spatial registration in [188]. When applying the intensity Preservation of the Amount (PA), areas expanded during the warping process are correspondingly reduced in intensity. Similarly, warping with the intensity Preservation of the Concentration (PC) also lowers mean values (see Fig. 24). This increases the interclass separation between Healthy Controls (HC) and patients with PD, but at the cost of losing morphological information [153]. ML and DL techniques can be also used for exploratory analysis and to determine morphological differences in brain structures. Leveraging a relationship to morphological analysis and inference maps is addressed by a DL architecture based on siamese networks to evaluate functional differences between brain regions to discern between HC and PD [190]. In summary, this methodology consists of the union of two identical neural networks sharing common weights that are updated simultaneously through an error back-propagation process. The key feature of this framework is that the outputs of both subnetworks (i.e., the embeddings) are compared according to a distance measure that represents the asymmetry between brain regions. Fig. 25 depicts the architecture of the siamese network proposed. Following this schema, the embeddings extracted from the outputs of the siamese network are used as input of a linear Support Vector Information Fusion 100 (2023) 101945 18 J.M. Górriz et al. Fig. 26. Projection over the first two dimensions of the embeddings associated with controls (blue) andPD (red). Machine (SVM) classifier. Fig. 26 includes a two-dimensional representation of the embeddings when comparing subjects from HC and PD classes. 7.2. AI supports automatic and early diagnosis/prognosis One of the diseases with the highest number of proposals for CAD systems is present in PD. These systems are not only based on image data but also on clinical information or speech signals. An example of these CAD systems is [28], which combines multiple input data sources that individually would lead to poor classification rates and high variability. Nevertheless, on the basis of information extracted from FP-CIT SPECT and MRI images, this work preserves the performance of the CAD system and minimizes its variability. Although the use of FP-CIT SPECT scans is one of the most reliable clinical tests for PD, it would be interesting to detect the disease using other less expensive alternatives such as MRI. With this in mind, [189] proposed the statistical analysis of significance maps by means of parametric and non-parametric approaches. Experimentally, MRI and FP-CIT SPECT scans from 40 HC and 40 PD participants have been compared by means of parametric maps obtained using the Statistical Parametric Mapping (SPM) and non-parametric maps using the Statistical Agnostic Mapping (SAM) [185]. Another prominent application of ML is the prediction of a disease progression. This provides a personalized prognostic for a patient result, which is essential for clinical practice and can be seen as a prediction of clinical markers over time. For example, [192] addresses this for PD using a non-linear decomposition of FP-CIT SPECT scans and an unsupervised ML schema. The authors model the composite variables with SVM to perform two different tasks: a differential diagnosis (i.e. classification) and a disease progression analysis (i.e. regression) using a longitudinal dataset. Whilst their Isometric Mapping (ISOMAP) approach decomposes the input dataset into a more uniformly distributed coordinate space, the results obtained are related to the intensity in the tails of the striatum. A Principal Component Analysis (PCA) approximates the asymmetry of the image. Two works addressed the quest for new biomarkers for early diagnosis of PD using speech signals. In [193], formant measures were combined with Convolutional Neural Networks (CNNs). The study used sustained phonations of the vowel /a/ from two speech corpora (Patient Voice Analysis dataset and Saarbrücken Voice Database) to train and test a CNN. The input was composed of six normalized formant features, and the CNN had 150,000 trainable parameters. The best results were obtained using the 𝑒𝐹 1–𝑒𝐹 2 formant feature set for a speech segment of 1 second and the 𝑒𝐹 2–𝑒𝐹 3 set for a 2-second segment. Fig. 27. Architecture of the CNN proposed in [194]. In [194], a new architecture based on a CNN with Auditory Receptive Fields (ARFs) in the convolutional layers was proposed (see Fig. 27). The input was an 800 ×200 spectrogram based on a 9-pole adaptive lattice-ladder linear prediction coding algorithm, calculated for 2-second speech segments. The ARF-CNN approach was tested on a small dataset of 6PD participants and 6healthy controls and showed competitive results with handcrafted features. Other disorders can also be tackled with AI. This is the case of the Smith-Magenis syndrome (SMS), a rare disease with low prevalence that involves intellectual deficits and motor and speech delay [195]. A study [196] evaluated the speech and language abilities of individuals with SMS using subharmonic components of the voice in the cepstral domain and found that individuals with SMS have significant delays in their speech and language development compared to typically developing peers. AD is also addressed using speech as a biomarker of the disease in [197]. The paper presented different rates related to Automatic Speech Analysis (ASA) as a non-invasive, preclinical discrimination between healthy aging and Mild Cognitive Impairment (MCI) with around 90% accuracy for ASA evaluation of reading tasks. Inspired by the biological attention mechanism, [29] proposes a lightweight attention-based CNN (ConvNet-CA) for discriminating abnormal brains from healthy brains based on patients’ Magnetic Resonance Imaging (MRI) scans. Features are first extracted by convolutional layers and summarized by max-pooling layers. An efficient channel-wise attention mechanism is utilized to learn the importance of each channel in feature maps. This process makes the model focus on the features that are relevant to a given classification task. Compared to the popular state-of-the-art CNNs, ConvNet-CA has proved efficient and effective in learning meaningful features with a shallow network architecture, achieving a multi-class classification accuracy of 94.88%± 3.64%. The model is evaluated on a dataset with only 197 scans in total, demonstrating the powerful representation capability and the model robustness to a small dataset. Special attention should be given to assessing MCI [198–200] since it is considered the stage between the mental changes that are seen between normal aging and early stages of dementia. Indeed, MCI is one of the main indicators of incipient AD among other neuropsychological diseases [201]. Diverse types of tests have already been developed, such as biological markers, different imaging modalities, and neuropsychological tests [202]. While effective, biological markers and imaging modalities are economically expensive, invasive in some cases, and require time to get a result, making them unsuitable as a population screening method. On the other hand, neuropsychological tests have reliability comparable to biomarker tests and are cheaper and quicker to interpret. Classical neuropsychological include graphic tests (Rey-Osterrieth Complex Figure test, Clock test, Trail Making test, etc.) [203–205] or tests based on oral production (categorical verbal fluency test, phonetic production test) [206]. They require very few resources for their application. However, the need for automation and a more objective assessment are motivating the development of new paradigms capable of monitoring daily behavior [207–209], or defining interactive applications through virtual reality [198,210]. Several Information Fusion 100 (2023) 101945 19 J.M. Górriz et al. tools for diagnosing and treating MCI that are inexpensive, minimally invasive, and easy to administer are now reviewed. One of the best-known methods for detecting MCI is the Clock Drawing Test (CDT), which is an easy method for looking for dementia symptoms, including those of AD, and is frequently used along with other screening exams. As stated in Section 2, DL architectures have demonstrated their usefulness in the extraction of visual patterns and in the classification of image data. Thus, [211] analyzed an automatic system for diagnosing Cognitive Impairment (CI) based on the paperand-pencil CDT. Two models are compared, one based on DL and another on traditional ML. The architecture of the DL model is a Convolutional Neural Network, whereas the traditional ML model uses Partial Least Squares (PLS) as the feature extraction method and SVM with a linear kernel to classify the extracted features. These experiments yielded good performance based solely on the cognitive test, and its accuracy is validated by means of an approach based on resubstitution with upper bound correction. This demonstrates the effectiveness of ML methods for CI diagnosis, especially in resource-poor areas. Many studies introduce ML and other AI techniques for identifying early cognitive deficits in adults in general [198], or for studying the results of applying these tests in particular [198–200]. Even some specialize in particular types of tests, such as graphic tests (Rey-Osterrieth Complex Figure test, Clock test, Trail Making test) [203–205] or tests based on oral production (categorical verbal fluency test, phonetic production test) [206]. Other works considered the automatic analysis of the Rey-Osterrieth complex figure (ROCF). Fig. 28 shows two examples of handmade drawings to show the complexity of the problem. [212] presents a neural network based on a Siamese architecture to assess the patient directly from the ROCF copy drawing. The results are not extraordinary due to the complexity of the problem since they are trying to diagnose from a single test when not all variants of MCI are related to the executive functions assessed by this test. Therefore, in [213], a more practical approach tries to obtain an automatic score without entering into the final assessment. This task is also complex because the final score is the sum of the partial contributions associated with the ROCF’s different components. In addition, there is not a large dataset to apply basic DL techniques, so they propose using Recursive Cortical Networks, which require fewer examples for training and have given excellent results in breaking captcha. This is a very early paper, so only very initial results are reported. For oral production analysis, [214] proposed transfer learning methods that address data scarcity and involve the least amount of customization steps. They analyze language in two separate modalities: speech and linguistic information. For the first modality, they employ audio files, and for the second one, transcripts are extracted from the audio files. The proposed methods consist of feature-based classifiers and pre-trained models such as ResNet152,HuBERT,BERT, and RoBERTa. With this, the authors find that transfer learning approaches outperform conventional classifiers and the proposed baseline model. In general, they improve important aspects of the process without necessarily editing prepossessing steps, domain knowledge, or transcripts. The assessment of the cognitive aspect of spatial cognition is the starting point of [215]. Spatial cognition is a function that strongly contributes to adaptation and can be impaired by brain injury. Assessment of these impairments is usually run with paper-and-pencil or behavioral tasks: this paper introduces an enhanced version of the Baking Tray Task, that generates new data, related to time, sequence, and so on. The authors show how AI can be applied to the assessment of spatial cognition, indicating that it can effectively analyze these new data thus leading to a more comprehensive assessment of spatial cognition. Due to this need for early diagnosis, or at least for evidence, using a sufficiently inexpensive and non-invasive method for screening, other types of techniques are also investigated. These techniques are not based on neuropsychological tests but on sensing the human being to detect characteristic signs or patterns of impairment (or, at least, Fig. 28. Two ROCF copy drawings. non-normality or suspicion of it). We include here work related to the analysis of physiological signals (such as EEG [216], wearable biometric devices [217,209], or even different NI modalities [218–221] although we are looking for non-invasive and inexpensive tests), and daily life behavior (such as patterns of activity at home [208,209] or semantic and acoustic patterns of speech [222–224]). [225] addresses a very impacting pathology: the AD that is one of the most common forms of dementia. Authors propose to complement medical procedures for AD diagnosis based on biochemical markers, medical images, and psychological tests with the analysis of resting state EEG. It has the advantage to be an inexpensive and non-invasive technique to collect information on brain activity. Authors show how to elaborate these signals to detect AD precociously. Finally, there are many problems associated with working with data taken from different populations and with different models [226]. One such problem, for example, is the absence of complete patient data caused by a wide variety of reasons, which imputation algorithms can alleviate. [227] work with an incomplete database of semantic category test scores (and personal and socio-demographic data) that is used to assess MCI, and attempt to complete it using imputation mechanisms that follow two strategies: assuming that these individuals would have scored poorly if they had taken the test, defining a ceiling score, and multiple imputation by fully conditional specification. The study concludes that, although ceiling imputation can be useful when values are lost in a missing at random situation and the correlation between values is clear, multiple imputation is completely unbiased in all aspects analyzed. 7.3. AI and autism spectrum disorder technology Autism Spectrum Disorder (ASD) also benefits from new technological advances. Finding markers for autism is one challenge that Information Fusion 100 (2023) 101945 20 J.M. Górriz et al. Fig. 29. ApEn: the stress-aware pen. could be resolved by technological solutions, to allow objective tests for diagnosis, classify disease severity, and indicate prognosis [228,229]. Moreover, information and communication technologies (ICTs) lead to an improvement in the conditions of support and accompaniment of the sufferers [230–233]. For example, an estimated 33% of people with ASD better retain information presented through computers or tablets [234]. Recent work on ASD device development, ML, voice recording, and robot-supported education solutions is the stress-aware pen (ApEn) [235]. It is shown in Fig. 29), and it is designed to detect stress-related behaviors by sensing the handwriting and hand-holding pressure, especially for Children with ASD. Two Flexiforce sensors are embedded to detect pressure through the pen lead and the pen body. To draw children’s attention to their stressrelated behaviors, three vibration motors and one LED light are used to provide feedback, as shown in Fig. 30. Further study is expected to personalize the stress measurement and the feedback mechanisms of the pen as well as the communication of this stress to the children and the parents via appropriate machine learning algorithms. It was developed to study stress-related behaviors in the natural environment and explore how to enhance everyday objects for stress detection and regulation. Differently from the approach with physiological signals, behavioral data are collected for immediate feedback. Although the design focuses on children with ASD, ApEn can be applied to different scenarios. Further research will establish the appropriate interaction design and will explore how to make the pen a connected object to better support stress detection and reduction. One important topic in ASD management is monitoring the person’s state. An acquisition platform especially developed for people with ASD is presented in [236], which development is reported in [237,238]. Fig. 31 shows a picture of the different devices that make up the platform: a soft wristband to measure heart rate, body temperature, and motor activity; a system to acquire environmental stimuli such as luminosity, environmental temperature, relative humidity, and atmospheric pressure, and a device with a 360-degree camera that measures the number of people and optical flow. Finally, an Android smartphone that manages the platform shows relevant information in the interface and also acts as a sound analysis sensor. All the information collected by the platform is stored in a remote database. The heart rate (HR) values remain similar in the four groups of activities. The project explores the correlations of accelerometer values and body temperature with the intensity of movements, by gross psychomotor tasks, such as obstacle courses. These values are further correlated to the environmental parameters to better support the engagement and enjoyment of these special users. An interesting trend in technology-supported research in ASD is finding digital biomarkers present in the phonation of people with Autistic Disorder and intellectual disability, with the purpose of better understanding the syndrome and being able to develop specific tools that contribute to improving their quality of life [161,239]. The mobile App Biometrophon allows a longitudinal study extracting up to 72 features from each phonation segment, including perturbation features as jitter, shimmer, and harmonic noise ratio, as well as a cepstral description of the glottal source. The combination of Physiological Tremor Amplitude, Neurological Tremor Frequency Flutter Tremor Amplitude, and Global Tremor Amplitudes, summarizing mean square root of tremor in all bands is the beneficial multimodal combination of phonetic signals. The tremor features provide information on the presence of defects, instabilities, or feedback problems in the neuromotor system linked to the activation of the musculus vocalis. The results shown in Fig. 32 are based on three samples from participant M1, corresponding to a male born in 1973 (48 years old at the time the recordings took place), who presents an intellectual disability, psychotic episodes, and epilepsy, with a CARS of 40 and a DEX of 29, separated on a week interval. Valid utterances of a sustained [a:] lasting more than 400 ms were selected from the recordings, corresponding to 12 valid segments during the two first sessions, and 18 valid segments during the third session. These estimations were compared with the normalized EDA value recorded by the wristband E4 using correlation. The study described in [239] of sustained vowel utterances from an ASD participant enables obtaining longitudinal estimations of vocal fold tremor, potentially associated with neurological excitement in performing vocalization tests. Relative relevant correlations have been found between NTA and FTA band tremor and surface skin conductance. The apparently controversial correlation results from the three recording sessions studied pose an important challenge in determining the valence of increasing neurological excitement produced during test performance. 7.4. Information fusion in NI using DL Combining data obtained by different methods is one of the most popular applications of DL. In the field of NI, different data sources can be combined to generate a stylized version to fuse two images from different sources. In this context, different data sources are sometimes available that provide structural or functional information, which, although they are usually analyzed separately, can be used together. features extracted from structural and functional NI to improve classification performance in CAD tools. Thus, it is possible to take advantage of Positron Emission Tomography (PET), generating a new image containing structural and functional information. For instance, the principles of neural style transfer to combine MRI and PET information, generating a new image containing structural and functional information [240]. The usefulness of this method has been evaluated with images from the Alzheimer Disease Neuroimaging Initiative (ADNI), which is characterized by the impairment of memory and one other superior cognitive function, which is frequently the language function. AD is the most common cause of dementia. Using the combination of the above techniques generates a new mixed-mode image (Fig. 33). Images from the ADNI have been used, demonstrating that using the new mixed mode image outperforms the classification accuracy obtained by individual MRI or PET images. 7.5. ML for neurophysiological biomarker analysis In a similar way that ML provides new opportunities in the field of NI processing, the analysis of neurophysiological signals is also benefited by them. One of the most prominent examples is the processing of Electroencephalography (EEG) signals. Neural oscillations captured by EEG supply relevant information that helps to unravel the neural mechanisms underlying cognitive events and neural disorders. EEG and Magnetoencephalography (MEG) methods record these brain fluctuations and provide priceless insight into both healthy and abnormal brain functioning. In this case, ML techniques can be used along with classical signal processing methods to expose complex patterns in multichannel signals such as EEG or MEG. The exploration of these Information Fusion 100 (2023) 101945 21 J.M. Górriz et al. Fig. 30. The feedback (left) and feedforward (right) modes of the stress-aware ApEn. Fig. 31. Picture of the monitoring platform’s devices. complex patterns can reveal specific features, like a specific neurological disorder, providing valuable information regarding the biological origin. This way, the search for brain activity patterns related to specific disorders such as Developmental Dyslexia (DD) allowing an objective diagnosis, has been a challenge. The diagnosis traditionally lies in behavioral tests which are easily affected by human’s subjective nature. Premature diagnosis of DD is difficult work, which makes it possible to apply personalized treatment tasks to dyslexic infants in the beginning phases of their development. Atypical oscillatory sampling could potentially lead to the phonological impairments characteristic of dyslexia in one or more temporal rhythms; in this sense, EEG signal measurement can help to diagnose DD early on. Thus, in [160], a One-Class Support Vector Machine (OCSVM) is introduced to select representative channels and bands of EEG recordings for both dyslexic and control groups. Based on the selected significant channels, two classical ML classifiers (K-Nearest Neighbours (KNN) and SVM) are separately trained to discriminate subjects with developmental dyslexia from normal control groups. They reported an average sensitivity even higher than the one obtained using traditional, neuropsychological tests and using objective data such as EEG. Some studies take into account the LEEDUCA project, which carried out a number of EEG experiments on children hearing Amplitude Modulated (AM) noise at different frequencies with the aim of exploring brain patterns related to the low-level processing of language, to detect discrepancies in the perception of oscillatory sampling that might be associated with dyslexia. On the other hand, there is an important work directed to explore the neural basis of DD, addressed by studying Cross-Frequency Coupling (CFC) dynamics, such as PhaseAmplitude Coupling (PAC), following previous works using complex Fig. 32. Longitudinal evolution of tremor features and EDA from male participant M1: (a) Session S1-2021.11.19; (b) Session S2-2021.11.26, 2021; (c) Session S3-2021.12.03. Information Fusion 100 (2023) 101945 22 J.M. Górriz et al. Fig. 33. Network architecture to generate the mixed mode image. Fig. 34. Average Holo-Hilbert spectrums for the cross-correlation signals of EEG channel T8 with each other EEG channel for dyslexic subjects. network modeling of EEG using band coupling [241]. They apply a recent emerging approach to infer CFC dynamics, Holo-Hilbert Spectral Analysis (HHSA). This is the next step in addressing the constraints of the current PAC approaches. They pursue HHSA on the above-described EEG database of the LEEDUCA project. Next, Holo-Hilbert spectra are used to examine the PAC changes and patterns in DD (Fig. 34). Finally, the discriminative ability of the spectra is being validated using ML approaches. These neuronal disorders, such as DD cause, in addition to variations in PAC as has just been seen, alterations in connectivity between different brain areas that can lead to facilitate early diagnosis. A different approach to figuring out differential patterns for DD relies on the causal relationships between brain areas, using the same data from the aforementioned LEEDUCA project [242]. In this work, the behavior of each EEG channel in the frequency domain was studied, obtaining the analytical phase by means of the Hilbert transform. Afterward, the cause–effect associations between the channels of each participant were shown by means of Granger causality, resulting in matrices that reflect the interaction between the various parts of the human brain. Thus, each subject was categorized as being either in the control group or in the experimental group. For this purpose, two ensemble algorithms were analyzed, showing that both can reach an acceptable classification efficiency in the delta band (AUC values up to 0.97) by applying the Gradient Boosting classifier. This idea of a different connectivity network is something that can be applied to other conditions, not just DD. Schizophrenia (SZ) is a brain condition that jeopardizes the health of many people worldwide. People with SZ always experience symptoms, including hallucinations and loss of sync of thoughts and feelings. Using DL and connectivity capabilities, [32] presents a method to detect SZ from EEG signaling. In this study, the dataset used for the experiments was provided by the Institute of Psychiatry and Neurology in Warsaw (Poland). First, EEG signals are split into 25-second time frames during the preprocessing stage. Then, in the feature extraction pass, DL and Functional Connectivity Features (FCF) are used concurrently. The DL model involves a CNN-LSTM network, and the functional connectivity techniques include the Synchronization Likelihood (SL), Fuzzy SL (FSL), and Simplified Interval FSL (SIT2FSL) type 2 approaches. In this next step, the DL features and the characteristics of each functional connectivity are combined using a concatenation layer and eventually, to further evaluation the performance, K-Fold with 𝐾= 5 was used in the classification step. The results show that the proposed method achieved an accuracy of 99.43%. EEG signals are therefore useful to model brain diseases with DD or SZ but also to study the medium-term consequences of other diseases, such as respiratory diseases. Sleep apnea syndrome is one of the prevalent sleep diseases and may affect brain function due to transient breathing losses that occur during sleep. Accurate identification and treatment of apnea by physicians can help guard against its long-term disruptive impact. EEG records brain activity from different areas may be an appropriate method to diagnose this problem. [243] propose a CAD taking into account the complexity characteristics of EEG. With this aim, EEG signals of 20 healthy people and 12 apneic patients who suffered from different types of apnea were decomposed into six frequency bands (delta, theta, alpha, sigma, beta, and gamma) by using bandpass Finite Impulse Response (FIR) filters. Complexity features such as fractals, Lempel–Ziv complexity (LZC), entropies, and the generalized Hurst exponent, first used to detect sleep apnea from EEG signals, were extracted from each frequency band. The Maximum Relevance Minimum Redundancy (mRMR) algorithm was applied to classify 120 features from three EEG channels. Finally, two popular classifiers, SVM and KNN, were used to detect sleep apnea. An accuracy of 99.33% was obtained with the SVM classifier, and the generalized Hurst exponent effectively contributed to apnea detection. Not only encephalography is relevant in the study of cognitive processes, but also MRI has been of great interest in recent years, and proof of this is the abundant literature that can be found in this regard. In both cases, these are non-invasive techniques that can help to see how the different cognitive processes that take place at the brain level are encoded, either on a spatial or temporal scale. Recently, combinations of different techniques that, through fusion methods, can combine signals of different natures in a coherent manner are gaining momentum. On the other hand, the library MVPAlab [244] makes a preliminary step to EEG-MRI data fusion for Representational Similarity Analysis (RSA) in EEG signals. This idea has been evaluated with a data set from a prerecorded EEG experiment designed to study the differences in priming between perceptual expectation and selective attention. The strengths and versatility of this multivariate technique and its potential applications in multimodal data fusion are discussed. The complete source code is fully integrated into the MVPAlab toolbox, which increases the wide number of analyses already implemented and the versatility of the tool. 7.6. Neurorehabilitation Computer graphics have always sought ways to make visual information more realistic and accessible to the user. With this objective in mind, its use in scientific research aims at providing accurate and high-quality virtual feedback. Indeed, technological advances have increased the power of processors and graphics, boosting computing and rendering capacity. Likewise, auxiliary technological resources such as motion-tracking devices have been improving in parallel, creating branches of development with a substantial impact on today’s world, such as VR and other related technologies. Information Fusion 100 (2023) 101945 23 J.M. Górriz et al. Fig. 35. Different presentations of chocolate corresponding to different music articulations [247]. Researchers are currently verifying whether VR or optical hand tracking modules can be considered systems capable of monitoring future patients of neurodegenerative diseases such as PD, AD, and ALS, among others [245]. The design methodology is based on an iterative process of development and improvement of the exercises. Capturing a set of features related to the locomotor capacity of the participant’s upper and lower trunk, using two serious games developed for VR, is the main objective. These features provide as much information as possible that may allow determining the biometrical characteristics of the user who performs each of the tasks and detecting small gestures, details, or patterns [246]. However, VR and, more specifically, the novel metaverse require a high level of immersion. Part of the immersive process is made up of the sensations or emotions it provokes in the player. For this reason, knowledge of how sound and sight evoke different emotions in the subject can be considered a top priority for the challenges ahead. Other exploratory approaches based on fMRI try to assess how the brain processes stimuli that are continuous/discontinuous in an auditory and time dimension (different musical articulations) and in a visual and spatial dimension (different presentations of food and paintings) [247]. In particular, professional musician volunteers are monitored through the use of fMRI while using a stimuli device (VisuaStim Digital) for presenting a set of activation blocks consisting of one image (depicting different presentations of food and paintings as shown in Fig. 35) and one musical piece (with either legato or martellato articulation). They explore coherence between the two stimuli (the number of elements shared by the stimuli when the temporal and spatial dimensions are simultaneously confronted). Moreover, other technologies or devices in this context have flourished in recent decades, e.g. robots (see Section 5). In fact, cognitive assistance and communication robots are becoming more and more famous (Nao, Moxie, Milo, etc.). Researchers from all over the world see in these small devices a communication support system for children with autism [248]. Indeed, pedagogical rehabilitation of autistic children through the design of a game using cyber–physical systems is a reality today. The hypothesis is that the following elements are learned with the game: directions, distance, color, teamwork, and socialization. Moreover, the scenario stimulates the three main therapy tasks in cases of autism: imitation, joint attention, and turn-taking. The experimentation of all the studies is based on small exercises that aspire to contrast the previous hypotheses. For example, hyperrealistic scenarios based on medieval games such as archery and javelin throwing, managing to capture up to 60 different features (see Fig. 36) can be properly designed [245]. Likewise, a questionnaire may be elaborated taking into account some of the most important points in the development of VR simulators such as level design, font size, listening to music while using VR goggles, lighting, and texturing. All these questions were directed to avoid the symptoms of motion sickness in the participants. On the other hand, other questions about usability, user-friendliness, and entertainment were also asked of the participants. Finally, the participants had the opportunity to rate the scenarios with a Likert scale. Three of the exercises that neurologists perform with Parkinson’s patients in their consultations can be emulated using VR but in a gamified Fig. 36. Representation of movement and gathering of main indices collected in archery video game. and funny way. To carry out this task, three calibration tests, focusing on biometric values of hands, are developed [245]. In other contexts, small 3D-printed non-humanoid mobile robots can be employed for the design of an educational game scenario. Two children and one teacher participate in a game where the robot does not physically interact with the children [248]. A simple scene is created with several positions in the shape of hexagonal holes. Two children take part in the game as the robot operator and the goal setter. The goal setter uses colored hexagrams which s/he puts in the target position. The robot operator controls the walking robot by means of a laptop or a tablet in order to reach the goal. During the game, the teacher observes the children’s actions and, if necessary, mentors or helps them. At the discretion of the teacher, the two children change roles. The robot can automatically detect the completion of the task (through the use of a color sensor) and measure time. In the pilot study, children with high-functioning autism (ASC) and neurotypical (NT) children participate in playing the same game. 7.7. Precision medicine through sensor-based technology Precision Medicine is a relatively new concept where its core premise is to build a personalized profile for each individual and provide insights into diagnosis, management, and treatment accordingly via the genetic, environmental, and lifestyle characteristics. Smart devices allow for the construction of such a profile in a real-time scenario and its subsequent study and analysis. The aim is to adapt already existing powerful resources widely employed in other areas such as data mining, ontological linking, medical expert systems, and DL, among others to construct such intricate and specific profiles. This would allow providing healthcare solutions that were not feasible to implement some years ago. This concept has been gaining increasing media attention and brought to the forefront of political actions such as the Precision Medicine Initiative [249]. Actigraphy, the tracking of sleep/activity cycles, plays an important role in the Precision Medicine setting, as it is a strong predictor of multiple disorders both physical and mental [250]. It has the potential to provide clinically important insights into physical activity, sleep, and circadian variability over long periods, particularly since commercial research-graded devices can record continuous passive data for months [251]. Many disorders arise due to perturbations of the metabolic system resulting from poor or inadequate daily physical activity or sleep [252]. Actigraphy is especially suited to provide insights into physio-mechanical activity and metabolic disorders through continuous monitoring of biophysical activity and indirect energy consumption. Since ancient times it has been a well-known fact that there exists a relationship between breathing and heart rate, several forms of meditation and relaxation use breathing as a way to control anxiety Information Fusion 100 (2023) 101945 24 J.M. Górriz et al. and reduce heart rate [253]. The vagus nerve plays a crucial role in controlling digestive, cardiovascular, respiratory, urinary, and endocrine functions, among others [250]. It connects the primary brain complex with the structures responsible for controlling the intestines and their environment, and the absorption of food, hormone, and neurotransmitter production. Aligned with this sympathovagal activity which may be controlled through respiration, the work of Posteguillo and Bonomini [254] proposed a methodology to study the interaction between heart rate variability and normal, fast, and slow breathing rates. Specifically, they selected twenty-three young health subjects (34.4±7.2years, 12 male, 11 female) submitted to 12 breaths/min (normal), 20 breaths/min (fast), and 6breaths/min (slow). Blood volume pulse was estimated by photoplethysmography with an Empatica E4 and had to pass a 2-Back test [255]. The results demonstrate the role of slow breathing as a down-regulator of emotional states, and that of fast breathing as a potential up-regulator, helping to understand how training based on respiratory maneuvers may modify cognitive load to cope with stressful situations. Nowadays, implanted cortical visual prostheses to replicate the perceptual sensation are highly demanded [256]. These devices provide visual cues to blind people so they can navigate their environment better. The original implant is composed of a system of an image acquisition camera, a VR headset, an eye-tracking system, an intracortical array, and a stimulus generator to capture the environment and the transitions between objects. The implant stimulates visual areas to generate phosphene triggers, which by training can provide the user with a contour map of the objects in view of the camera, by seeing the actual phosphene-composed map. The device takes as input visual images and applies algorithmic transformations to the images to map the different transitions and uses deep brain stimulation to train the interface between the machine and live tissue to provide impulses that generate the map. To study the effects of the visual stimulation and the perceptual sensations of the implanted system, a rig for researchers was set up to have a perception of the device’s workings using an identical setting except that the cues were visual instead of using deep brain stimulation. The device takes as input visual images through the camera and applies algorithmic transformations to the images to map the different transitions. This new setup was tested on scenery that would emulate a real setting (see Fig. 37). A set of tests assessed the mobility and orientation of five volunteers to check on adaptability. The average walking time in seconds and the number of collisions were compared between completely blind participants (with a walking cane) and those using the simulated prosthetic vision aid. Whereas the use of the walking cane allowed easy detection of obstacles by completely blind participants, the simulated prosthetic vision system required some adaptation before achieving the same performance level, which allowed setting up a processing strategy as the starting point to meet real-time constraints reconfigurability. The possibility of using limited-resolution visual prostheses to perform everyday tasks was studied in the work of Waclawczyk et al. [257], to assess the impact of limited vision restoration, assuming one-eye implants of low spatial resolution, and lack of stereoscopic depth perception. The goal was to quantify the improvements in everyday life activity. The study determined that the degree to which the participants can effectively use artificial vision in everyday life might be the determining factor in the successful use of visual prostheses. Adaptation and learning periods are also important aspects to be considered, the most recommended strategy being a hierarchical approach from the simplest to the most complex tasks, such as motion detection, object recognition, and navigation. 8. Discussion 8.1. DL The fact that trained DL systems are black boxes raises suspicion in users from many application areas, foremost in medical image interpretation or assisted diagnostic systems. XAI is getting increasingly Fig. 37. (a) Raw image from the camera attached to the headset. (b) Monocular Depth Estimation processed image. (c) Augmented Reality using ’ssd-mobilenet-v1’ Object Detection DL model. (d) SPV image. more attention in order to provide representations of the working of the said black box that can be followed by human reasoning in order to justify decisions made on the basis of DL recommendations. A preferred representation is that of propositional rules, while some authors propose explanations in terms of attention mechanisms. Specifically, [37] exploited the discrete cosine transform (DCT) of the feature maps generated in the hidden layers in order to extract rule representations of the functioning of the CNN. In Section 2, a variant of SDAEs was proposed to characterize whether greater capabilities of feature representation can be obtained when two layers are introduced in the stacking process instead of a single one. The results showed a reduction of the computational cost of 15–20%. Therefore, a question arising in this context is whether with three la yers the performance could still increase in terms of computational cost and predictive accuracy. Regarding explainability, propositional rules were extracted from aggregated DIMLPs that learned CNN feature maps related to an MNIST benchmark classification problem. From the rules, it was found that varying a single antecedent in the frequency domain impacted several pixel intensities in the luminosity domain. An important objective is to determine whether the proposed approach is also valid for other classification problems. DRL has shown its power in some quite difficult problems, such as learning to play the game of Go or to predict the spatial folding of proteins. An interesting objective would be to apply explainability through symbolic rule extraction to problems in which the outputs of the deep networks would correspond to actions, which in turn would represent classifications (e.g. jumping; running; etc.). In this way, it would be possible to determine at any point in the reinforcement learning process what knowledge an agent has acquired. However, the critical issue of reward generation from external agents remains open. How to include a human in the loop without undesired interference in the learning process is more often considered an economical way to close the reward loop, and it is showing advantages in specific case demonstrations [36]. 8.1.1. Limits and challenges One of the strong limits of current DL approaches comes from its dependence on reliable and sound data. The need of data augmentation Information Fusion 100 (2023) 101945 25 J.M. Górriz et al. techniques is paramount when data is scarce (i.e. the number of samples is small relative to the population, even if each data sample is large, such as it is the case in medical image applications) [258]. Transference of data augmentation between domains, for instance, using imagebased data augmentation for speech signals [39], provides additional resources to tackle this difficult issue. But even with the help of data augmentation, there is a strong need for well-curated and annotated data [259]. Such need is extensive to DRL applications where the recourse to simulation is commonplace [260]. An increasingly noted limitation of DL-reported results is their low level of statistical confidence assessment [261]. For instance, it is very rare that the authors report results of permutation tests as in [262,263], due to the colossal computational requirements. However, as the DL based systems are pervading all areas of critical decision-making, a strong requirement for their deployment should be a thorough confidence analysis [264], and the ability to pose refutability tests and the exigence of reproducibility of the results [265]. Very sparse reward problems still represent a major challenge in DRL. Here, the problem was solved by introducing intermediate rewards representing intuitive heuristics, depending on a particular case. It would be an advantage if in the future it were possible to automatically determine intermediate rewards or at least for certain classes of problems. Regarding SDAE a clear challenge for the future is to determine whether the use of multi-layer based design of SDAEs can represent an advantage in more complex problems in terms of augmenting the predictive accuracy. The approach to rule extraction over feature map-based CNNs using transfer learning to simpler models is general. However, with many more convolutional layers and a higher number of kernels, the current technique will take much longer to run and is unlikely to be usable, unless higher compression ratios are applied with DCT. Another approach could be to transfer each feature map to a single DIMLP network and then aggregate all DIMLPs into a higher layer. The rules would then be generated in two successive steps, first from the aggregation layer and then back to the lower layers (at the level of the feature maps). 8.2. Bio-inspired systems The objectives in the field are to find the best combinations of metaheuristics in each fitness landscape, as well as to define suitable memetic combinations between metaheuristics with local searches. These local strategies often incorporate application-specific knowledge, thus integrating domain knowledge into the global search inherent in population-based search methods. Papers [78,79] of the BICA session present examples of such memetic combinations in two different application areas. Also, the incorporation of self-adaptation mechanisms in the defining parameters of a metaheuristic, as opposed to their experimental adjustment, continues to be another line of research in this field. The Bacteria algorithm goes a step further in the sense that all previous algorithms based on bacteria are focused on bacteria foraging, which is different from bacteria survival. The introduction of common behavior mechanisms such as conjugation offers a new field of research with interesting potential. The proposal for predicting emotions starting from a lexicon associated with a specific target population, which considers geographical and social parameters, offers a great opportunity to develop new mechanisms for dealing with emotions. Dealing with the problem of assessing the performance of the generator of generative adversarial networks, the authors present a novel approach to reinforce a proposal of a new metric based on the Fourier spectrum. This approach may be used for classification problems. Finally, there has also been an effort to use hyperheuristics (heuristics to choose heuristics) [266]. The goal of a hyperheuristic is to define a combination of low-level heuristics to efficiently explore a search space. The goal in mind with these hyperheuristics is that they can tailor the combination or selection of low-level heuristics to each particular search space. This goal requires an appropriate and usually large set of low-level heuristics, as well as automatically obtaining the selection/combination mechanisms for them (i.e., by an evolutionary algorithm). But the scope of hyperheuristics also includes efforts made to automatically define new heuristics, i.e., applying an evolutionary algorithm to refine or combine a set of heuristics to obtain new heuristic strategies optimized for the problem at hand. In both aspects of hyperheuristic research, especially in the latter, GP is used primarily because it provides naturally evolved programs for selection or as new heuristics. [82,83] includes examples of this use of hyperheuristics. These objectives will continue to guide the field of bio-inspired algorithms, promoting new ideas for the field itself or for other related fields, and will undoubtedly continue to be one of the research lines in the future. 8.2.1. Limits and challenges In every area of interest, it is necessary to take a careful point of view. In a rapid diffusion (and probably misunderstanding) of the concepts behind AI, it is possible to find in non-expert population expectations that are far from realistic developments. AI is not a magic concept, though a compilation of techniques, that usually require the support of non-artificial disciplines. For example, currently, there is a global discussion on when an image can be considered an artistic creation. It is possible to look at a particular image for which it is possible to argue if it corresponds to an artificial creation or a real-world representation. This is the case presented in [267], which requires metrics for a precise evaluation, based on Fourier spectrum image analysis. The use of neural networks is then supported by an additional metric called CSD (Circular Spectrum Distance) to evaluate generative adversarial network images. The enormous amount of data in this field requires every day an increasing processing capability, in particular, in training and classification processes. Fortunately, these capabilities are quite achievable today, but it will take a while to confirm that the approaches under development are proven useful. This is a challenge for each of the aforementioned techniques: neural networks, lexical availability methodology, and bacteria behavior. 8.3. Affective computing One of the main current goals regarding VR and emotion recognition is to reach reliable conclusions when studying the differences between using virtual humans on a computer screen (desktop VR) and a head-mounted display (immersive VR). In this regard, the realism of virtual characters for the different target participants needs to be studied, especially when focusing on people with facial emotion recognition deficits. Moreover, a fair comparison between VR and augmented reality in emotion-based scenarios is a hot topic to be exploited in future research. Furthermore, the relationship between motion sickness symptoms and VR, as well as other difficulties that participants may experience with these new technologies, needs to be further investigated [268]. Another broad objective is the extraction and classification of psychophysiological features to determine the associations between brain connectivity and emotional processing [269]. Any proposal related to emotion induction/detection/recognition/classification must rely heavily on ML techniques for the massive processing and classification of data acquired by a variety of biosensor types. The use of models based on support vector machines and neural networks opens up a wide range of possibilities for improved detection of physiological, perceptual, and behavioral responses, as well as the creation and implementation of neurocognitive and emotional rehabilitation therapies. Emphasis should also be placed on new DL techniques such as CNN, deep belief networks, and capsular networks, among others [270]. Information Fusion 100 (2023) 101945 32 J.M. Górriz et al. [74] J.D. Ser, E. Osaba, D. Molina, X.-S. Yang, S. Salcedo-Sanz, D. Camacho, S. Das, P.N. Suganthan, C.A.C. Coello, F. Herrera, Bio-inspired computation: Where we stand and what’s next, Swarm Evol. Comput. 48 (2019) 220–250, http: //dx.doi.org/10.1016/j.swevo.2019.04.008. [75] L. Arufe, R. Rasconi, A. Oddi, R. Varela, M.Á. González, Compiling single round QCCP-X quantum circuits by genetic algorithm, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 88–97, http://dx.doi.org/10.1007/978-3-031-06527-9_9. [76] P. Barredo, J. Puente, Robust makespan optimization via genetic algorithms on the scientific workflow scheduling problem, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 77–87, http://dx.doi.org/10.1007/978-3-031-06527-9_8. [77] H. Díaz, J.J. Palacios, I. González-Rodríguez, C.R. Vela, Elite artificial bee colony for makespan optimisation in job shop with interval uncertainty, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 98–108, http://dx.doi.org/10. 1007/978-3-031-06527-9_10. [78] P.G. Gómez, I. González-Rodríguez, C.R. Vela, Reducing energy consumption in fuzzy flexible job shops using memetic search, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 140–150, http://dx.doi.org/10.1007/978-3-031-065279_14. [79] J.L. Filgueiras, D. Varela, J. Santos, Energy minimization vs. Deep learning approaches for protein structure prediction, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 109–118, http://dx.doi.org/10.1007/978-3-031-065279_11. [80] D. Varela, J. Santos, Protein structure prediction in an atomic model with differential evolution integrated with the crowding niching method, Nat. Comput. 21 (4) (2020) 537–551, http://dx.doi.org/10.1007/s11047-020-09801-7. [81] D. Varela, J. Santos, Niching methods integrated with a differential evolution memetic algorithm for protein structure prediction, Swarm Evol. Comput. 71 (2022) 101062, http://dx.doi.org/10.1016/j.swevo.2022.101062. [82] F.J. Gil-Gala, M. Ðurasević, M.R. Sierra, R. Varela, Building heuristics and ensembles for the travel salesman problem, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 130–139, http://dx.doi.org/10.1007/978-3-031-065279_13. [83] M. Ðurasević, L. Planinić, F.J. Gil-Gala, D. Jakobović, Constructing ensembles of dispatching rules for multi-objective problems, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 119–129, http://dx.doi.org/10.1007/978-3-031-065279_12. [84] A.R. Contreras, P.V. Hernández, P. Pinacho-Davidson, M.A.P. J., A bacteriabased metaheuristic as a tool for group formation, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 443–451, http://dx.doi.org/10.1007/978-3-031-065279_44. [85] R. Plutchik, The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice, Am. Sci. 89 (4) (2001) 344–350. [86] P. Salcedo-Lagos, P. Pinacho-Davidson, J.M.A. Pinninghoff, G.G. Kotz, A.R. Contreras, An approach to emotions through lexical availability, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 433–442, http://dx.doi.org/10.1007/9783-031-06527-9_43. [87] S. Poria, E. Cambria, R. Bajpai, A. Hussain, A review of affective computing: From unimodal analysis to multimodal fusion, Inf. Fusion 37 (2017) 98–125, http://dx.doi.org/10.1016/j.inffus.2017.02.003. [88] A. Dzedzickis, A. Kaklauskas, V. Bucinskas, Human emotion recognition: Review of sensors and methods, Sensors 20 (3) (2020) 592, http://dx.doi.org/10.3390/ s20030592. [89] B. García-Martínez, A. Fernández-Caballero, A. Martínez-Rodrigo, R. Alcaraz, P. Novais, Evaluation of brain functional connectivity from electroencephalographic signals under different emotional states, Int. J. Neural Syst. 32 (10) (2022) http://dx.doi.org/10.1142/s0129065722500265. [90] M. Monferrer, J.J. Ricarte, M.J. Montes, A. Fernández-Caballero, P. FernándezSotos, Psychosocial remediation in depressive disorders: A systematic review, J. Affect. Disord. 290 (2021) 40–51, http://dx.doi.org/10.1016/j.jad.2021.04.052. [91] J. Bowlby, M. Fry, M.D.S. Ainsworth, Child Care and the Growth of Love, Penguin Books, 1965, p. 256. [92] M. Ainsworth, Infancy in Uganda: Infant Care and the Growth of Love, Johns Hopkins Press, 1967. [93] M.S. Ainsworth, J. Bowlby, An ethological approach to personality development, Am. Psychol. 46 (4) (1991) 333–341, http://dx.doi.org/10.1037/0003066x.46.4.333. [94] J. Bowlby, Attachment, Basic Books, 2008. [95] R. Volpe, The Secure Child, Information Age Pub., 2010, p. 238. [96] M.D.S. Ainsworth, M.C. Blehar, E. Waters, S.N. Wall, Patterns of Attachment, Psychology Press, 2015, http://dx.doi.org/10.4324/9780203758045. [97] L. Tickle-Degnen, R. Rosenthal, The nature of rapport and its nonverbal correlates, Psychol. Inq. 1 (4) (1990) 285–293, http://dx.doi.org/10.1207/ s15327965pli0104_1. [98] C.I.N. B. Reeves, The media equation: How people treat computers, television, and new media like real people and places, Press (1996). [99] J. Cassell, A.J. Gill, P.A. Tepper, Coordination in conversation and rapport, in: Proceedings of the Workshop on Embodied Language Processing, EmbodiedNLP ’07, Association for Computational Linguistics, USA, 2007, pp. 41–50. [100] M.M. van Stralen, H.D. Vries, A.N. Mudde, C. Bolman, L. Lechner, Determinants of initiation and maintenance of physical activity among older adults: A literature review, Health Psychol. Rev. 3 (2) (2009) 147–207, http://dx.doi. org/10.1080/17437190903229462. [101] M. Dainton, L. Stafford, Routine maintenance behaviors: A comparison of relationship type, partner similarity and sex differences, J. Soc. Pers. Relatsh. 10 (2) (1993) 255–271, http://dx.doi.org/10.1177/026540759301000206. [102] J. Gratch, A. Okhmatovskaia, F. Lamothe, S. Marsella, M. Morales, R.J. van der Werf, L.-P. Morency, Virtual rapport, in: International Workshop on Intelligent Virtual Agents, Springer, 2006, pp. 14–27. [103] S.L. Koole, D. Atzil-Slonim, E. Butler, S. Dikker, W. Tschacher, T. Wilderjans, In sync with your shrink, in: Applications of Social Psychology, Routledge, 2020, pp. 161–184, http://dx.doi.org/10.4324/9780367816407-9. [104] S.C.F. Hendrikse, J. Treur, T.F. Wilderjans, S. Dikker, S.L. Koole, On the same wavelengths: Emergence of multiple synchronies among multiple agents, in: Multi-Agent-Based Simulation XXII, Springer International Publishing, 2022, pp. 57–71, http://dx.doi.org/10.1007/978-3-030-94548-0_5. [105] S.C.F. Hendrikse, S. Kluiver, J. Treur, T.F. Wilderjans, S. Dikker, S.L. Koole, How virtual agents can learn to synchronize: An adaptive joint decisionmaking model of psychotherapy, Cogn. Syst. Res. 79 (2023) 138–155, http: //dx.doi.org/10.1016/j.cogsys.2022.12.009. [106] B. Harry, P.E. Keller, Tutorial and simulations with ADAM: An adaptation and anticipation model of sensorimotor synchronization, Biol. Cybernet. 113 (4) (2019) 397–421, http://dx.doi.org/10.1007/s00422-019-00798-6. [107] K. Sanlaville, G. Assayag, F. Bevilacqua, C. Pelachaud, Emergence of synchrony in an Adaptive Interaction Model, 2015, http://dx.doi.org/10.48550/ARXIV. 1506.05573. [108] S.S. Wiltermuth, C. Heath, Synchrony and cooperation, Psychol. Sci. 20 (1) (2009) 1–5, http://dx.doi.org/10.1111/j.1467-9280.2008.02253.x. [109] B. Tarr, J. Launay, R.I.M. Dunbar, Silent disco: Dancing in synchrony leads to elevated pain thresholds and social closeness, Evol. Hum. Behav. 37 (5) (2016) 343–349, http://dx.doi.org/10.1016/j.evolhumbehav.2016.02.004. [110] S.L. Koole, W. Tschacher, Synchrony in psychotherapy: A review and an integrative framework for the therapeutic alliance, Front. Psychol. 7 (2016) http://dx.doi.org/10.3389/fpsyg.2016.00862. [111] A. Williams, T. O’Leary, E. Marder, Homeostatic regulation of neuronal excitability, Scholarpedia 8 (1) (2013) 1656, http://dx.doi.org/10.4249/ scholarpedia.1656. [112] N. Chandra, E. Barkai, A non-synaptic mechanism of complex learning: Modulation of intrinsic neuronal excitability, Neurobiol. Learn. Mem. 154 (2018) 30–36, http://dx.doi.org/10.1016/j.nlm.2017.11.015. [113] D. Debanne, Y. Inglebert, M. l Russier, Plasticity of intrinsic neuronal excitability, Curr. Opin. Neurobiol. 54 (2019) 73–82, http://dx.doi.org/10.1016/j.conb. 2018.09.001. [114] A. Zhang, X. Li, Y. Gao, Y. Niu, Event-driven intrinsic plasticity for spiking convolutional neural networks, IEEE Trans. Neural Netw. Learn. Syst. 33 (5) (2022) 1986–1995, http://dx.doi.org/10.1109/tnnls.2021.3084955. [115] C.J. Shatz, The developing brain, Sci. Am. 267 (3) (1992) 60–67, http://dx.doi. org/10.1038/scientificamerican0992-60. [116] D. Hebb, The Organization of Behavior, Lawrence Erlbaum, 2002, p. 336. [117] S.C.F. Hendrikse, J. Treur, T.F. Wilderjans, S. Dikker, S.L. Koole, On the interplay of interpersonal synchrony, short-term affiliation and long-term bonding: A second-order multi-adaptive neural agent model, in: IFIP Advances in Information and Communication Technology, Springer International Publishing, 2022, pp. 37–57, http://dx.doi.org/10.1007/978-3-031-08333-4_4. [118] F. de Vignemont, T. Singer, The empathic brain: How, when and why? Trends in Cognitive Sciences 10 (10) (2006) 435–441, http://dx.doi.org/10.1016/j.tics. 2006.08.008. [119] T. Singer, C. Lamm, The Social Neuroscience of Empathy, Wiley-Blackwell, 2009, http://dx.doi.org/10.5167/UZH-25655. [120] J. Decety, P.L. Jackson, The functional architecture of human empathy, Behav. Cogn. Neurosci. Rev. 3 (2) (2004) 71–100, http://dx.doi.org/10.1177/ 1534582304267187. [121] J. Treur, Biological and computational perspectives on the emergence of social phenomena: Shared understanding and collective power, in: Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2012, pp. 168–191, http: //dx.doi.org/10.1007/978-3-642-34645-3_8. [122] Z.A. Memon, J. Treur, Designing social agents with empathic understanding, in: Computational Collective Intelligence. Semantic Web, Social Networks and Multiagent Systems, Springer Berlin Heidelberg, 2009, pp. 279–293, http://dx. doi.org/10.1007/978-3-642-04441-0_24. Information Fusion 100 (2023) 101945 33 J.M. Górriz et al. [123] Z.A. Memon, J. Treur, An agent model for cognitive and affective empathic understanding of other agents, in: Transactions on Compuational Collective Intelligence VI, Springer Berlin Heidelberg, 2012, pp. 56–83, http://dx.doi.org/ 10.1007/978-3-642-29356-6_3. [124] C.-Y. Wang, G.-D. Chen, C.-C. Liu, B.-J. Liu, Design an empathic virtual human to encourage and persuade learners in e-learning systems, in: Proceedings of the First ACM International Workshop on Multimedia Technologies for Distance Learning, ACM, 2009, http://dx.doi.org/10.1145/1631111.1631117. [125] D. Hudson, T.J. Wiltshire, M. Atzmueller, Visualization methods for exploratory subgroup discovery on time series data, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 34–44, http://dx.doi.org/10.1007/978-3-031-06527-9_4. [126] M. Atzmueller, F. Lemmerich, VIKAMINE – open-source subgroup discovery, pattern mining, and analytics, in: Machine Learning and Knowledge Discovery in Databases, Springer Berlin Heidelberg, 2012, pp. 842–845, http://dx.doi.org/ 10.1007/978-3-642-33486-3_60. [127] M.A. Vicente-Querol, A. Fernández-Caballero, J.P. Molina, L.M. GonzálezGualda, P. Fernández-Sotos, A.S. García, Facial affect recognition in immersive virtual reality: Where is the participant looking? Int. J. Neural Syst. 32 (10) (2022) http://dx.doi.org/10.1142/s0129065722500290. [128] D. Palacios-Alonso, J. Barbas-Cubero, L. Betancourt-Ortega, M. FernándezFernández, Measuring motion sickness through racing simulator based on virtual reality, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 494–504, http://dx.doi.org/10.1007/978-3-031-06242-1_49. [129] B. García-Martínez, A. Fernández-Caballero, Influence of neutral stimuli on brain activity baseline in emotional experiments, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 475–484, http://dx.doi.org/10.1007/978-3-031-062421_47. [130] A. Quintero-Zea, J. Martínez-Vargas, D. Gómez, N. Trujillo, J.D. López, Classification of psychophysiological patterns during emotional processing using SVM, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 485–493, http://dx. doi.org/10.1007/978-3-031-06242-1_48. [131] L. Barrett, How Emotions are Made, Pan Macmillan, 2018. [132] P. Langley, J.E. Laird, S. Rogers, Cognitive architectures: Research issues and challenges, Cogn. Syst. Res. 10 (2) (2009) 141–160, http://dx.doi.org/10.1016/ j.cogsys.2006.07.004. [133] S. Doncieux, D. Filliat, N. Díaz-Rodríguez, T. Hospedales, R. Duro, A. Coninx, D.M. Roijers, B. Girard, N. Perrin, O. Sigaud, Open-ended learning: A conceptual framework based on representational redescription, Front. Neurorobotics 12 (2018) http://dx.doi.org/10.3389/fnbot.2018.00059. [134] G. Baldassarre, M. Mirolli, Intrinsically motivated learning systems: An overview, in: Intrinsically Motivated Learning in Natural and Artificial Systems, Springer Berlin Heidelberg, 2012, pp. 1–14, http://dx.doi.org/10.1007/978-3642-32375-1_1. [135] R.J. Duro, J.A. Becerra, J. Monroy, F. Bellas, Perceptual generalization and context in a network memory inspired long-term memory for artificial cognition, Int. J. Neural Syst. 29 (06) (2019) 1850053, http://dx.doi.org/10.1142/ s0129065718500533. [136] A. Romero, B. Meden, F. Bellas, R.J. Duro, Autonomous knowledge representation for efficient skill learning in cognitive robots, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 253–263, http://dx.doi.org/10.1007/978-3-031-065279_25. [137] E. Nivel, K.R. Thórisson, B. Steunebrink, J. Schmidhuber, Anytime bounded rationality, in: Artificial General Intelligence, Springer International Publishing, 2015, pp. 121–130, http://dx.doi.org/10.1007/978-3-319-21365-1_13. [138] A. Romero, F. Bellas, R.J. Duro, Open-ended learning of reactive knowledge in cognitive robotics based on neuroevolution, in: Lecture Notes in Computer Science, Springer International Publishing, 2021, pp. 65–76, http://dx.doi.org/ 10.1007/978-3-030-86271-8_6. [139] S. Thrun, Lifelong learning algorithms, in: Learning to Learn, Springer, US, 1998, pp. 181–209, http://dx.doi.org/10.1007/978-1-4615-5529-2_8. [140] G.I. Parisi, R. Kemker, J.L. Part, C. Kanan, S. Wermter, Continual lifelong learning with neural networks: A review, Neural Netw. 113 (2019) 54–71, http://dx.doi.org/10.1016/j.neunet.2019.01.012. [141] L. Ferrero, V. Quiles, M. Ortiz, J.V. Juan, E. Iáñez, J.M. Azorín, Inter-session transfer learning in MI based BCI for controlling a lower-limb exoskeleton, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 243–252, http://dx.doi.org/10. 1007/978-3-031-06527-9_24. [142] N. Milano, M. Ponticorvo, Spatial frames of reference and action: A study with evolved neuro-agents, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 515–523, http://dx.doi.org/10.1007/978-3-031-06527-9_51. [143] F.A.M. García, J.M.C. Troncoso, F. de la Paz López, J.R. Álvarez-Sánchez, Autonomous robot navigation by area centroid algorithm using depth cameras, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 264–275, http://dx.doi.org/ 10.1007/978-3-031-06527-9_26. [144] D.P. Kingma, M. Welling, An introduction to variational autoencoders, Found. Trends Mach. Learn. 12 (4) (2019) 307–392, http://dx.doi.org/10.1561/ 2200000056. [145] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Commun. ACM 63 (11) (2020) 139–144, http://dx.doi.org/10.1145/3422622. [146] J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Adv. Neural Inf. Process. Syst. 33 (2020) 6840–6851, http://dx.doi.org/10.48550/ARXIV. 2006.11239. [147] I. van den Berk-Smeekens, M. van Dongen-Boomsma, M.W.P.D. Korte, J.C.D. Boer, I.J. Oosterling, N.C. Peters-Scheffer, J.K. Buitelaar, E.I. Barakova, T. Lourens, W.G. Staal, J.C. Glennon, Adherence and acceptability of a robotassisted pivotal response treatment protocol for children with autism spectrum disorder, Sci. Rep. 10 (1) (2020) http://dx.doi.org/10.1038/s41598-020-650483. [148] I. van den Berk-Smeekens, M.W.P. de Korte, M. van Dongen-Boomsma, I.J. Oosterling, J.C. den Boer, E.I. Barakova, T. Lourens, J.C. Glennon, W.G. Staal, J.K. Buitelaar, Pivotal response treatment with and without robot-assistance for children with autism: A randomized controlled trial, Eur. Child Adolesc. Psychiatry 31 (12) (2021) 1871–1883, http://dx.doi.org/10.1007/s00787-02101804-8. [149] G. Benedicto, M. Val, E. Fernández, F.S. Ferrer, J.M. Ferrández, Autism spectrum disorder (ASD): Emotional intervention protocol, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 310–322, http://dx.doi.org/10.1007/978-3031-06242-1_31. [150] L.K. Koegel, R.L. Koegel, J.K. Harrower, C.M. Carter, Pivotal response intervention I: Overview of approach, J. Assoc. Pers. Sev. Handicap. 24 (3) (1999) 174–185, http://dx.doi.org/10.2511/rpsd.24.3.174. [151] T. Schulz, K.S. Fuglerud, Creating vignettes for a robot-supported education solution for children with autism spectrum disorder, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 323–331, http://dx.doi.org/10.1007/978-3-031-062421_32. [152] F.J. Martinez-Murcia, J.M. Górriz, J. Ramírez, A. Ortiz, Convolutional neural networks for neuroimaging in Parkinson’s disease: Is preprocessing needed? Int. J. Neural Syst. 28 (10) (2018) 1850035, http://dx.doi.org/10. 1142/s0129065718500351. [153] D. Castillo-Barnes, F.J. Martinez-Murcia, A. Ortiz, D. Salas-Gonzalez, J. Ramírez, J.M. Górriz, Morphological characterization of functional brain imaging by isosurface analysis in Parkinson’s disease, Int. J. Neural Syst. 30 (09) (2020) 2050044, http://dx.doi.org/10.1142/s0129065720500446. [154] C. Jimenez-Mesa, I.A. Illan, A. Martin-Martin, D. Castillo-Barnes, F.J. MartinezMurcia, J. Ramirez, J.M. Gorriz, Optimized one vs one approach in multiclass classification for early Alzheimer’s disease and mild cognitive impairment diagnosis, IEEE Access 8 (2020) 96981–96993, http://dx.doi.org/10.1109/ access.2020.2997736. [155] D. Castillo-Barnes, L. Su, J. Ramírez, D. Salas-Gonzalez, F.J. Martinez-Murcia, I.A. Illan, F. Segovia, A. Ortiz, C. Cruchaga, M.R. Farlow, C. Xiong, N.R. GraffRadford, P.R. Schofield, C.L. Masters, S. Salloway, M. Jucker, H. Mori, J. Levin, J.M. Gorriz, D.I.A.N. (DIAN), Autosomal dominantly inherited alzheimer disease: Analysis of genetic subgroups by machine learning, Inf. Fusion 58 (2020) 153–167, http://dx.doi.org/10.1016/j.inffus.2020.01.001. [156] J.E. Arco, A. Ortiz, N.J. Gallego-Molina, J.M. Górriz, J. Ramírez, Enhancing multimodal patterns in neuroimaging by siamese neural networks with selfattention mechanism, Int. J. Neural Syst. (2023) http://dx.doi.org/10.1142/ s0129065723500193. [157] S. Raffard, R.N. Salesse, C. Bortolon, B.G. Bardy, J. Henriques, L. Marin, D. Stricker, D. Capdevielle, Using mimicry of body movements by a virtual agent to increase synchronization behavior and rapport in individuals with schizophrenia, Sci. Rep. 8 (1) (2018) http://dx.doi.org/10.1038/s41598-01835813-6. [158] G.B. Chand, D.B. Dwyer, G. Erus, A. Sotiras, E. Varol, D. Srinivasan, J. Doshi, R. Pomponio, A. Pigoni, P. Dazzan, R.S. Kahn, H.G. Schnack, M.V. Zanetti, E. Meisenzahl, G.F. Busatto, B. Crespo-Facorro, C. Pantelis, S.J. Wood, C. Zhuo, R.T. Shinohara, H. Shou, Y. Fan, R.C. Gur, R.E. Gur, T.D. Satterthwaite, N. Koutsouleris, D.H. Wolf, C. Davatzikos, Two distinct neuroanatomical subtypes of schizophrenia revealed using machine learning, Brain 143 (3) (2020) 1027–1038, http://dx.doi.org/10.1093/brain/awaa025. [159] N.J. Gallego-Molina, A. Ortiz, F.J. Martínez-Murcia, I. Rodríguez-Rodríguez, Unraveling dyslexia-related connectivity patterns in EEG signals by holo-Hilbert spectral analysis, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 43–52, http://dx.doi.org/10.1007/978-3-031-06242-1_5. Information Fusion 100 (2023) 101945 34 J.M. Górriz et al. [160] M.A. Formoso, A. Ortiz, F.J. Martinez-Murcia, N. Gallego, J.L. Luque, Detecting phase-synchrony connectivity anomalies in EEG signals. application to dyslexia diagnosis, Sensors 21 (21) (2021) 7061, http://dx.doi.org/10.3390/s21217061. [161] C. Bridgemohan, D.M. Cochran, Y.J. Howe, K. Pawlowski, A.W. Zimmerman, G.M. Anderson, R. Choueiri, L. Sices, K.J. Miller, M. Ultmann, J. Helt, P.W. Forbes, L. Farfel, S.J. Brewster, J.A. Frazier, A.M. Neumeyer, Investigating potential biomarkers in autism spectrum disorder, Front. Integr. Neurosci. 13 (2019) http://dx.doi.org/10.3389/fnint.2019.00031. [162] M. Leming, J.M. Górriz, J. Suckling, Ensemble deep learning on large, mixedsite fMRI datasets in autism and other tasks, Int. J. Neural Syst. 30 (07) (2020) 2050012, http://dx.doi.org/10.1142/s0129065720500124. [163] World Mental Health Report: Transforming Mental Health for All - Executive Summary, World Health Organization, 2022, p. 296. [164] C. Reyes-Daneri, F.J. Martínez-Murcia, A. Ortiz, Capacity estimation from environmental audio signals using deep learning, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 114–124, http://dx.doi.org/10.1007/978-3-031-062421_12. [165] J.M. Ferrandez, A. Alfaro, P. Bonomini, J.M. Tormos, L. Concepcion, F. Pelayo, E. Fernandez, Brain plasticity: Feasibility of a cortical visual prosthesis for the blind, in: Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439), vol. 3, 2003, pp. 2027–2030 Vol.3, http://dx.doi.org/10.1109/IEMBS.2003.1280133. [166] J.M. Ferrandez, E. Liano, P. Bonomini, J.J. Martinez, J. Toledo, E. Fernandez, A customizable multi-channel stimulator for cortical neuroprosthesis, in: 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2007, http://dx.doi.org/10.1109/iembs.2007.4353390. [167] A. Martínez-Álvarez, A. Olmedo-Payá, S. Cuenca-Asensi, J.M. Ferrández, E. Fernández, RetinaStudio: A bioinspired framework to encode visual information, Neurocomputing 114 (2013) 45–53, http://dx.doi.org/10.1016/j.neucom.2012. 07.035. [168] J. Sorinas, M.D. Grima, J.M. Ferrandez, E. Fernandez, Identifying Suitable Brain Regions and trial size segmentation for positive/negative emotion recognition, Int. J. Neural Syst. 29 (02) (2019) 1850044, http://dx.doi.org/10.1142/ s0129065718500442. [169] M. Val-Calvo, J.R. Alvarez-Sanchez, J.M. Ferrandez-Vicente, E. Fernandez, Affective robot story-telling human-robot interaction: Exploratory real-time emotion estimation analysis using facial expressions and physiological signals, IEEE Access 8 (2020) 134051–134066, http://dx.doi.org/10.1109/access.2020. 3007109. [170] M.P. Bonomini, P.D. Arini, G.E. Gonzalez, B. Buchholz, M.E. Valentinuzzi, The allometric model in chronic myocardial infarction, Theor. Biol. Med. Model. 9 (1) (2012) http://dx.doi.org/10.1186/1742-4682-9-15. [171] A.I.R. Soler, M.P. Bonomini, C.F. Biscay, F. Ingallina, P.D. Arini, Modelling of the electrocardiographic signal during an angioplasty procedure in the right coronary artery, J. Electrocardiol. 62 (2020) 65–72, http://dx.doi.org/10.1016/ j.jelectrocard.2020.08.003. [172] M.P. Bonomini, D.F. Ortega, L.D. Barja, N. Mangani, P.D. Arini, Depolarization spatial variance as a cardiac dyssynchrony descriptor, Biomed. Signal Process. Control 49 (2019) 540–545, http://dx.doi.org/10.1016/j.bspc.2018.12.009. [173] M.P. Bonomini, M.V. Calvo, A.D. Morcillo, F. Segovia, J.M.F. Vicente, E. Fernandez-Jover, The effect of breath pacing on task switching and working memory, Int. J. Neural Syst. 30 (06) (2020) 2050028, http://dx.doi.org/10. 1142/s0129065720500288. [174] B. del Cisne Macas Ordónez, J.M. Ferrández-Vicente, M.P. Bonomini, QRS-t angle as a biomarker for LBBB strict diagnose, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 586–594, http://dx.doi.org/10.1007/978-3-031-062421_58. [175] O.-B. Tysnes, A. Storstein, Epidemiology of Parkinson’s disease, J. Neural Transm. 124 (8) (2017) 901–905, http://dx.doi.org/10.1007/s00702-017-1686y. [176] N.R. Swerdlow, S.G. Bhakta, G.A. Light, Room to move: Plasticity in early auditory information processing and auditory learning in schizophrenia revealed by acute pharmacological challenge, Schizophr. Res. 199 (2018) 285–291, http://dx.doi.org/10.1016/j.schres.2018.03.037. [177] P. Johns, Clinical Neuroscience an Illustrated Colour Text, Elsevier Health Sciences, 2014. [178] L. Sigcha, D.G. Calleja, I. Pavón, J.M. López, G. de Arcas, Monitoring motor symptoms in Parkinson’s disease under long term acoustic stimulation, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 189–198, http://dx.doi.org/ 10.1007/978-3-031-06242-1_19. [179] P. Gómez-Vilda, A. Gómez-Rodellar, D. Palacios-Alonso, A. Álvarez-Marquina, Effects of neuroacoustic stimulation on two study cases of Parkinson’s disease dysarthria, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 209–218, http://dx.doi.org/10.1007/978-3-031-06242-1_21. [180] G. Gálvez-García, A. Gómez-Rodellar, D. Palacios-Alonso, G. de Arcas-Castro, P. Gómez-Vilda, Neuroacoustical stimulation of Parkinson’s disease patients: A case study, in: From Bioinspired Systems and Biomedical Applications to Machine Learning, Springer International Publishing, 2019, pp. 329–339, http: //dx.doi.org/10.1007/978-3-030-19651-6_32. [181] A. Gómez-Rodellar, J. Mekyska, P. Gómez-Vilda, L. Brabenec, P. Simko, I. Rektorova, Evaluation of TMS effects on the phonation of Parkinson’s disease patients, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 199–208, http://dx.doi.org/10.1007/978-3-031-06242-1_20. [182] A. Gómez-Rodellar, P. Gómez-Vilda, J. Ferrández-Vicente, A. Tsanas, Characterizing masseter surface electromyography on EEG-related frequency bands in Parkinson’s disease neuromotor dysarthria, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 219–228, http://dx.doi.org/10.1007/978-3-031-062421_22. [183] L. Brabenec, P. Klobusiakova, P. Simko, M. Kostalova, J. Mekyska, I. Rektorova, Non-invasive brain stimulation for speech in Parkinson’s disease: A randomized controlled trial, Brain Stimul. 14 (3) (2021) 571–578, http://dx.doi.org/10. 1016/j.brs.2021.03.010. [184] D. Hassabis, D. Kumaran, C. Summerfield, M. Botvinick, Neuroscience-inspired artificial intelligence, Neuron 95 (2) (2017) 245–258, http://dx.doi.org/10. 1016/j.neuron.2017.06.011. [185] J.M. Gorriz, C. Jimenez-Mesa, R. Romero-Garcia, F. Segovia, J. Ramirez, D. Castillo-Barnes, F.J. Martinez-Murcia, A. Ortiz, D. Salas-Gonzalez, I.A. Illan, C.G. Puntonet, D. Lopez-Garcia, M. Gomez-Rio, J. Suckling, Statistical agnostic mapping: A framework in neuroimaging based on concentration inequalities, Inf. Fusion 66 (2021) 198–212, http://dx.doi.org/10.1016/j.inffus.2020.09.008. [186] C.P.E. Rollins, J.R. Garrison, M. Arribas, A. Seyedsalehi, Z. Li, R.C.K. Chan, J. Yang, D. Wang, P. Liò, C. Yan, Z. hui Yi, A. Cachia, R. Upthegrove, B. Deakin, J.S. Simons, G.K. Murray, J. Suckling, Evidence in cortical folding patterns for prenatal predispositions to hallucinations in schizophrenia, Transl. Psychiatry 10 (1) (2020) http://dx.doi.org/10.1038/s41398-020-01075-y. [187] K. Doi, Computer-aided diagnosis in medical imaging: Historical review, current status and future potential, Comput. Med. Imaging Graph. 31 (4–5) (2007) 198–211, http://dx.doi.org/10.1016/j.compmedimag.2007.02.002. [188] D. Castillo-Barnes, J.E. Arco, C. Jimenez-Mesa, J. Ramirez, J.M. Górriz, D. SalasGonzalez, Evaluating intensity concentrations during the spatial normalization of functional images for Parkinson’s disease, in: J.M. Ferrández Vicente, J.R. Álvarez-Sánchez, F. de la Paz López, H. Adeli (Eds.), Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 176–186, http://dx.doi.org/10.1007/978-3-031-062421_18. [189] C. Jimenez-Mesa, D. Castillo-Barnes, J.E. Arco, F. Segovia, J. Ramirez, J.M. Górriz, Analyzing statistical inference maps using MRI images for Parkinson’s disease, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 166–175, http://dx. doi.org/10.1007/978-3-031-06242-1_17. [190] J.E. Arco, A. Ortiz, D. Castillo-Barnes, J.M. Górriz, J. Ramírez, Quantifying inter-hemispheric differences in Parkinson’s disease using siamese networks, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 156–165, http://dx.doi.org/ 10.1007/978-3-031-06242-1_16. [191] D. Castillo-Barnes, C. Jimenez-Mesa, F.J. Martinez-Murcia, D. Salas-Gonzalez, J. Ramírez, J.M. Górriz, Quantifying differences between affine and nonlinear spatial normalization of FP-CIT SPECT images, Int. J. Neural Syst. 32 (05) (2022) http://dx.doi.org/10.1142/s0129065722500198. [192] J.A. Simón-Rodríguez, F.J. Martinez-Murcia, J. Ramírez, D. Castillo-Barnes, J.M. Gorriz, Modelling the progression of the symptoms of Parkinson’s disease using a nonlinear decomposition of 123iFP-CIT SPECT images, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 104–113, http://dx.doi.org/10.1007/978-3031-06242-1_11. [193] A. Álvarez-Marquina, A. Gómez-Rodellar, P. Gómez-Vilda, D. Palacios-Alonso, F. Díaz-Pérez, Identification of Parkinson’s disease from speech using CNNs and formant measures, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 332–342, http://dx.doi.org/10.1007/978-3-031-06242-1_33. [194] P. Gómez-Vilda, A. Gómez-Rodellar, D. Palacios-Alonso, A. Álvarez-Marquina, A. Tsanas, Characterization of hypokinetic dysarthria by a CNN based on auditory receptive fields, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 343–352, http://dx.doi.org/10.1007/978-3-031-06242-1_34. [195] F. Greenberg, R.A. Lewis, L. Potocki, D. Glaze, J. Parke, J. Killian, M.A. Murphy, D. Williamson, F. Brown, R. Dutton, C. McCluggage, E. Friedman, M. Sulek, J.R. Lupski, Multi-disciplinary clinical study of smith-magenis syndrome (deletion 17p11.2), Am. J. Med. Genet. 62 (3) (1996) 247–254, http://dx.doi.org/10. 1002/(sici)1096-8628(19960329)62:3{<}247::aid-ajmg9{>}3.0.co;2-q. Information Fusion 100 (2023) 101945 35 J.M. Górriz et al. [196] R. Martínez-Olalla, D. Palacios-Alonso, I. Hidalgo-delaGuía, E. GarayzabalHeinze, P. Gómez-Vilda, Evaluation of the presence of subharmonics in the phonation of children with smith magenis syndrome, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 353–362, http://dx.doi.org/10.1007/978-3-031-062421_35. [197] O. Ivanova, J.J.G. Meilán, Speech analysis in preclinical identification of Alzheimer’s disease, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 363–368, http://dx.doi.org/10.1007/978-3-031-06242-1_36. [198] Y. Liu, W. Tan, C. Chen, C. Liu, J. Yang, Y. Zhang, A review of the application of virtual reality technology in the diagnosis and treatment of cognitive impairment, Front. Aging Neurosci. 11 (2019) http://dx.doi.org/10.3389/fnagi. 2019.00280. [199] M.J. Kang, S.Y. Kim, D.L. Na, B.C. Kim, D.W. Yang, E.-J. Kim, H.R. Na, H.J. Han, J.-H. Lee, J.H. Kim, K.H. Park, K.W. Park, S.-H. Han, S.Y. Kim, S.J. Yoon, B. Yoon, S.W. Seo, S.Y. Moon, Y. Yang, Y.S. Shim, M.J. Baek, J.H. Jeong, S.H. Choi, Y.C. Youn, Prediction of cognitive impairment via deep learning trained with multi-center neuropsychological test data, BMC Med. Inform. Decis. Mak. 19 (1) (2019) http://dx.doi.org/10.1186/s12911-019-0974-x. [200] M. Ansart, S. Epelbaum, G. Bassignana, A. Bône, S. Bottani, T. Cattai, R. l Couronné, J. Faouzi, I. Koval, M. Louis, E. Thibeau-Sutre, J. Wen, A. Wild, N. Burgos, D. Dormont, O. Colliot, S. Durrleman, Predicting the progression of mild cognitive impairment using machine learning: A systematic, quantitative and critical review, Med. Image Anal. 67 (2021) 101848, http://dx.doi.org/10. 1016/j.media.2020.101848. [201] R.C. Petersen, S. Negash, Mild cognitive impairment: An overview, CNS Spectr. 13 (1) (2008) 45–53, http://dx.doi.org/10.1017/s1092852900016151. [202] R.C. Petersen, R.O. Roberts, D.S. Knopman, B.F. Boeve, Y.E. Geda, R.J. Ivnik, G.E. Smith, C.R. Jack, Mild cognitive impairment, Arch. Neurol. 66 (12) (2009) http://dx.doi.org/10.1001/archneurol.2009.266. [203] J. Dahmen, D. Cook, R. Fellows, M. Schmitter-Edgecombe, An analysis of a digital variant of the trail making test using machine learning techniques, Technol. Health Care 25 (2) (2017) 251–264, http://dx.doi.org/10.3233/thc161274. [204] S. Chen, D. Stromer, H.A. Alabdalrahim, S. Schwab, M. Weih, A. Maier, Automatic dementia screening and scoring by applying deep learning on clockdrawing tests, Sci. Rep. 10 (1) (2020) http://dx.doi.org/10.1038/s41598-02074710-9. [205] R.O. Canham, S.L. Smith, A.M. Tyrrell, Automated scoring of a neuropsychological test: The rey osterrieth complex figure, in: Proceedings of the 26th Euromicro Conference. EUROMICRO 2000. Informatics: Inventing the Future, IEEE Comput. Soc, 2000, http://dx.doi.org/10.1109/eurmic.2000.874519. [206] J.M. Guerrero, M. Rincón, H. Peraita, R. Martínez-Tomás, Diagnosis of cognitive impairment compatible with early diagnosis of Alzheimer’s disease, Methods Inf. Med. 55 (01) (2016) 42–49, http://dx.doi.org/10.3414/me14-01-0071. [207] B. Ghoraani, L.N. Boettcher, M.D. Hssayeni, A. Rosenfeld, M.I. Tolea, J.E. Galvin, Detection of mild cognitive impairment and Alzheimer’s disease using dual-task gait assessments and machine learning, Biomed. Signal Process. Control 64 (2021) 102249, http://dx.doi.org/10.1016/j.bspc.2020.102249. [208] I. Rawtaer, R. Mahendran, E.H. Kua, H.P. Tan, H.X. Tan, T.-S. Lee, T.P. Ng, Early detection of mild cognitive impairment with in-home sensors to monitor behavior patterns in community-dwelling senior citizens in Singapore: Cross-sectional feasibility study, J. Med. Internet Res. 22 (5) (2020) e16854, http://dx.doi.org/10.2196/16854. [209] A.A. Wanigatunga, F. Liu, H. Wang, J.K. Urbanek, Y. An, A.P. Spira, R.J. Dougherty, Q. Tian, A. Moghekar, L. Ferrucci, E.M. Simonsick, S.M. Resnick, J.A. Schrack, Daily physical activity patterns as a window on cognitive diagnosis in the baltimore longitudinal study of aging (BLSA), in: M.A. Ikram (Ed.), J. Alzheimer. Dis. 88 (2) (2022) 459–469, http://dx.doi.org/10.3233/jad-215544. [210] O. Kim, Y. Pang, J.-H. Kim, The effectiveness of virtual reality for people with mild cognitive impairment or dementia: A meta-analysis, BMC Psychiatry 19 (1) (2019) http://dx.doi.org/10.1186/s12888-019-2180-x. [211] C. Jiménez-Mesa, J.E. Arco, M. Valentí-Soler, B. Frades-Payo, M.A. Zea-Sevilla, A. Ortiz, M. Ávila-Villanueva, D. Castillo-Barnes, J. Ramírez, T. del Ser-Quijano, C. Carnero-Pardo, J.M. Górriz, Automatic classification system for diagnosis of cognitive impairment based on the clock-drawing test, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 34–42, http://dx.doi.org/10.1007/978-3031-06242-1_4. [212] E. Estella-Nonay, M. Bachiller-Mayoral, S. Valladares-Rodriguez, M. Rincón, Automatic diagnosis of mild cognitive impairment using siamese neural networks, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 416–425, http://dx. doi.org/10.1007/978-3-031-06242-1_41. [213] F.J. Pinilla, R. Martínez-Tomás, M. Rincón, Automatic scoring of rey-osterrieth complex figure test using recursive cortical networks, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 455–463, http://dx.doi.org/10.1007/978-3-03106242-1_45. [214] G.M. Monica, M.T. Rafael, A comparison of feature-based classifiers and transfer learning approaches for cognitive impairment recognition in language, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 426–435, http://dx.doi.org/10. 1007/978-3-031-06242-1_42. [215] M. Ponticorvo, M. Coccorese, O. Gigliotta, P. Bartolomeo, D. Marocco, Artificial intelligence applied to spatial cognition assessment, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 407–415, http://dx.doi.org/10.1007/978-3-031-062421_40. [216] J.P. Amezquita-Sanchez, N. Mammone, F.C. Morabito, S. Marino, H. Adeli, A novel methodology for automated differential diagnosis of mild cognitive impairment and the Alzheimer’s disease using EEG signals, J. Neurosci. Methods 322 (2019) 88–95, http://dx.doi.org/10.1016/j.jneumeth.2019.04.013. [217] N. Saif, P. Yan, K. Niotis, O. Scheyer, A. Rahman, M. Berkowitz, R. Krikorian, H. Hristov, G. Sadek, S. Bellara, R.S. Isaacson, Feasibility of using a wearable biosensor device in patients at risk for Alzheimer’s disease dementia, J. Prev. Alzheimer. Dis. (2019) 1–8, http://dx.doi.org/10.14283/jpad.2019.39. [218] F. Yang, J. Jiang, I. Alberts, M. Wang, T. Li, X. Sun, A. Rominger, C. Zuo, K. Shi, Combining PET with MRI to improve predictions of progression from mild cognitive impairment to Alzheimer’s disease: An exploratory radiomic analysis study, Ann. Transl. Med. 10 (9) (2022) 513, http://dx.doi.org/10.21037/atm21-4349. [219] J. Jiang, M. Wang, I. Alberts, X. Sun, T. Li, A. Rominger, C. Zuo, Y. Han, K. Shi, for the Alzheimer’s Disease Neuroim Initiative, Using radiomics-based modelling to predict individual progression from mild cognitive impairment to Alzheimer’s disease, Eur. J. Nucl. Med. Mol. Imaging 49 (7) (2022) 2163–2173, http://dx.doi.org/10.1007/s00259-022-05687-y. [220] S.P. Caminiti, T. Ballarini, A. Sala, C. Cerami, L. Presotto, R. Santangelo, F. Fallanca, E.G. Vanoli, L. Gianolli, S. Iannaccone, G. Magnani, D. Perani, L. Parnetti, P. Eusebi, G. Frisoni, F. Nobili, A. Picco, E. Scarpini, FDG-PET and CSF biomarker accuracy in prediction of conversion to different dementias in a large multicentre MCI cohort, NeuroImage Clin. 18 (2018) 167–177, http://dx.doi.org/10.1016/j.nicl.2018.01.019. [221] J. Zamani, A. Sadr, A.-H. Javadi, Classification of early-MCI patients from healthy controls using evolutionary optimization of graph measures of restingstate fMRI, for the Alzheimer’s disease neuroimaging initiative, in: S.D. Ginsberg (Ed.), PLoS One 17 (6) (2022) e0267608, http://dx.doi.org/10.1371/journal. pone.0267608. [222] S. Luz, F. Haider, S. de la Fuente, D. Fromm, B. MacWhinney, Detecting Cognitive Decline Using Speech Only: The Adresso Challenge, Cold Spring Harbor Laboratory, 2021, http://dx.doi.org/10.1101/2021.03.24.21254263. [223] R. Chakraborty, M. Pandharipande, C. Bhat, S.K. Kopparapu, Identification of dementia using audio biomarkers, 2020, http://dx.doi.org/10.48550/ARXIV. 2002.12788, arXiv. [224] S. de la Fuente Garcia, C.W. Ritchie, S. Luz, Artificial intelligence, speech, and language processing approaches to monitoring Alzheimer’s disease: A systematic review, J. Alzheimer. Dis. 78 (4) (2020) 1547–1574, http://dx.doi.org/10.3233/ jad-200888. [225] E. Perez-Valero, J. Minguillon, C. Morillas, F. Pelayo, M.A. Lopez-Gordo, Detection of Alzheimer’s disease using a four-channel EEG montage, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 436–445, http://dx.doi.org/10. 1007/978-3-031-06242-1_43. [226] A. Gomez-Valades, R. Martinez-Tomas, M. Rincon, Integrative base ontology for the research analysis of Alzheimer’s disease-related mild cognitive impairment, Front. Neuroinform. 15 (2021) http://dx.doi.org/10.3389/fninf.2021.561691. [227] A.G.-V. Batanero, M.R. Zamorano, R.M. Tomás, J.G. Martín, Evaluating imputation methods for missing data in a MCI dataset, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 446–454, http://dx.doi.org/10.1007/978-3-031-062421_44. [228] M.J. Leming, S. Baron-Cohen, J. Suckling, Single-participant structural similarity matrices lead to greater accuracy in classification of participants than function in autism in MRI, Mol. Autism 12 (1) (2021) http://dx.doi.org/10.1186/ s13229-021-00439-5. [229] J.M. Górriz, J. Ramírez, F. Segovia, F.J. Martínez, M.-C. Lai, M.V. Lombardo, S. Baron-Cohen, J.S. and, A machine learning approach to reveal the NeuroPhenotypes of autisms, Int. J. Neural Syst. 29 (07) (2019) 1850058, http: //dx.doi.org/10.1142/s0129065718500582. [230] Z. Salimi, E. Jenabi, S. Bashirian, Are social robots ready yet to be used in care and therapy of autism spectrum disorder: A systematic review of randomized controlled trials, Neurosci. Biobehav. Rev. 129 (2021) 1–16, http: //dx.doi.org/10.1016/j.neubiorev.2021.04.009. [231] V. Knight, B.R. McKissick, A. Saunders, A review of technology-based interventions to teach academic skills to students with autism spectrum disorder, J. Autism Dev. Disord. 43 (11) (2013) 2628–2648, http://dx.doi.org/10.1007/ s10803-013-1814-y. Information Fusion 100 (2023) 101945 36 J.M. Górriz et al. [232] P.W.S. Leung, S.X. Li, C.S.O. Tsang, B.L.C. Chow, W.C.W. Wong, Effectiveness of using mobile technology to improve cognitive and social skills among individuals with autism spectrum disorder: Systematic literature review, JMIR Mental Health 8 (9) (2021) e20892, http://dx.doi.org/10.2196/20892. [233] M. van Otterdijk, M. de Korte, I. van den Berk-Smeekens, J. Hendrix, M. van Dongen-Boomsma, J. den Boer, J. Buitelaar, T. Lourens, J. Glennon, W. Staal, E. Barakova, The effects of long-term child–robot interaction on the attention and the engagement of children with autism, Robotics 9 (4) (2020) 79, http://dx.doi.org/10.3390/robotics9040079. [234] R. McEwen, Mediating sociality: The use of iPod touch™devices in the classrooms of students with autism in Canada, Inf. Commun. Soc. 17 (10) (2014) 1264–1279, http://dx.doi.org/10.1080/1369118x.2014.920041. [235] J. Li, E. Barakova, J. Hu, W. Staal, M. van Dongen-Boomsma, ApEn: A stressaware pen for children with autism spectrum disorder, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 281–290, http://dx.doi.org/10.1007/978-3-03106242-1_28. [236] J.M. Vicente-Samper, E. Ávila-Navarro, J.M. Sabater-Navarro, Feasibility study of a ML-based ASD monitoring system, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 270–280, http://dx.doi.org/10.1007/978-3-031-062421_27. [237] J.M. Vicente-Samper, E. Avila-Navarro, J.M. Sabater-Navarro, Data acquisition devices towards a system for monitoring sensory processing disorders, IEEE Access 8 (2020) 183596–183605, http://dx.doi.org/10.1109/access.2020. 3029692. [238] J.M. Vicente-Samper, E. Avila-Navarro, V. Esteve, J.M. Sabater-Navarro, Intelligent monitoring platform to evaluate the overall state of people with neurological disorders, Appl. Sci. 11 (6) (2021) 2789, http://dx.doi.org/10. 3390/app11062789. [239] M. Jodra-Chuan, P. Maestro-Domingo, V. Rodellar-Biarge, Anxiety monitoring in autistic disabled people during voice recording sessions, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 291–300, http://dx.doi.org/10.1007/978-3031-06242-1_29. [240] A. Ortiz, J.E. Arco, M.A. Formoso, N.J. Gallego-Molina, I. Rodríguez-Rodríguez, J. Martínez-Murcia, J.M. Górriz, J. Ramírez, Towards mixed mode biomarkers: Combining structural and functional information by deep learning, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 95–103, http://dx.doi.org/10. 1007/978-3-031-06242-1_10. [241] N.J. Gallego-Molina, A. Ortiz, F.J. Martínez-Murcia, M.A. Formoso, A. Giménez, Complex network modeling of EEG band coupling in dyslexia: An exploratory analysis of auditory processing and diagnosis, Knowl.-Based Syst. 240 (2022) 108098, http://dx.doi.org/10.1016/j.knosys.2021.108098. [242] I. Rodríguez-Rodríguez, A. Ortiz, M.A. Formoso, N.J. Gallego-Molina, J.L. Luque, Inter-channel Granger causality for estimating EEG phase connectivity patterns in dyslexia, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 53–62, http://dx.doi.org/10.1007/978-3-031-06242-1_6. [243] B. Gholami, M.H. Behboudi, A. Khadem, A. Shoeibi, J.M. Gorriz, Sleep apnea diagnosis using complexity features of EEG signals, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 74–83, http://dx.doi.org/10.1007/978-3-031-06242-1_8. [244] D. López-García, J.M. González-Peñalver, J.M. Górriz, M. Ruz, Representational similarity analysis: A preliminary step to fMRI-EEG data fusion in MVPAlab, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 84–94, http://dx.doi.org/ 10.1007/978-3-031-06242-1_9. [245] D. Palacios-Alonso, A. López-Arribas, G. Meléndez-Morales, E. Núñez-Vidal, A. Gómez-Rodellar, J.M. Ferrández-Vicente, P. Gómez-Vilda, A pilot and feasibility study of virtual reality as gamified monitoring tool for neurorehabilitation, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 239–248, http://dx.doi.org/ 10.1007/978-3-031-06242-1_24. [246] C. Rodrigo-Rivero, C.G. del Olmo, A. Álvarez-Marquina, P. Gómez-Vilda, F. Domínguez-Mateos, D. Palacios-Alonso, Acquisition of relevant hand-wrist features using leap motion controller: A case of study, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 229–238, http://dx.doi.org/10.1007/978-3-031-062421_23. [247] O. de Juan-Ayala, V. Caruana, J.J. Campos-Bueno, J.M. Ferrández, E. Fernández, Pairing of visual and auditory stimuli: A study in musicians on the multisensory processing of the dimensions of articulation and coherence, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 249–258, http://dx.doi.org/ 10.1007/978-3-031-06242-1_25. [248] V. Nikolov, M. Dimitrova, I. Chavdarov, A. Krastev, H. Wagatsuma, Design of educational scenarios with BigFoot walking robot: A cyber-physical system perspective to pedagogical rehabilitation, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 259–269, http://dx.doi.org/10.1007/978-3-031-06242-1_26. [249] L. Neergard, Obama proposes ‘precision medicine’ to end one-size-fits-all, Drug Discov. Devel. (2015). [250] J.E. Hall, Guyton and Hall Textbook of Medical Physiology, Elsevier, 2020, p. 1152. [251] A. Tsanas, E. Woodward, A. Ehlers, Objective characterization of activity, sleep, and circadian rhythm patterns using a wrist-worn actigraphy sensor: Insights into posttraumatic stress disorder, JMIR mHealth and uHealth 8 (4) (2020) e14306, http://dx.doi.org/10.2196/14306. [252] C. Ozemek, R. Arena, Precision in promoting physical activity and exercise with the overarching goal of moving more, Prog. Cardiovasc. Dis. 62 (1) (2019) 3–8, http://dx.doi.org/10.1016/j.pcad.2018.12.001. [253] B. Aysin, E. Aysin, Effect of respiration in heart rate variability (HRV) analysis, in: 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2006, http://dx.doi.org/10.1109/iembs.2006.260773. [254] N.A. Posteguillo, M.P. Bonomini, The effect of breathing maneuvers on the interaction between pulse fluctuation and heart rate variability, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 369–379, http://dx.doi.org/10. 1007/978-3-031-06242-1_37. [255] S.M. Jaeggi, M. Buschkuehl, W.J. Perrig, B. Meier, The concurrent validity of the N-back task as a working memory measure, Memory 18 (4) (2010) 394–412, http://dx.doi.org/10.1080/09658211003702171. [256] M.V. Calvo, R.M. Ruiz, L. Soo, D. Wacławczyk, F. Grani, J.M. Ferrández, E.F. Jover, Horizon cyber-vision: A cybernetic approach for a cortical visual prosthesis, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 380–394, http://dx.doi.org/10.1007/978-3-031-06242-1_38. [257] D. Waclawczyk, L. Soo, M. Val, R. Morollon, F. Grani, E. Fernandez, The assessment of activities of daily living skills using visual prosthesis, in: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications, Springer International Publishing, 2022, pp. 395–404, http://dx.doi.org/10. 1007/978-3-031-06242-1_39. [258] C. Shorten, T.M. Khoshgoftaar, A survey on image data augmentation for deep learning, J. Big Data 6 (1) (2019) http://dx.doi.org/10.1186/s40537-019-01970. [259] F.J. Garcia-Espinosa, A.S. Montemayor, A. Cuesta-Infante, Automatic annotation for weakly supervised pedestrian detection, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 308–317, http://dx.doi.org/10.1007/978-3-031-065279_30. [260] L. Almón-Manzano, R. Pastor-Vargas, J.M.C. Troncoso, Deep reinforcement learning in agents’ training: Unity ML-agents, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 391–400, http://dx.doi.org/10.1007/978-3-031-065279_39. [261] L. Myllyaho, M. Raatikainen, T. Männistö, T. Mikkonen, J.K. Nurminen, Systematic literature review of validation methods for AI systems, J. Syst. Softw. 181 (2021) 111050, http://dx.doi.org/10.1016/j.jss.2021.111050. [262] J.D. Lope, M. Graña, A hybrid time-distributed deep neural architecture for speech emotion recognition, Int. J. Neural Syst. 32 (06) (2022) http://dx.doi. org/10.1142/s0129065722500241. [263] C. Jimenez-Mesa, J. Ramirez, J. Suckling, J. Vöglein, J. Levin, J.M. Gorriz, A non-parametric statistical inference framework for deep learning in current neuroimaging, Inf. Fusion 91 (2023) 598–611, http://dx.doi.org/10.1016/j. inffus.2022.11.007. [264] A.C. Yu, B. Mohajer, J. Eng, External validation of deep learning algorithms for radiologic diagnosis: A systematic review, Radiol. Artif. Intell. 4 (3) (2022) http://dx.doi.org/10.1148/ryai.210064. [265] E. Gibney, Could machine learning fuel a reproducibility crisis in science? Nature 608 (7922) (2022) 250–251, http://dx.doi.org/10.1038/d41586022-02035-w. [266] J.H. Drake, A. Kheiri, E. Özcan, E.K. Burke, Recent advances in selection hyperheuristics, European J. Oper. Res. 285 (2) (2020) 405–428, http://dx.doi.org/ 10.1016/j.ejor.2019.07.073. [267] J. Gamazo, J.M. Cuadra, M. Rincón, An efficient and rotation invariant Fourier-based metric for assessing the quality of images created by generative models, in: Bio-Inspired Systems and Applications: From Robotics to Ambient Intelligence, Springer International Publishing, 2022, pp. 413–422, http://dx. doi.org/10.1007/978-3-031-06527-9_41. [268] E. Chang, H.T. Kim, B. Yoo, Virtual reality sickness: A review of causes and measurements, Int. J. Hum.-Comput. Interact. 36 (17) (2020) 1658–1682, http://dx.doi.org/10.1080/10447318.2020.1778351. [269] R. Underwood, E. Tolmeijer, J. Wibroe, E. Peters, L. Mason, Networks underpinning emotion: A systematic review and synthesis of functional and effective connectivity, NeuroImage 243 (2021) 118486, http://dx.doi.org/10. 1016/j.neuroimage.2021.118486. [270] W. Mellouk, W. Handouzi, Facial emotion recognition using deep learning: Review and insights, Procedia Comput. Sci. 175 (2020) 689–694, http://dx. doi.org/10.1016/j.procs.2020.07.101. Information Fusion 100 (2023) 101945 37 J.M. Górriz et al. [271] B. García-Martínez, A. Fernández-Caballero, L. Zunino, A. Martínez-Rodrigo, Recognition of emotional states from EEG signals with nonlinear regularityand predictability-based entropy metrics, Cogn. Comput. 13 (2) (2020) 403–417, http://dx.doi.org/10.1007/s12559-020-09789-3. [272] R. Sánchez-Reolid, F.L. de la Rosa, M.T. López, A. Fernández-Caballero, Onedimensional convolutional neural networks for low/high arousal classification from electrodermal activity, Biomed. Signal Process. Control 71 (2022) 103203, http://dx.doi.org/10.1016/j.bspc.2021.103203. [273] M. Balconi, A. Frezza, M.E. Vanutelli, Emotion regulation in schizophrenia: A pilot clinical intervention as assessed by EEG and optical imaging (functional near-infrared spectroscopy), Front. Hum. Neurosci. 12 (2018) http://dx.doi.org/ 10.3389/fnhum.2018.00395. [274] J. Diemer, G.W. Alpers, H.M. Peperkorn, Y. Shiban, A. Mühlberger, The impact of perception and presence on emotional reactions: A review of research in virtual reality, Front. Psychol. 6 (2015) http://dx.doi.org/10.3389/fpsyg.2015. 00026. [275] A. Fernández-Caballero, E. Navarro, P. Fernández-Sotos, P. González, J.J. Ricarte, J.M. Latorre, R. Rodriguez-Jimenez, Human-avatar symbiosis for the treatment of auditory verbal hallucinations in schizophrenia through virtual/augmented reality and brain-computer interfaces, Front. Neuroinform. 11 (2017) http://dx.doi.org/10.3389/fninf.2017.00064. [276] J. Gutiérrez-Maldonado, M. Rus-Calafell, J. González-Conde, Creation of a new set of dynamic virtual reality faces for the assessment and training of facial emotion recognition ability, Virtual Real. 18 (1) (2013) 61–71, http: //dx.doi.org/10.1007/s10055-013-0236-7. [277] J. del Aguila, L.M. González-Gualda, M.A. Játiva, P. Fernández-Sotos, A. Fernández-Caballero, A.S. García, How interpersonal distance between avatar and human influences facial affect recognition in immersive virtual reality, Front. Psychol. 12 (2021) http://dx.doi.org/10.3389/fpsyg.2021.675515. [278] A.S. García, P. Fernández-Sotos, P. González, E. Navarro, R. Rodriguez-Jimenez, A. Fernández-Caballero, Behavioral intention of mental health practitioners toward the adoption of virtual humans in affect recognition training, Front. Psychol. 13 (2022) http://dx.doi.org/10.3389/fpsyg.2022.934880. [279] E. Wingerden, E. Barakova, T. Lourens, P.S. Sterkenburg, Robot-mediated therapy to reduce worrying in persons with visual and intellectual disabilities, J. Appl. Res. Intellect. Disabil. 34 (1) (2020) 229–238, http://dx.doi.org/10. 1111/jar.12801. [280] F.J. Toledo-Moreo, J.J. Martínez-Alvarez, J. Garrigós-Guerrero, J.M. FerrándezVicente, FPGA-based architecture for the real-time computation of 2-d convolution with large kernel size, J. Syst. Archit. 58 (8) (2012) 277–285, http://dx.doi.org/10.1016/j.sysarc.2012.06.002. [281] H. Wang, D.-Y. Yeung, Towards Bayesian deep learning: A framework and some existing methods, IEEE Trans. Knowl. Data Eng. 28 (12) (2016) 3395–3408, http://dx.doi.org/10.1109/tkde.2016.2606428. [282] A.D. Kiureghian, O. Ditlevsen, Aleatory or epistemic? Does it matter? Struct. Saf. 31 (2) (2009) 105–112, http://dx.doi.org/10.1016/j.strusafe.2008.06.020. [283] R. Cipolla, Y. Gal, A. Kendall, Multi-task learning using uncertainty to weigh losses for scene geometry and semantics, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, http://dx.doi.org/10. 1109/cvpr.2018.00781. [284] A.C. Damianou, N.D. Lawrence, Deep Gaussian processes, 2012, http://dx.doi. org/10.48550/ARXIV.1211.0358, arXiv. [285] J.E. Arco, A. Ortiz, J. Ramírez, F.J. Martínez-Murcia, Y.-D. Zhang, J.M. Górriz, Uncertainty-driven ensembles of multi-scale deep architectures for image classification, Inf. Fusion 89 (2023) 53–65, http://dx.doi.org/10.1016/j.inffus. 2022.08.010. [286] V. Vapnik, The Nature of Statistical Learning Theory, Springer science & business media, 1999. [287] I. Castiglioni, L. Rundo, M. Codari, G.D. Leo, C. Salvatore, M. Interlenghi, F. Gallivanone, A. Cozzi, N.C. D’Amico, F. Sardanelli, AI applications to medical images: From machine learning to deep learning, Phys. Medica 83 (2021) 9–24, http://dx.doi.org/10.1016/j.ejmp.2021.02.006. [288] L. Xu, X. Wang, L. Bai, J. Xiao, Q. Liu, E. Chen, X. Jiang, B. Luo, Probabilistic SVM classifier ensemble selection based on GMDH-type neural network, Pattern Recognit. 106 (2020) 107373, http://dx.doi.org/10.1016/j.patcog.2020. 107373. [289] X. Qian, Z. Zhou, J. Hu, J. Zhu, H. Huang, Y. Dai, A comparative study of kernel-based vector machines with probabilistic outputs for medical diagnosis, Biocybern. Biomed. Eng. 41 (4) (2021) 1486–1504, http://dx.doi.org/10.1016/ j.bbe.2021.09.003. [290] A. Lombardi, D. Diacono, N. Amoroso, P. Biecek, A. Monaco, L. Bellantuono, E. Pantaleo, G. Logroscino, R.D. Blasi, S. Tangaro, R. Bellotti, A robust framework to investigate the reliability and stability of explainable artificial intelligence markers of mild cognitive impairment and Alzheimer’s disease, Brain Inform. 9 (1) (2022) http://dx.doi.org/10.1186/s40708-022-00165-5. [291] W. Yan, G. Qu, W. Hu, A. Abrol, B. Cai, C. Qiao, S.M. Plis, Y.-P. Wang, J. Sui, V.D. Calhoun, Deep learning in neuroimaging: Promises and challenges, IEEE Signal Process. Mag. 39 (2) (2022) 87–98, http://dx.doi.org/10.1109/msp.2021. 3128348. [292] K.M. Poloni, R.J. Ferrari, A deep ensemble hippocampal CNN model for brain age estimation applied to Alzheimer’s diagnosis, Expert Syst. Appl. 195 (2022) 116622, http://dx.doi.org/10.1016/j.eswa.2022.116622. [293] H. Han, Implementation of Bayesian multiple comparison correction in the second-level analysis of fMRI data: With pilot analyses of simulation and real fMRI datasets based on voxelwise inference, Cogn. Neurosci. 11 (3) (2019) 157–169, http://dx.doi.org/10.1080/17588928.2019.1700222. [294] M. Sharmin, M.M. Hossain, A. Saha, M. Das, M. Maxwell, S. Ahmed, From research to practice: Informing the design of autism support smart technology, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ACM, 2018, http://dx.doi.org/10.1145/3173574.3173676. [295] X. Jiang, L.E. Boyd, Y. Chen, G.R. Hayes, ProCom, in: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, ACM, 2016, http://dx.doi.org/10.1145/2968219.2971445. [296] O. Weisberg, A. GalOz, R. Berkowitz, N. Weiss, O. Peretz, S. Azoulai, D. KoplemanRubin, O. Zuckerman, TangiPlan, in: Proceedings of the 2014 Conference on Interaction Design and Children, ACM, 2014, http://dx.doi.org/10.1145/ 2593968.2610475. [297] J. Dafflon, P.F.D. Costa, F. Váša, R.P. Monti, D. Bzdok, P.J. Hellyer, F. Turkheimer, J. Smallwood, E. Jones, R. Leech, A guided multiverse study of neuroimaging analyses, Nature Commun. 13 (1) (2022) http://dx.doi.org/10. 1038/s41467-022-31347-8. [298] P. Sharma, A.P. Shukla, A review on brain tumor segmentation and classification for MRI images, in: 2021 International Conference on Advance Computing and Innovative Technologies in Engineering, ICACITE, IEEE, 2021, http://dx. doi.org/10.1109/icacite51222.2021.9404662. [299] S.A. Ajagbe, K.A. Amuda, M.A. Oladipupo, O.F. AFE, K.I. Okesola, Multiclassification of alzheimer disease on magnetic resonance images (MRI) using deep convolutional neural network (DCNN) approaches, Int. J. Adv. Comput. Res. 11 (53) (2021) 51–60, http://dx.doi.org/10.19101/ijacr.2021.1152001. [300] R. Kumari, A. Nigam, S. Pushkar, An efficient combination of quadruple biomarkers in binary classification using ensemble machine learning technique for early onset of Alzheimer disease, Neural Comput. Appl. 34 (14) (2022) 11865–11884, http://dx.doi.org/10.1007/s00521-022-07076-w. [301] S. Wang, M.E. Celebi, Y.-D. Zhang, X. Yu, S. Lu, X. Yao, Q. Zhou, M.-G. Miguel, Y. Tian, J.M. Gorriz, I. Tyukin, Advances in data preprocessing for biomedical data fusion: An overview of the methods, challenges, and prospects, Inf. Fusion 76 (2021) 376–421, http://dx.doi.org/10.1016/j.inffus.2021.07.001. [302] G. Chen, D.S. Pine, M.A. Brotman, A.R. Smith, R.W. Cox, P.A. Taylor, S.P. Haller, Hyperbolic trade-off: The importance of balancing trial and subject sample sizes in neuroimaging, NeuroImage 247 (2022) 118786, http://dx.doi. org/10.1016/j.neuroimage.2021.118786. [303] P.P. Angelov, E.A. Soares, R. Jiang, N.I. Arnold, P.M. Atkinson, Explainable artificial intelligence: An analytical review, WIREs Data Min. Knowl. Discov. 11 (5) (2021) http://dx.doi.org/10.1002/widm.1424. [304] A. Lombardi, J.M.R.S. Tavares, S. Tangaro, Editorial: Explainable artificial intelligence (XAI) in systems neuroscience, Front. Syst. Neurosci. 15 (2021) http://dx.doi.org/10.3389/fnsys.2021.766980. [305] I.B. Galazzo, F. Cruciani, L. Brusini, A. Salih, P. Radeva, S.F. Storti, G. Menegaz, Explainable artificial intelligence for magnetic resonance imaging aging brainprints: Grounds and challenges, IEEE Signal Process. Mag. 39 (2) (2022) 99–116, http://dx.doi.org/10.1109/msp.2021.3126573. [306] C. Jiménez-Mesa, J.E. Arco, M. Valentí-Soler, B. Frades-Payo, M.A. Zea-Sevilla, A. Ortiz, M. Ávila-Villanueva, D. Castillo-Barnes, J. Ramírez, T.D. Ser-Quijano, C. Carnero-Pardo, J.M. Górriz, Using explainable artificial intelligence in the clock drawing test to reveal the cognitive impairment pattern, Int. J. Neural Syst. 33 (04) (2023) http://dx.doi.org/10.1142/s0129065723500156. [307] M. Kiani, J. Andreu-Perez, H. Hagras, S. Rigato, M.L. Filippetti, Towards understanding human functional brain development with explainable artificial intelligence: Challenges and perspectives, IEEE Comput. Intell. Mag. 17 (1) (2022) 16–33, http://dx.doi.org/10.1109/mci.2021.3129956. [308] M. Ghassemi, L. Oakden-Rayner, A.L. Beam, The false hope of current approaches to explainable artificial intelligence in health care, The Lancet Dig. Health 3 (11) (2021) e745–e750, http://dx.doi.org/10.1016/s2589-7500(21) 00208-9. [309] J. DiPietro, A. Kelemen, Y. Liang, C. Sik-Lanyi, Computerand robot-assisted therapies to aid social and intellectual functioning of children with autism spectrum disorder, Medicina 55 (8) (2019) 440, http://dx.doi.org/10.3390/ medicina55080440. [310] M.S. Jaliaawala, R.A. Khan, Can autism be catered with artificial intelligenceassisted intervention technology? A comprehensive survey, Artif. Intell. Rev. 53 (2) (2019) 1039–1069, http://dx.doi.org/10.1007/s10462-019-09686-8. [311] K. Kutt, D. Drążyk, L. Żuchowska, M. Szelążek, S. Bobek, G.J. Nalepa, BIRAFFE2, a multimodal dataset for emotion-based personalization in rich affective game environments, Sci. Data 9 (1) (2022) http://dx.doi.org/10.1038/ s41597-022-01402-6.