Assist-as-needed EMG-based control strategy for wearable powered assistive devices

Luís Carlos Rodrigues Moreira Assist-As-Needed EMG-based Control Strategy for Wearable Powered Assistive Devices Dissertação de Mestrado Mestrado Integrado em Engenharia Biomédica Ramo Eletrónica Médica Trabalho realizado sob a orientação de Professora Doutora Cristina P. Santos, Universidade do Minho Doutora Joana Sofia Campos Figueiredo, Universidade do Minho Doutora Elena García Armada, Consejo Superior de Investigaciones Científicas – Universidad Politécnica de Madrid Dezembro de 2019 Universidade do Minho Escola de Engenharia ii DIREITOS DE AUTOR E CONDIÇÕES DE UTILIZAÇÃO DO TRABALHO POR TERCEIROS Este é um trabalho académico que pode ser utilizado por terceiros desde que respeitadas as regras e boas práticas internacionalmente aceites, no que concerne aos direitos de autor e direitos conexos. Assim, o presente trabalho pode ser utilizado nos termos previstos na licença abaixo indicada. Caso o utilizador necessite de permissão para poder fazer um uso do trabalho em condições não previstas no licenciamento indicado, deverá contactar o autor, através do RepositóriUM da Universidade do Minho. Licença concedida aos utilizadores deste trabalho iii ACKNOWLEDGMENTS Firstly, I want to thank my parents and brother for all the support, conditions, comfort and emotional strength that was given to me over the last five years. All the academic success was achieved because of your presence. I would like to thank my advisor, Professor Cristina P. Santos, for the opportunity to work on this project, for all the guidance and motivational conversations towards the achievement of the main goals. To my co-worker, Doctor Joana Figueiredo, I want to thank for all the availability, for give me the best advices, for being a constant support along this dissertation and for all the patience. You are an example to follow. Thanks to my lab colleagues for all the good daily disposition, coffees, game of cards and for all the delicious Wednesday’ snacks. To the best friends that Braga gave me, João Mendes Lopes, Cristiana Pinheiro, Margarida Machado, Ana Pereira, Carla Pereira, Luciana Meneses and Pedro Mouta. We started together and we will continue always together. When and where is the next dinner? You are great people! To all my musician friends of Banda Musical e Cultural da Vila de Rio de Moinhos, thank you for all the moments out of the work, for all the social meetings, barbecues and good music that we made together. Last but not least, I have to thank my girlfriend, Mariana Costa, for being present in all the most important moments of my life, for giving the support and the strength to never give up and for all the times that you said “Força Engenheirinho!”. You were essential in the success of this journey. Thank you very much, Luís Moreira iv STATEMENT OF INTEGRITY I hereby declare having conducted this academic work with integrity. I confirm that I have not used plagiarism or any form of undue use of information or falsification of results along the process leading to its elaboration. I further declare that I have fully acknowledged the Code of Ethical Conduct of the University of Minho. v RESUMO A assistência e reabilitação robótica usando dispositivos de assistência ativos vestíveis (WPADs), como ortóteses e exosqueletos, tem crescido na área da reabilitação com o fim de recuperar e aumentar a função motora de sujeitos com alterações neurológicas. Estes dispositivos devem fornecer uma assistência personalizada, uma vez que a condição física e a fadiga muscular variam de paciente para paciente. Nesta área, sinais de eletromiografia (EMG) têm sido usados para controlar WPADs, dada a sua capacidade de inferir a intenção de movimento do utilizador. Contudo, em casos de deficiência motora, os sinais de EMG apresentam menor amplitude quando comparados com sinais de EMG em condições saudáveis e, portanto, o uso de WPADs geridos por sinais de EMG pode não oferecer a assistência que o paciente necessita. O principal objetivo desta dissertação visa o desenvolvimento de uma estratégia de controlo baseada em EMG capaz de fornecer assistência quando necessário, para futura integração num sistema ortótico ativo e inteligente (SmartOs). Para atingir este objetivo foram desenvolvidos e validados os seguintes elementos: (i) sistema de EMG para adquirir sinais de atividade muscular dos músculos mais relevantes no movimento da articulação do tornozelo; (ii) ferramenta de machine learning para estimação do binário da articulação do tornozelo para servir como referência na estratégia de controlo; e (iii) ferramenta de estimação do binário real do tornozelo considerando sinais de EMG dos músculos Tibialis Anterior (TA) e Gastrocnemius Lateralis (GASL) e ângulo real do tornozelo. O sistema de EMG apresentou correlações satisfatórias com um sistema comercial. O binário de referência para o tornozelo foi gerado com base no ângulo de referência da mesma articulação, velocidade de marcha (de 1 até 4 km/h) e dados antropométricos (alturas de 1.51 m até 1.83 e massas de 52.0 kg até 83.7 kg), usando cinco algoritmos de machine learning : Support Vector Machine , Random Forest , Multilayer Perceptron , Long-Short Term Memory e Convolutional Neural Network . CNN apresentou a melhor performance, prevendo binários de referência do tornozelo com um fit entre 74.7 e 89.8 % e Normalized Root Mean Square Errors (NRMSE) entre 3.16 e 8.02 %. A estimativa do torque com base em sinais de EMG requer a inclusão de um maior número de músculos, uma vez que sinais de EMG dos músculos TA e GASL não foram suficientes. PALAVRAS-CHAVE Dispositivos de Assistência Ativos Vestíveis, Estratégia de Controlo baseada em EMG, Modelos de Regressão, Eletromiografia, Reabilitação da Marcha vi ABSTRACT Robotic-based gait rehabilitation and assistance using Wearable Powered Assistive Devices (WPADs), such as orthosis and exoskeletons, has been growing in the rehabilitation area to recover and augment the motor function of neurologically impaired subjects. These WPADs should provide a personalized assistance, since physical condition and muscular fatigue modify from patient to patient. In this field, electromyography (EMG) signals have been used to control WPADs given their ability to infer the user’s motion intention. However, in cases of motor disability conditions, EMG signals present lower magnitudes when compared to EMG signals under healthy conditions. Thus, the use of WPADs managed by EMG signals may not have potential to provide the assistance that the patient requires. The main goal of this dissertation aims the development of an Assisted-As-Needed (AAN) EMGbased control strategy for a future insertion in a Smart Active Orthotic System (SmartOs). To achieve this goal, the following elements were developed and validated: (i) an EMG system to acquire muscle activity signals from the most relevant muscles during the motion of the ankle joint; (ii) machine learning-based tool for ankle joint torque estimation to serve as reference in the AAN EMG-based control strategy; and (iii) a tool for real EMG-based torque estimation using Tibialis Anterior (TA) and Gastrocnemius Lateralis (GASL) muscles and real ankle joint angles. EMG system showed satisfactory pattern correlations with a commercial system. The reference ankle joint torque was generated based on predicted reference ankle joint kinematics, walking speed information (from 1 to 4 km/h) and anthropometric data (body height from 1.51 m to 1.83 m and body mass from 52.0 kg to 83.7 kg), using five machine learning algorithms: Support Vector Regression (SVR), Random Forest (RF), Multilayer Perceptron (MLP), Long-Short Term Memory (LSTM) and Convolutional Neural Network (CNN). CNN provided the best performance, predicting the reference ankle joint torque with fitting curves ranging from 74.7 to 89.8 % and Normalized Root Mean Square Errors (NRMSEs) between 3.16 and 8.02 %. EMG-based torque estimation beneficiates of a higher number of muscles, since EMG data from TA and GASL are not enough to estimate the real ankle joint torque. KEYWORDS Wearable Powered Assistive Devices, EMG-based Control Strategy, Regression Models, Electromyography, Gait Rehabilitation vii CONTENTS Acknowledgments ............................................................................................................................... iii Resumo............................................................................................................................................... v Abstract.............................................................................................................................................. vi List of Figures ...................................................................................................................................... x List of Tables ..................................................................................................................................... xii List of Abbreviations and Acronyms ................................................................................................... xiv Chapter 1 – Introduction ..................................................................................................................... 1 1.1. Motivation ........................................................................................................................... 2 1.2. Problem Statement ............................................................................................................. 3 1.3. Goals and Research Questions ............................................................................................ 3 1.4. Contributions ...................................................................................................................... 5 1.5. Thesis Outline ..................................................................................................................... 6 Chapter 2 – State of the Art ................................................................................................................ 7 2.1. Physiological Aspects .......................................................................................................... 7 2.2. Electromyographic Signals ................................................................................................... 8 2.2.1. Overview ..................................................................................................................... 8 2.2.2. Commercial Systems ................................................................................................... 9 2.3. Assistive Control Strategies ................................................................................................ 10 2.3.1. EMG-based Control Strategies .................................................................................... 10 viii 2.3.2. AAN EMG-based Control Strategies ............................................................................ 13 2.3.3. Discussion ................................................................................................................. 15 2.4. EMG-based Torque Estimation ........................................................................................... 16 2.4.1. Proportional Gain Methods ......................................................................................... 16 2.4.2. Musculoskeletal Models ............................................................................................. 17 2.4.3. Empirical Methods ..................................................................................................... 19 2.4.4. Discussion ................................................................................................................. 23 2.5. General Conclusions .......................................................................................................... 24 Chapter 3 – System Overview ........................................................................................................... 25 3.1. SmartOs Description ......................................................................................................... 25 3.2. Proposed AAN EMG-based Control Strategy ....................................................................... 27 3.3. Wired EMG Acquisition System .......................................................................................... 28 3.3.1. Hardware Specifications ............................................................................................ 29 3.3.2. Experimental Validation Protocol ................................................................................ 32 3.3.3. EMG Signal Processing .............................................................................................. 33 3.3.4. Results and Discussion .............................................................................................. 33 3.4. General Conclusions .......................................................................................................... 35 Chapter 4 – Ankle Kinematics Trajectory Generation ......................................................................... 36 4.1. Introduction....................................................................................................................... 36 4.2. Methods ............................................................................................................................ 38 4.2.1. Data Acquisition ........................................................................................................ 38 4.2.2. Regression Model Implementation ............................................................................. 40 4.2.3. Regression Model Evaluation Metrics ......................................................................... 43 4.3. Results and Discussion ...................................................................................................... 44 4.4. General Conclusions .......................................................................................................... 48 Chapter 5 – Ankle Kinetics Trajectories Generation ........................................................................... 50 5.1. Introduction....................................................................................................................... 50 5.2. Methods ............................................................................................................................ 51 5.2.1. Regression Models..................................................................................................... 51 ix 5.2.2. Data Preparation ....................................................................................................... 57 5.2.3. Machine Learning Evaluation Metrics ......................................................................... 59 5.3. Results .............................................................................................................................. 59 5.3.1. Support Vector Regression ......................................................................................... 59 5.3.2. Random Forest .......................................................................................................... 60 5.3.3. Multilayer Perceptron ................................................................................................. 61 5.3.4. Long Short-Term Memory Neural Network .................................................................. 63 5.3.5. Convolutional Neural Network .................................................................................... 65 5.4. Discussion and General Conclusions.................................................................................. 67 Chapter 6 – AAN EMG-based Control Strategy ................................................................................... 70 6.1. Introduction....................................................................................................................... 70 6.2. Reference Ankle Joint Torque Prediction ............................................................................ 71 6.3. EMG-based Real Joint Torque Estimation ........................................................................... 73 6.3.1. Model Presentation .................................................................................................... 73 6.3.2. Model Adaptation ....................................................................................................... 74 6.3.3. Model Validation ........................................................................................................ 76 6.4. General Conclusions .......................................................................................................... 78 Chapter 7 – Conclusions .................................................................................................................. 79 7.1. Future Work ...................................................................................................................... 82 References ....................................................................................................................................... 84 xvi QF Quadriceps Femoris R Correlation Coefficient R2 Coefficient of Determination RBFNN Radial Basis Function Neural Network RF Random Forest RF Rectus Femoris RMSE Root Mean Square Error RMSJ Root Mean Square Jerk RQ Research Question sEMG Surface EMG SM Semimembranosus SmartOS Smart Active Orthotic System SOL Soleus ST Semitendinosus SVR Support Vector Regression TA Tibialis Anterior TF Tensor Fasciae TSt Terminal Stance TSw Terminal Swing VI Vastus Intermedius VL Vastus Lateralis vm Maximal speed of the muscle VM Vastus Medialis WPAD Wearable Powered Assistive Device γ Optimal Fiber Length 1 CHAPTER 1 – INTRODUCTION This dissertation presents the work developed in the scope of the fifth year of the Integrated Master’s in Biomedical Engineering during the academic year of 2018/19. During the first semester, skills in the field of the human gait cycle were achieved at Marsi Bionics S.L. in Madrid, Spain, integrated into an ERASMUS placement program. With this experience, it was possible to learn competences regarding the human gait pattern based on the analysis of its main characteristics and strategies. As a result, a hybrid dynamic and kinematic model to simulate a healthy human gait cycle was constructed, fed and validated with data collected at the gait laboratory of the company. The work presented in this dissertation was developed during the second semester and the concepts acquired in the first semester were fundamental to achieve the main goals of this project. This dissertation was developed at BiRD LAB (Biomedical Robotic Devices Laboratory) of the Center of MicroElectroMechanical Systems (CMEMs), at University of Minho, Braga, Portugal. This dissertation addresses the development of a control strategy for personalized human gait rehabilitation with a Wearable Powered Assistive Device (WPAD). To achieve this goal, an Assisted-As-Needed (AAN) EMGbased control strategy was projected, combining concepts of machine learning, to predict the reference gait kinematics and kinetics oriented to the user, with the development of musculoskeletal models to determine the real gait kinematics and kinetics of the user in real-time. 2 1.1. Motivation According to [1], stroke events correspond to the second leading cause of death and the third leading cause of disability in the world. The phenomenon behind these episodes is related to the absence of oxygen in the brain tissue due to a rupture of an artery in the brain (hemorrhagic stroke) or due to a blocking in the blood flow (ischemic stroke) [2]. Based on [3], 63% of the stroke survivors cannot walk without external support, not being able to perform their daily life activities. Consequently, the patient’s quality life is affected due to social and work exclusion, costly medical assistance and early retirement [4]. The hemiparesis and hemiplegia are the two major consequences derived from a stroke. Hemiparetic patients exhibit weakness on one side of the body, whereas hemiplegic patients present complete paralysis on one side of the body. With this information, the residual lower limb muscle force of the hemiparetic patients can be considered into their gait rehabilitation [5]. The lower limbs rehabilitation has been changed in the last years, to deal with (i) the disadvantages associated with the interand intra-therapist variances; (ii) the dependency of the malleability of the patient’s joint (commonly affected by spasticity); and (iii) the absence of precise and repeatable movements during therapy. For this purpose, in the rehabilitation area, the robotic assistance integrating WPADs, such as orthosis and exoskeletons, has steadily gained importance [6]. The first generation of WPADs for lower limbs, integrating trajectory tracking control strategies, is responsible to concern a cyclic pattern to the user, based on pre-programmed trajectories [7]. However, in these cases, the patient participation is reduced, since the effort required by the user to perform the walking motion is small. According to [8], [9], the rehabilitation process is more efficient if the encouragement of the patient participation is achieved. On the other side, it was already proved that these trajectory tracking strategies, when applied to hemiparetic patients, are not efficient in the rehabilitation context [10], [11]. In such circumstances, the incapacity level varies from patient to patient and it also varies during the rehabilitation process. From this perspective, since the physical condition and muscular fatigue comprehend different aspects that modify from patient to patient, a personalized assistance should be provided [12]. A human-machine interaction has been highlighted by applying bioinspired control architectures with user-oriented assistive control strategies integrated in WPADs. Moreover, these control strategies are designed to consider information about the user’s motor condition and motion intention, by using biosignals acquired with biomedical sensors [13]. In this context, Electromyography (EMG) signals have been 3 widely used since they can forecast information related to the user’s motion intention, namely 20 – 100 ms before the user’s lower limb motion [6], [14], [15]. In this sense, the provision of functional assistance is possible if EMG-based assistive control strategies are implemented into WPADs, avoiding the muscle atrophy. 1.2. Problem Statement Notwithstanding the EMG-based control assistive strategies help to avoid muscle atrophy, this strategy is destinated to follow the intentions of the user. However, in cases of impairments of the lower limbs, the muscle weakness results in lower EMG signals when compared to EMG signals from healthy subjects [16]. Consequently, the use of EMG-based control strategies to manage the assistance delivered by WPADs may not provide the assistance that the patient needs to walk [17]. To combat this phenomenon, while considering the patient motion intention, studies have proposed AAN control strategies to provide the assistance needed by the patient to perform the walking motion [6]. The conventional AAN strategies consider the trajectory of the user’s lower limbs and a predefined reference trajectory, not considering the human condition and motion intention [6]. Due to this, there are difficulties related to (i) the synchronism between the reference trajectory and the user’s movement; and (ii) the adaptation of the reference trajectory according to the user-specific needs and intentions [18]. These drawbacks can be overpassed using EMG signals, taking advantage of their anticipatory performance, by the construction of an AAN EMG-based control strategy. Moreover, the EMG signals can be useful as a metric of the muscle weakness and, thus, when integrated in AAN control strategies, a personalized assistance could be provided [13]. This dissertation explores the potential of AAN strategies based on EMG signals for a future integration into a WPAD, in order to provide a personalized assistance in real-time, considering the physical condition and the motion intention of each user. 1.3. Goals and Research Questions The ultimate goal of this dissertation aims the development of an AAN EMG-based control strategy towards the personalized rehabilitation of the ankle joint using Smart Active Orthotic System (SmartOs). The strategy should be adapted for each user, providing only the required assistance in real-time. The development of EMG-based control strategies requires the conversion of EMG signals into joint torques, particularly ankle joint torques [14], [19]. On the other side, the joint torques are dependent on 4 the joint kinematics and, thus, to perform the ankle joint torques prediction for specific subjects, the ankle joint kinematics should be well adapted. For this purpose and considering the main goal of this project, several objectives were established: • Objective 1: To perform a literature search to collect the main control strategies already developed to assist and to restore the lower limbs functions, using EMG signals. Perform a literature search to collect the most relevant works to convert EMG signals into joint torque values. This objective is addressed in Chapter 2. • Objective 2: To develop an EMG system to monitor the muscular activity of lower limb muscles, particularly the most important muscles responsible for the ankle motion. The system must correctly detect the muscle activations with low noise and with few motion artifacts. Validate the effectiveness of the developed EMG system with a commercial solution. This objective is addressed in Chapter 3. • Objective 3: To implement and validate an effective method to estimate a reference joint position trajectory, namely ankle joint kinematics, tackling the variability of the walking speed and subject anthropometric data. This objective is addressed in Chapter 4. • Objective 4: To develop and validate an automatic and accurate machine learning-based method to estimate a reference joint torque trajectory, considering the user-oriented reference joint kinematics and the variability of the walking speed and subject anthropometric data. This objective is addressed in Chapter 5. • Objective 5: To implement and validate an efficient method to estimate the real user’s joint torque, based on EMG signals and joint angles towards the implementation of the AAN EMG-based strategy. This objective is addressed in Chapter 6. With this dissertation, four Research Questions (RQs) were identified and answered, in order to complete the main challenges of the project: • RQ1: Which are the contributions and the main differences of the EMG-based control and the AAN EMG-based control strategies? • RQ2: Is it possible to obtain joint torque measures only using EMG signals? • RQ3: Is it possible to predict reference walking kinematics and kinetics trajectories relying exclusively on the walking speed and anthropometric data? 5 • RQ4: Can EMG-based torque estimation strategy present a good performance? 1.4. Contributions The main contributions of this dissertation are: • A descriptive literature review reporting the assistive control strategies based on EMG signals and integrated into WPADs; • A descriptive literature review reporting the EMG-based torque estimation methods; • A tool to estimate reference ankle joint position trajectories based on the walking speed and anthropometric data; • A machine learning-based method for reference ankle joint torque estimation considering useroriented reference ankle joint kinematics trajectories, walking speed and anthropometric data; • A method to perform an EMG-based torque estimation using EMG signals and real ankle joint position trajectories, towards the implementation into a WPAD destinated for ankle joint assistance and rehabilitation. Furthermore, the developed work allowed the publication of four conference papers: • Moreira, L., Pinheiro, C., Lopes, J. M., Sanz-Merodio, D., Figueiredo, J., Santos, C. P., & Garcia, E. (2019). Study of Gait Cycle Using a Five-Link Inverted Pendulum Model: First Developments. In 2019 IEEE 6th Portuguese Meeting on Bioengineering (ENBENG) (pp. 1–4). IEEE. https://doi.org/10.1109/ENBENG.2019.8692451 • Lopes, J. M., Moreira, L. Pinheiro, C., Sanz-Merodio, D., Figueiredo, J., Santos, C. P., & Garcia, E. (2019). Three-Link Inverted Pendulum for Human Balance Analysis: A Preliminary Study. In 2019 IEEE 6th Portuguese Meeting on Bioengineering (ENBENG) (pp. 1–4). IEEE. https://doi.org/10.1109/ENBENG.2019.8692531 • Pinheiro, C., Lopes, J. M., Moreira, C., Sanz-Merodio, D., Figueiredo, J., Santos, C. P., & Garcia, E. (2019). Kinematic and kinetic study of sit-to-stand and stand-to-sit movements towards a human-like skeletal model. In 2019 IEEE 6th Portuguese Meeting on Bioengineering (ENBENG) (pp. 1–4). IEEE. https://doi.org/10.1109/ENBENG.2019.8692569 • Fernandes, P. N., Figueiredo, J., Moreira, L., Félix, P., Correia, A., Moreno, J. C., & Santos, C. P. (2019). EMG-based Motion Intention Recognition for Controlling a Powered Knee Orthosis. In 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) (pp. 1–6). IEEE. https://doi.org/10.1109/ICARSC.2019.8733628 6 • Moreira, L., Figueiredo, J., Garcia, E. & Santos, C. P. (2020). Myoelectric Control Strategies Applied in Powered Lower Limb Assistive Devices: A Review. In Robotics and Autonomous Systems (Submitted). 1.5. Thesis Outline This dissertation is organized in the following seven chapters. Chapter 2 presents the state of the art addressed to four main points: (i) the main muscles responsible for the lower limbs joint motion; (ii) the most used EMG systems to acquire EMG signals; (iii) the EMG-based control and AAN EMG-based control strategies already implemented and integrated into WPADs to restore the lower limbs functions; and (iv) the EMG-based torque estimation methods already developed. Chapter 3 exhibits an insight about SmartOs architecture, presenting information about its constituent hardware and software. It is also presented the proposed AAN EMG-based control strategy and the chapter ends with the hardware of the EMG system developed in this dissertation. Chapter 4 presents a regression model to generate reference ankle joint kinematics, based on walking speed and anthropometric data. Additionally, it describes the data acquisition protocol to validate all the algorithms developed in this dissertation. In Chapter 5, machine learning algorithms are developed and validated to predict the ankle joint torques, that will be used as a reference parameter on the control strategy proposed in Chapter 3. Chapter 6 presents the validation of the results obtained in Chapter 4 and Chapter 5, where the output results obtained in Chapter 4 serve as input data to the best algorithm developed in Chapter 5. Moreover, the chapter describes the implemented algorithm responsible for EMG-based real joint torque estimation and presents its validation. Lastly, Chapter 7 presents the main conclusions of this master dissertation, the research questions are answered and topics for a future work are proposed. 7 CHAPTER 2 – STATE OF THE ART This chapter begins with a brief description of the healthy and the pathological walking motion, identifying the main muscles affected by a stroke event and the main muscles responsible for the motion of the knee and ankle joints. Subsequently, commercial EMG systems are presented, as well as their main characteristics. It is followed by an exhaustive review of the control strategies already developed and applied into WPADs, using EMG signals to assist and restore the lower limbs functions. At last, studies related to the conversion of the EMG signals into torque values are also presented, since most of the EMG-based control strategies perform this conversion [14], [19]. 2.1. Physiological Aspects Stroke events cause functional or neuromuscular changes, depending on the location of the affected area of the brain. According to [16], [20], these changes are related to a loss of strength in the hemiparetic leg, being verified weakness in the flexor muscles and spasticity in the extensor muscles, causing an increase of the joint stiffness. Studies concluded that, generally, in stroke survivors, the Soleus (SOL) and Gastrocnemius (GAS) muscles (responsible for the plantar flexion motion of the ankle joint) are contracted due to spasticity, while the Tibialis Anterior (TA) muscle (responsible for the dorsiflexion motion of the same joint) remains weak [2], [16], [20], [21]. This is the main reason of the drop foot disorder in patients that suffered a stroke event [22]. Regarding the knee joint, the knee extensors ( Vastus Medialis (VM), Vastus Lateralis (VL), Vastus Intermedius (VI) and Rectus Femoris (RF)) exhibit spasticity and the knee flexors ( Biceps Femoris (BF), Semitendinosus (ST) and Semimembranosus (SM)) are weak, being 8 verified a knee hyperextension and a decrease in the flexion movement of this joint during the gait cycle [16], [20], [21]. In this sense, considering the muscles affected by stroke events and according to [23], the muscles that provide more information about the ankle joint motion are the TA muscle for dorsiflexion and the SOL or the GAS muscles for plantar flexion. Concerning the knee joint, the muscles that present more information are the BF muscle for flexion and the VM and the RF muscles for extension. 2.2. Electromyographic Signals 2.2.1. Overview The contraction and the relaxation episodes of the muscles are controlled by the nervous system, through electric signals delivered by the neurons to the muscles. This electrical activity can be recorded using EMG, providing a powerful forecast information related to the motion intention of the user [6], [14], [15], [24]. EMG signals can be measured using needles inserted directly in the muscles (intramuscular EMG – iEMG, also known as fine-wire EMG – fEMG) or using electrodes placed on the skin (surface EMG - sEMG). The choice of the method to measure the electrical activity of the muscles depends on the properties of the muscles to study. Comparing both, the first enounced method presents less cross-talk, since it receives lower muscle activities of the muscles around the desired muscle [25]. However, due to the difficulty of insertion, greater invasiveness and higher cost, this method is not normally used in EMG studies [25], [26]. In contrast, sEMG technique is commonly used, since it is non-invasive, practical, inexpensive and it may be used by non-clinical assessors. In addition, some studies reported that this method, when applied to the SOL, GAS and TA muscles, presents EMG amplitudes similar to iEMG and a negligible value of cross-talk [27]–[29]. Regarding the sEMG (referred as “EMG” along this dissertation), there are two configurations to adopt, namely the monopolar and the bipolar. The monopolar configuration uses two electrodes: detection electrode placed on the skin, above the muscle in study; and a reference electrode that should be placed on an electrically neural tissue or a bone [30]. However, according to [31], this configuration is not recommended, since the signal-noise ratio and the spatial resolution of the EMG signals are reduced, when compared to EMG acquisitions performed with bipolar configurations. On the other side, the bipolar configuration uses three electrodes: two detection electrodes placed above the target muscle, distanced by 10 - 20 mm [32], [33] and a reference electrode. In this sense, the bipolar configuration is 9 preferable, since the common noise of both detection electrodes is eliminated, increasing the signal-noise ratio and producing a cleaner EMG signal [33], [34]. 2.2.2. Commercial Systems The characteristics of the EMG signals are well established in the literature and the most relevant are the amplitude range between ±10 mV, the response frequency between 0 and 500 Hz with the dominant energy between 50 and 150 Hz and the stochastic nature, characterized by a Gaussian distribution function [35]. Delsys, Noraxon, MotionLab, Plux, BTS Bioengineering, Contemplas, Cometa, Biopac, Shimmer, Cadwell, Myontec and Athos are examples of companies that produce EMG acquisition systems able to acquire EMG signals with the aforementioned characteristics. An EMG acquisition system can reach the tens of thousands of euros and, thus, depending on the finality of each project, it is required to analyze the specifications of each system. Table 1 summarizes the main characteristics of the 4 more used commercial EMG acquisition systems. Table 1. EMG systems specifications Delsys [36], [37] Noraxon [38] Motion Lab [39] MuscleBAN [40], [41] EMG Signal Input Range ± 11 mV ± 24 mV ± 250 mV * Resolution 16 bits 16 bits 16 bits 16 bits EMG Signal Bandwith 20 – 450 Hz Minimum: 20 – 500 Hz 10 – 2000 Hz 1 – 1000 Hz Maximum: 5 – 1500 Hz Sampling Rate 2000 Hz 2000 Hz or 4000 Hz 4000 Hz 4000 Hz Transmission Range 40 m 40 m 18 m 10 m Gain 1 1 10 – 500 1100 Sensor Size 27 x 37 x 13 mm 24 x 37 x 16 mm 38 x 19 x 9 mm 28 x 70 x 12 mm Mass 14 g 14 g 20 g 25 g Autonomy 2 – 3 h 8 h * 8 h Temperature Range 5 – 45 degrees Celsius 0 – 38 degrees Celsius 20 – 40 degrees Celsius * *Information not specified 16 Furthermore, the works developed by [17], [56]–[60] reveal to be a suitable approach to assist and rehabilitate the knee joint, when the user requires support. Based on the results, two situations were verified: (i) if the support ratio provided by the WPAD increases, the user’s torque decreases, but the knee joint angle remains equal; and (ii) if the support ratio decreases, the user’s torque increases and the knee joint angle remains equal once again. This indicates that the AAN strategy provides the assistance to perform the movements when required, offering great potential to motor rehabilitation and user’s motor autonomy. Nonetheless, the muscle model should be improved and suitable for each muscle, in order to consider different activation delays of each muscle. Based on the AAN EMG-based model strategies already developed, it is concluded that if the user does not perform any movement, the exoskeleton provides the enough amount of torque to complete the desired knee joint trajectory. On the other hand, if the wearer has an electric signal associated, the exoskeleton only provides a torque upon the user’s torque, in order to achieve a correct knee joint trajectory. 2.4. EMG-based Torque Estimation Most of the implemented control strategies involved the joint torque estimation of the real user’s joint torque, based on the EMG signals acquired from specific muscles. For this purpose, a literature search was made to review the methods applied to convert the EMG signals into torque values, namely proportional gain methods, musculoskeletal models and empirical models, as presented in Table 4. 2.4.1. Proportional Gain Methods The studies developed by [13], [43], [44] followed the same methodology to obtain the joint torque, based on a proportional gain method. They carried out a calibration step to find proportional gains per joint, for flexion and extension motions. For this purpose, the participants had to maintain a fixed joint angle, while the orthosis/exoskeleton actuator performed a fixed torque value. This process implied that the participant needed to perform a joint torque to maintain the required joint angle. At the same time, the EMG signals were read and, a proportional gain that relates the EMG signals with the joint torque was found. In the three works, the estimated joint torque was similar to the real one. In [13], a Normalized Root Mean Square Error (NRMSE) and a phase delay between the measured torque and the estimated torque was were calculated, achieving values of 12 % and 22 ms, respectively. In [44], the RMSE and the Root Mean Square Jerk (RMSJ) between the estimated and the measured torque was 3.56 ± 0.63 Nm and 2.85 ± 0.78 Nm, respectively. 17 2.4.2. Musculoskeletal Models The Hill-type model proposed by [46] and developed in [47] was the first musculoskeletal model proposed to estimate joint torque using EMG signals and joint angles. Further variations of this model have been developed, improving the calibration process, in order to minimize the model uncertainty. In this calibration process, muscle parameters are identified to construct the model, following a trade-off between the accuracy of the model and its complexity [51]. Based on [46], [47], the work of [48] predicts the knee joint torque, using EMG signals from thirteen muscles and the knee joint angles. Eighteen parameters were found in the calibration process to construct the model. Among these parameters, the force produced by the muscle-tendon unit ( Fmt ), depends on the pennation angle (ɸ) that depends on L0 m . Consequently, L0 m is function of the percentage change in optimal fiber length, γ. This percentage was varied in this study to investigate the importance of a correct calibration step in musculoskeletal models. The feasibility of the conversion was assessed by six subjects, who performed approximately 204 tasks, including dynamometer, running and sidestepping trials. Coefficient of determination ( R2 ) of 0.91 ± 0.04 was obtained between the knee joint torque predicted by the model and by inverse dynamics. A mean residual error below 0.2 Nm/kg normalized to body weight was achieved. These results were obtained with γ = 15 %. When γ was set to 0 %, a R2 of 0.85 was obtained, showing the sensitivity of the model to the parameters of the calibration step. Based on the works developed by [47], [48], other six works were developed aiming the simplification of the model, the reduction of the conversion time and the reduction of the conversion errors. In [62] the user motion intentions for the knee joint were mapped, using EMG signals (listed in Table 4) and knee joint angles. At the same time, the joint angles and the ground reaction forces served as input to the seven-link biomechanical model (composed by two legs with feet, shanks, thighs and the torso) to obtain the knee joint torque through inverse dynamics. When compared to [48], only two parameters were calibrated (presented in Table 4). The effectiveness of the methodology was evaluated in one healthy subject during stepping up stairs and with flexion and extension motions of the knee joint. A good correlation between both joint torques was achieved, but the amplitudes of the curves were different in some cases. The study advanced by [49] considered fewer calibration parameters than [48], aiming the reduction of the time of the EMG-based torque conversion. For this purpose, the tendon strain parameter (found in the calibration process of [48]) was neglected, since this parameter represents only 3.3 % of the tendon length when a maximum isometric force is generated. This neglection allows the estimation 18 of the L0 m without the time-consuming Runge-Kutta-Fehlerg integration, proposed in [48]. Considering the estimated and the experimental knee joint torque, a correlation coefficient ( R ) of 0.892 ± 0.047 and a RMSE of 8.10 ± 1.02 Nm, when a healthy subject performed 10 gait trials. Moreover, the calibration was completed in 63.4 ± 1.20 s and the joint torque estimation in this simplified model took 0.0630 s, whereas the complete model [48] took 3 h and 0.691 ± 0.0146 s to accomplish the same the calibration and the estimation tasks, respectively. Inspired in the model developed by [47], in [51] the knee joint torques were estimated, using the EMG signals of six muscles (listed in Table 4) and the knee joint angles. Eight parameters were determined in the calibration process (also presented in Table 4), since the authors considered this number as an acceptable tradeoff between the complexity of the model and its uncertainty. Experiments were carried out with a healthy subject performing flexion and extension movements of the knee joint. A NRMSE between the torque obtained with the model and the measured torque from inverse dynamics was 12.4 %. Moreover, in [63], a musculoskeletal model was developed, based on the Hill-type muscle model developed by [48]. EMG data were recorded from three muscles of a healthy person, along with the joint angles and they acted as input in the model. In the calibration step, four parameters were identified, as presented in Table 4. The RMSE and the R2 were calculated to evaluate the model performance, obtaining values below 1.99 Nm and an average of 0.89, respectively. In [44], in addition to the developed linear proportional model, an EMG-driven Hill-type neuromuscular model was also proposed, based on [47]. Two muscles were used to perform the conversion and only three parameters were determined in the calibration process. The effectiveness of the conversion was tested in eight healthy subjects, performing maximum isometric voluntary contractions at different angles. RMSE of 3.49 ± 0.57 Nm and RMSJ of 1.21 ± 0.51 Nm were obtained between the estimated and the measured torque. An EMG-driven musculoskeletal model was proposed in [64], in real-time, to predict the user intentions of the ankle and knee joint motions. This musculoskeletal model was based on [48], where the EMG signals and the ankle and knee joint angles are the model’s inputs. This methodology was implemented in a Raspberry Pi 2 to investigate the computational cost. Three different tasks were performed by five different participants: (i) walking at self-selected speed, (ii) knee squat, and (iii) calf rise motion. The estimated torques by the musculoskeletal model were compared with the calculated joint torques using inverse dynamics. The results of the EMG-driven model were presented in 2.7 ± 0.48 ms. RMSE were always below 0.37 ± 0.12 Nm/kg and the smallest value observed was 0.01 ± 0.01 Nm/kg. 19 Pearson coefficients were also calculated and the values were always above 0.43 ± 0.36, and the highest correlation value obtained was 0.9 ± 0.07. 2.4.3. Empirical Methods From the literature research, it was verified that empirical methods, such as neuro-fuzzy models [53], [55] and neural networks [11], [65], [66] have been applied to estimate the joint torque based on EMG signals. Study [53] only used EMG data of eight muscles as input in a neuro-fuzzy model to obtain the hip and knee joint torques. To construct this model, twenty fuzzy IF-THEN control rules were defined and a value of joint torque was matched. In [55], the EMG signals along with the joint angles served as input to obtain the joint torque. In both works is not presented the comparison between the real torque and the estimated torque. When performing different movements, if the EMG signals were reduced, the research concluded that the EMG-based controller is adapted to the user. A Multilayer Perceptron (MLP) neural network was developed in [66] to estimate the knee joint torque. EMG data normalized to the maximal isometric activation, along with the body mass, height, age, gender, joint velocity, and joint position were entered as input variables in the three-layer neural network. The second layer was composed by a variable number of hidden units. The third layer estimated joint torque of the knee joint. The results ( R = 0.96) demonstrated the effectiveness of this neural network to estimate the knee joint torque. Later, in [65], the authors improved the MLP presented in [66]. First, the research concluded that how many more muscles were incorporated, better results are achieved. Second, better results were obtained when normalized data at each isometric angle was used in training process of the neural network, because it was proved that the curve of the normalized joint torque at each angle is similar to the curve of the normalized EMG. Third, results pointed out that MLP neural network has a better joint torque estimation when compared with other neural networks, such as Fully Connected Cascade. At last, five neurons in the second layer were found as the best number of hidden units. A RBFNN with a two-step learning strategy was developed by [11] to convert the EMG signals into torque values. The performance of the methodology was evaluated through simulations and experiments with four healthy subjects only for the swing phase. A RMSE of 2 Nm and a R higher than 0.8 were found, between the measured and the estimated joint torque. 20 Table 4. Methods to convert EMG signals into joint torque values Study Conversion Method Used in Control? Calibration Parameters Inputs Joint Muscles Participants (number) Results [43] Proportional Gain Method Yes Two gain parameters/joint EMG data Hip Knee BF, VM, GM and RF Healthy (1) * [13] Proportional Gain Method Yes Two gain parameters/joint EMG data Knee VL, VM, SM and ST Healthy (2) NRMSE = 12 % Phase delay = 22 ms [44] Proportional Gain Method Yes One gain parameter/joint Joint angles and EMG data Ankle TA and GAS Healthy (8) RMSE = 3.56 ± 0.63 Nm RMSJ = 2.85 ± 0.78 Nm [48] Musculoskeletal Model No Eighteen parameters ** Joint angles and EMG data Knee Thirteen muscles ** Healthy (6) R2 = 0.91 ± 0.04 Mean Residual Error/body weight < 0.2 Nm/kg [62] Musculoskeletal Model No F max , F mt , and A Joint angles and EMG data Knee * Healthy (*) Good correlation between the shape of the knee joint torque obtained by inverse dynamics and by the EMGbased model [49] Musculoskeletal Model No F max , Fmt , L0 m , ɸ and maximal speed of the muscle ( v m ) Joint angles and EMG data Knee SM, ST, BF, Sartorius (SAR), TF, GRA, VL, VM, VI, RF, Gastrocnemius Medialis (GASM) and Gastrocnemius Lateralis (GASL) Healthy (1) R = 0.892 ± 0.047 RMSE = 8.1 ± 1.02 Nm Calibration time = 63.4 ± 1.20 s Joint torque estimation time = 0.0630 s [51] Musculoskeletal Model Yes F max , F mt , L0 m , ɸ, A and three constants Joint angles and EMG data Knee RF, VM, VL, BF, SM and ST Healthy (1) NRMSE = 12.4 % 21 [63] Musculoskeletal Model No F max , F mt , L0 m , ɸ Joint angles and EMG data Knee RF, VM and VL Healthy (1) RMSE = 1.99 R2 = 0.89 [44] Musculoskeletal Model Yes F max , F mt , L0 m and A Joint angles and EMG data Ankle TA and GAS Healthy (8) RMSE = 3.49 ± 0.57 Nm RMSJ = 1.21 ± 0.51 Nm [64] Musculoskeletal Model No F max , F mt , L0 m and A Joint angles and EMG data Ankle and Knee BF, GASL, GASM, GRA, RF, SAR, SOL, SM, VL, VM, TA, Peroneus Longus (PL) and Peroneus Tertius (PT) Healthy (5) Joint torque estimation time = 2.7 ± 0.48 ms RMSE < 0.37 ± 0.12 Nm/kg Pearson correlations > 0.43 ± 0.36 [53] Empirical Model Yes *** EMG data GRF data Hip Knee TF, RF, VL, VM, AL, GRA, BF and ST Healthy (1) * [55] Empirical Model Yes *** Joint angles and EMG data Hip Knee TF, RF, VL, AL, GRA, VM, BF and ST Healthy (3) * [66] Empirical Model No *** Body mass, body height, age, gender, joint velocity, joint position and EMG data Knee VL and BF Healthy (20) R = 0.96 [65] Empirical Model No *** Body mass, body height, age, gender, joint velocity, joint position and EMG data Knee VL, BF, RF, VM and ST Healthy (1) • More muscles imply a better joint torque estimation; • The data should be normalized at each isometric angle; • The second layer should have five neurons. 22 [11] Empirical Model Yes *** Joint angles, velocities and EMG data Hip QF and BF Healthy (4) RMSE < 2 Nm R > 0.8 * Not specified ** Consult [48] for more information *** Not Required 23 2.4.4. Discussion Among the different strategies, three methods were identified to convert the EMG signals into joint torque values: Proportional Gain Methods, Musculoskeletal Models and Empirical Methods. It was reported by [46], [67] that the muscle activation, the architectural and biomechanical properties of the muscle fibers and tendons are determinant factors in the force of muscle-tendon units. For this purpose, in a study developed by [68], reported that only the use of EMG signals in proportional gain method is not enough to predict the torque of the ankle joint. Moreover, [69] reported that the proportional torque control only using EMG values is not reliable for two-fold reasons: (i) the human body muscles and coactivation effects are exclusive from person to person; and (ii) the EMG readings are dependent on the electrodes placement, location and skin impedance. Study [70] showed the differences between using only EMG signals and motion data fusion (EMG and other sensors data), concluding that the fusion of biomechanical data with EMG signals improves joint torque estimation. Additionally, study [43] combined floor reaction force with EMG signals to improve the hip torque at extension motion and consequently, overcome the user discomfort during stance phase. Regarding the musculoskeletal models [44], [48], [49], [51], [62]–[64], there are several factors to consider to obtain a good EMG-torque conversion, as follows: the number of calibration parameters to find, the number of muscles to acquire the EMG signals, the complexity and the accuracy of the model, and the time to perform the conversion. The complexity of the model increases with the increment of (i) the number of calibration parameters; and (ii) the number of muscles. Consequently, the computational burden will increase. The works developed by [48], [62] are not capable to perform the conversion in real-time, in contrast with the works developed later by [44], [49], [51], [63], [64]. The real-time conversion is an important aspect to consider when it is desired to assist patients using EMG signals in torque controllers. As already referred, the proportional gain and the musculoskeletal methods require calibration steps that may not be realized by pathological individuals. For this purpose, the empirical models, as the ones presented in [11], [53], [55], [65], [66], can be a suitable strategy to convert the EMG signals into torque values, avoiding calibration steps. In the study developed by [65], some improvements in a MLP neural network were done. It was reported that the number of muscles, the inputs normalization and the number of hidden units had a strong impact in the performance of the conversion. However, the dataset used to construct the neural network presented twenty subjects where the age range of the participants, as well as the range of physical activity of the participants was small. Moreover, the EMG-torque conversion implemented in both 24 works [65], [66] is not completed in real-time. In [11], the joint torque is obtained through a RBFNN, where the increment of the number of muscles leads good results without calibration steps. However, the error of the torque estimation increases when the hip movement of the orthosis’s joint changed its direction. At last, this methodology was only developed for the swing phase, requiring more advances to cover all the gait cycle. 2.5. General Conclusions There is interest in using EMG signals to control WPADs, since they are directly related to user motion intention. Two types of control strategies integrating EMG data were identified, the EMG-based control and the AAN EMG-based control. Both strategies contributed to muscular activity improvement and endurance. EMG-based control strategies can be used to re-train the lower limbs of patients with impairments. However, it is not expectable that these strategies have the potential to assist their motor condition. AAN EMG-based control strategies have the potential to assist and rehabilitate the lower limbs, such that the user can achieve autonomy to perform their daily life activities, in contrast to EMG-based control strategies. It is expectable that the integration of AAN EMG-based control strategies into WPADs may foster motor assistance, re-train, rehabilitation and autonomy when applied to disabled users. However, validation tests with pathological subjects are still missing to validate this phenomenon. Furthermore, until the moment, there is no explanation by the existence of AAN EMG-based control strategies only destinated to the knee joint. Further, this review concluded that most of the control methods which use EMG signals to control WPADs convert these EMG signals into torque values. There is evidence that only EMG readings are not enough to obtain the accurate values of joint torques, being necessary to fuse EMG data with biomechanical data, such as, joint angles, velocities, accelerations. Moreover, to perform the EMG-based torque estimation in AAN EMG-based control strategies, it was noticed that only musculoskeletal models were used. Overall, this dissertation aims the development of an AAN EMG-based control strategy destinated to the ankle joint assistance, using a musculoskeletal model fed with EMG signals, joint angles and velocities to perform the EMG-based torque estimation, since there were not found literature evidences of the use of these strategies to assist the referred joint. 25 CHAPTER 3 – SYSTEM OVERVIEW In this chapter, it is presented an overview relative to the orthotic system used in this dissertation to assist individuals with disabilities in the ankle joint. The control architectures already implemented in this orthotic system are briefly explained and the AAN EMG-based control strategy is proposed. At last, the chapter ends with the description of the constructed EMG acquisition system, that represents an essential element of the proposed control strategy. 3.1. SmartOs Description SmartOs is a WPAD embedded with wearable sensors destinated to the analysis and control of the gait motion. SmartOs was developed in [71] and it presents the capacity of providing a personalized and repetitive gait training, such as joint trajectory tracking control, to assist the user according with its needs and to enable the abnormal gait pattern correction, such as drop foot gait in stroke survivors. Besides that, this WPAD invokes the user participation, in order to rehabilitate the ankle joint, by considering the user’s motion intention based on muscular information (an example of an EMG-based control strategy) and it was designed to achieve walking speeds ranging from 0.5 to 1.6 km/h [71]. Moreover, SmartOs also present a biofeedback system integrated in its architecture to accelerate the familiarization of the user to itself and to accelerate the gait recovery. The conceptual design of SmartOs is presented in Figure 1 and a brief explanation of each block is provided. Nonetheless, the scope of this dissertation is the projection of an AAN EMG-based control strategy that has not yet been developed. 32 Analyzing Figure 5, differences with respect to the attenuation levels between the experimental and theoretical implementations are visible. Based on the theoretical FR, it would be expectable to achieve higher levels of attenuation in the desirable range of frequencies. However, despite of lower levels of attenuation have been verified experimentally, this phenomenon does not compromise the proper functioning of the projected system. Moreover, the experimental attenuation levels introduced by the filters at frequencies ranging from 80 to 300 Hz are better than the expected ones, since at these frequencies, the signal is not attenuated. The final EMG system can be seen in Figure 6. Since this system was designed to acquire EMG signals from only one muscle, other boards must be used to acquire signals from other muscles. Thus, an input/output supply was projected to supply energy between consecutive boards. Figure 6. Final EMG system. 3.3.2. Experimental Validation Protocol To analyze the feasibility of the developed EMG system, a validation trial was performed with a healthy male subject (with 23 years old, height of 1.70 m and weight of 78.1 kg) walking in a treadmill at 1 km/h, for 1 minute. The choice of the walking speed was based on the minimum (0.5 km/h) and maximum (1.6 km/h) range of speeds allowed in SmartOs [71]. The MuscleBAN EMG system of [40] 33 was used as ground truth. The detection electrodes were placed on the TA muscle and the reference electrode was placed on the knee, as exhibited in Figure 7. Figure 7. Electrode configuration adopted for a validation test to measure the electrical activity of the TA muscle. 3.3.3. EMG Signal Processing During the data collection, EMG data from ten gait cycles were acquired. To this data, the DC offset component of 1.65 V introduced in hardware was removed to achieve a signal with null mean. Then, the rectification of the signal was performed, taking the absolute value of the signal. The envelope of the EMG signal was determined using the Root Mean Square (RMS) value of the signal with a 300 ms movable window. The RMS feature was used to create the EMG envelop since it is the most suitable method to represent a physical meaning of muscle force [74]. To confirm this fact, a recent study showed that the RMS method is better correlated with the muscle contraction force when compared with other methods, such as mean absolute value, median frequency and mean power frequency, since an increase/decrease of the muscle force contraction is more easily detected with RMS value [75]. The choice of the window length (below 300 ms) was based on [76]. Higher values of window length do not cause an increase of the information. In some cases, it was reported that an extreme increase of the window length could produce a loss of information [76]. 3.3.4. Results and Discussion Results of the experimental validation trial are presented in Figure 8, where it is possible to verify that the signals acquired with the projected EMG boards present a considerable correlation level with the signals acquired with the system of [40]. Notwithstanding the ground truth system enable a better 34 muscular activation detection (verified by the higher amplitude voltage), comparing both systems, the muscular activations are detected at the same instant, conferring robustness to the constructed EMG system. Besides the comparison of the EMG signals acquired with different systems, another comparison was performed, in order to analyze the feasibility of the measure based on literature evidences. In this sense, the black perimeter delimited in Figure 8 was zoomed and the EMG signals of both systems were filled. The reason why this perimeter was chosen is because the range time selected corresponds to a single gait cycle. The EMG signals inside the zoomed perimeter were compared with literature EMG signals of a subject walking freely and, based on the results, it is possible to infer that the signals acquired with the proposed EMG system are similar to the expected ones. Figure 8. EMG signals acquired with the projected EMG system (blue line) and with a system from [40] (red line). The black signal represents the Literature data, where HS, LR, MSt, TSt, PSw, ISw, MSw and TSw mean Heel Strike, Load Response, Mid Stance, Terminal Stance, Pre-Swing, Initial Swing, Mid Swing and Terminal Swing, respectively. 35 3.4. General Conclusions In this chapter, the main characteristics and strategies presented in SmartOs were presented. Since it was identified the necessity to construct an user-oriented strategy, an AAN EMG-based control approach was proposed, aiming its future insertion into the presented WPAD. Furthermore, an EMG acquisition system was constructed to acquire signals from the main muscles reviewed in Chapter 2. A good correlation between the signals acquired with the constructed EMG system and the acquisition system of [40] was found. Moreover, the signals acquired with the developed EMG system were also compared with literature evidences with respect to the activation of the TA along a single stride. With the achieved results, it was concluded that the EMG system was correctly projected and it can be used and integrated in the proposed AAN EMG-based control strategy. Once constructed the EMG system to collect EMG signals from the muscles of interest, another requirement of the proposed AAN EMG-based control strategy is the prediction of user-oriented reference walking kinematics and kinetic trajectories, that will be the focus of the Chapter 4 and Chapter 5, respectively. 36 CHAPTER 4 – ANKLE KINEMATICS TRAJECTORY GENERATION To implement the AAN EMG-based control strategy, it is required to predict the ankle joint torques, to serve as the reference in the proposed architecture. Since the joint torque is associated with the joint kinematics [77], a fundamental step to predict the ankle joint torque consists of the generation of ankle joint kinematics for each subject. Moreover, in this field, some works already reported the possibility to model ankle joint angles based on the walking speed [78], [79]. For this purpose, it is necessary an ankle kinematics prediction model based on well-known data from the subject. In this sense, body height and body mass, as well as the walking speed are possibilities to accomplish this issue. From this perspective, this chapter is focused on the generation of reference ankle joint kinematics along the gait cycle, oriented to the subject. 4.1. Introduction Usually, pre-recorded joint trajectories of healthy individuals represent the most used method to create joint kinematics in the sagittal plane to serve as reference in assistance applications of the human gait. These trajectories are normally recorded at slow (3.2 – 5.0 km/h), normal (5.0 – 6.5 km/h) and fast (6.5 – 7.5 km/h) speeds [80]. However, the walking speed of stroke, or ISCI, or Parkinson survivor is, approximately, from 1.8 km/h to 2.5 km/h [81]. A possible solution to overcome this issue could be the recording of joint trajectories using numerous walking speeds, in order to construct a huge database to be used as reference in gait assistant applications. However, this solution is not suitable due to the high time consumption of the data collection and large amount of data, becoming impracticable to 37 advance with this approach. In this direction, regression models have been proposed to enable the prediction of joint kinematics trajectories during gait cycle [78], [79], [82]. According to [78], the ankle angle waveform during gait cycle presents a linear and quadratic relationship with the walking speed. In this sense, if joint trajectories of healthy individuals at slow walking speeds are used as reference in control strategies integrated into WPADs, the assistance will not be efficient, because people with neurological injuries prefer lower walking speeds and, consequently, different joint trajectories. For this reason, in [78], four regression equations based on the walking speed were developed to predict the peaks of the ankle joint dorsiflexion and plantar flexion during the stance and swing phases. The R2 was used as metric to evaluate the accuracy of the model. However, the highest values obtained were 0.1110, demonstrating unsatisfactory results. The equations and the results obtained by [78] can be consulted in Table 7, where 𝑣 represents the walking speed. Table 7. Regression equations and the correspondent results, achieved in [78], for each peak of the ankle joint angle Parameter Equation 𝑹𝟐 Peak Ankle Plantar Flexion (Stance Phase) −1.7583𝑣 + 9.190961 0.0496 Peak Ankle Dorsiflexion (Stance Phase) −2.4𝑣 + 13.62415 0.105 Peak Ankle Plantar Flexion (Swing Phase) 3.7834𝑣 + 12.88073 0.0870 Peak Ankle Dorsiflexion (Swing Phase) 4.16𝑣2 – 10.7498𝑣 + 10.03869 0.111 Contrary to the four equations presented by [78], in [79], only one regression equation was developed considering the linear and quadratic relationship between the walking speed and the ankle joint ankle. Better results were achieved using the equation exhibited in Table 8. Table 8. Regression equation and the correspondent results, achieved in [79], where a , b and c represent regression coefficients of the model Parameter Equation RMSE θ 𝑎𝑣2+𝑏𝑣+𝑐 2.79 ± 2.05º In accordance with other studies, the effect of the body height in the construction of the ankle trajectory is not relevant [80]. However, in [82], it was reported that if the variability of the body height is higher, this parameter may present a larger effect on the ankle trajectory. Based on this suggestion, in this dissertation, the body height was added to the equation presented in Table 8, as proposed by [82]. Thus, the ankle joint trajectories are dependent on the walking speed and body height of each subject. With this approach, the necessity of recording joint trajectories at many different body heights and 38 different walking speeds is avoided. The results of this approach are compared and validated with data acquired with a motion-capture system (Oqus; Qualysis – Motion Capture System, Göteborg, Sweden) under specific experimental conditions. 4.2. Methods 4.2.1. Data Acquisition The ground truth data were collected during locomotion tests. Despite only lower limbs joint kinematic and anthropometric data are required, joint kinetic and EMG data were also collected, being useful data for the next chapters. A. Volunteers The study was performed with sixteen adult subjects (8 males and 8 females with mean age of 23.8 ± 2.02 years, mean weight of 67.5 ± 10.8 kg and mean height of 1.69 ± 0.109 m), with no evidence of any type of physical and physiological disorder that could interfere with their walking pattern, performed walking tests. The minimum and maximum body height registered in the data collection was 1.51 and 1.83 m, respectively and, consequently, the reconstruction of the ankle joint trajectories can only be performed in this range. B. Materials The data acquisition was performed using a motion-capture system with 12 cameras (Oqus; Qualysis – Motion Capture System, Göteborg, Sweden), acquiring at 200 Hz, five force platforms embedded in the floor (with characteristics presented in Table 9), acquiring at 200 Hz and an 8-channel wireless electromyograph (Delsys, Massachussetts, United States of America) [37], acquiring at 2000 Hz. Table 9. Force plates used in the data acquisition Force Plate System Model Quantity Bertec, Ohio, United States of America FP4060 2 Bertec, Ohio, United States of America FP6090 2 Kistler, Winterthur, Switzerland 9281 EA – FP4060 1 39 C. Subject Preparation The Newington-Helen Hayes model was adopted as marker set, integrating four more markers placed in trochanter, medial tuberosity of the femur, medial malleolus and in the first metatarsal head to obtain results more accurate [83]. In this sense, a total of 24 reflective markers were used. In relation to the EMG signals, they were extracted from TA, GASL, BF and VL of both legs. Both markers and EMG sensors are presented in Figure 9. Figure 9. EMG sensors and markers placement on the user’s body. First, the user was instrumented with EMG sensors. Then, two maximum voluntary contractions were performed for each muscle, in order to normalize all the EMG data to the maximum isometric contraction registered. After this step, the 24 reflective markers were placed on the body and then, the subjects were asked to perform a standing static calibration with their arms crossed in front of the chest, looking forward and with their feet in a comfortable position, aligned with the shoulders. This step was useful to fit the anthropometric data to the body model of the acquisition system. D. Walking Experiments All subjects were instructed to perform 10 walking trials on a 10-meter flat surface with 5 embedded force platforms, at seven different speeds (1.0, 1.5, 2.0, 2.5, 3.0, 3.5 and 4.0 km/h), controlled with a metronome. Between each speed change, the subjects rested for one minute and they performed a habituation trial to become acquainted with the new walking speed. E. Data Collection and Processing The data collected were the joint angles, angular velocity and angular acceleration for ankle, knee and hip joints, the ground reaction force and the EMG data from TA, GASL, BF and VL. 40 Following the data collection of joint angles, the kinematic and kinetic data were filtered with a lowpass Butterworth filter. The cutoff frequency chosen was 6 Hz, since in [84], a frequency around 6 and 7 Hz was determined as the optimal cutoff frequency to smooth trajectories during the walking motion. Regarding the EMG data, a band-pass filter with 20 and 450 Hz as cutoff frequencies was applied to the raw data. Further, the EMG envelop was determined using the RMS value of the signal with a 300 ms movable window. In this chapter, only ankle joint position trajectories will be used. The kinetic and EMG data will be useful for the Chapters 5 and 6. 4.2.2. Regression Model Implementation The implementation of the regression model to estimate ankle joint trajectories during walking motion was performed using MATLAB ® and it was inspired in the work developed in [82]. Based on the collected kinematic data, the ankle joint angles were split into individual gait cycles, where the first sample corresponds to the heel-strike event. For each trial (that represents a single gait cycle), the joint angle values and the instant frame at seven key-events (listed in Table 10) were extracted. Table 10. Key-events considered to construct the ankle joint angle Key-event Instant of the stride 1 Heel-Strike 2 Minimum Angle of the Stance Phase 3 Minimum Angular Velocity of the Stance Phase 4 Maximum Angle of the Stance Phase 5 Minimum Angle of the Swing Phase 6 Maximum Angle of the Swing Phase 7 Heel-Strike To detect all these events, two strategies were adopted: (i) creation of a detection algorithm based on the second derivative of the ankle joint angle (angular velocity); (ii) creation of a detection algorithm with reference to the maximums and minimums peaks of the ankle joint angle. In both strategies, the 1st and the 7th event correspond to the first and final samples of each trial, respectively. In case (i), the events number 2, 4, 5 and 6 were extracted based on a zero-cross detection. Every time that the angular velocity of the ankle joint crossed the zero value, an event was marked. To distinguish between a dorsiflexion or a plantar flexion event, the previous and the next samples in relation 41 to the zero-cross value were also considered. If the previous and the next samples corresponded to positive and negative values, respectively, a dorsiflexion event was considered, whereas if the previous and the next samples correspond to negative and positive values, respectively, a plantar flexion event was marked. Only two plantar flexion and two dorsiflexion events must be detected. At last, the event number 3 corresponds to the minimum angular velocity between the minimum (second event) and the maximum angle of the stance phase (forth event). Figure 10 presents the detection of all key-events using the strategy (i). Figure 10. Identification of the seven key-events enounced on Table 10, for a walking speed of 4 km/h. However, mainly at slow walking speeds, the event detection procedure does not work efficiently due to irregularities in the pattern of the ankle joint angles and angular velocities collected, as presented in Figure 11. Figure 11. Poor identification of the seven key-events enounced on Table 10, for a walking speed of 1 km/h. 48 Despite the best result has been achieved for a walking speed of 4 km/h, based on Figure 13, the walking speed that is able to produce results with a smaller error distribution is 3 km/h, because at this velocity, 99.3 % of the predictions present a RMSE below 5 º, a R between 0.871 and 0.981, a GOF from 30 to 70 % and a NRMSE below 20 %. Additionally, there is another limitation in the current regression model. The proposed methodology will provide the same ankle joint trajectories for individuals with the same body height, walking at the same walking speed. However, based on the data collected, it is seen that the inter-subject variation for individuals in the same conditions is considerable. This phenomenon can be visualized in Figure 16. In this sense, as future work, a user-oriented regression method should be developed, in order to consider more characteristics of the subjects (e.g. body mass), constructing different joint trajectories for subjects in the same conditions. Figure 16: Subject 6 and 10 with a body height of 1.80 m, walking with a speed of 4 km/h. 4.4. General Conclusions In this chapter, a regression model was implemented to generate reference ankle joint trajectories in the sagittal plane for subjects with body heights from 1.51 m to 1.83 m and walking speeds from 1 to 4 km/h, according to the walking speed and body height of each subject. The results were satisfactory and similar to the results achieved in [82]. However, improvements are needed. The reasons that justify improvements can be related with two facts: the data collection and the regression model. If the data collection had been performed in a treadmill, certainly, the stability of the subjects walking at slow speeds would not have been compromised. Therefore, the ankle joint data derived from the walking motion would not have been so irregular. Furthermore, using the proposed regression model, subjects in the same conditions of speed and body height will acquire the same walking trajectory. However, it was seen that 49 the inter-subject variation is significant and, for this reason, it is recommended to develop a model oriented to the user, considering more data, such as body mass. 50 CHAPTER 5 – ANKLE KINETICS TRAJECTORIES GENERATION This chapter focuses on modeling of reference ankle joint torques based on five inputs, namely the reference ankle joint angles determined in the Chapter 4 and their derivatives (angular velocity and acceleration), body height of the subject and walking speed. The chapter starts with a brief explanation of the dependency of the joint torque on the body mass and then, machine learning methods used to create the reference joint torques are presented. At last, the results of each model are exhibited, compared and discussed, ending the chapter with the best approach to model the reference ankle joint torque. In this context, with the current and the previous chapter, the creation of reference ankle joint torques based on the walking speed, body height and body mass of each subject can be achieved. 5.1. Introduction In the Chapter 4, based on [78], [82] and according to the achieved results, it was seen that the ankle joint kinematics during the gait cycle depends on the walking speed and the body height. Besides that, during the walking motion, the ankle plantar flexors muscles are responsible for providing the support required to boost the body forward [86]. In this sense, with an increase of the body mass, it is expectable that the support given by the ankle joint should be higher, increasing the muscle function and the joint torque and power. Some studies already verified that the ankle joint kinetics depends not only on the walking speed, but also on the body mass [87]–[89]. Thus, in this dissertation, all the joint torques were normalized by the body mass. 51 In the last years, studies have revealed that machine learning algorithms have capacity to model nonlinear relationships of data from the walking motion. In this field, most of these machine learning algorithms are selected to classify and to recognize locomotion modes [90]–[92]. There is no evidence of the use of machine learning were found to predict joint kinetics oriented to the subject during gait cycle. In this connection, this chapter focuses on the use of different machine learning algorithms to model the ankle joint kinetics. To explore the relation between variables, two approaches of machine learning can be used: supervised and unsupervised learning techniques. A supervised learning technique develops a prediction model, using a training dataset with a relation between the input and the output data. Then, this developed model can predict the output for a new dataset and its generalization capacity with high predictive accuracy is dependent of the variance of the training dataset. On the other hand, in unsupervised learning techniques, the learning process is based on learning patterns through grouping instances. While supervised learning techniques comprehend regression and classification algorithms, unsupervised learning techniques correspond to dimensionality reduction or clustering [93]. Thus, supervised learning techniques are used in this chapter to predict the reference ankle joint torque. 5.2. Methods In this section, different regression models are implemented to achieve the most proximal ankle joint torque for each subject. Based on a literature search, the most investigated machine learning approaches used in regression problems include Support Vector Machine (SVR), Random Forest (RF) and Artificial Neural Networks (ANN). In the field of ANN, MLP neural networks and Deep Learning architectures, including Long-Short Term Memory (LSTM) and Convolutional Neural Networks (CNN), have been widely used to solve regression problems [65], [66], [94]–[96]. In this sense, these regression models (SVR, RF, MLP, LSTM and CNN) were implemented, trained and optimized in MATLAB ® and the best method was chosen to predict the reference ankle joint torque. 5.2.1. Regression Models A. Support Vector Regression As in Support Vector Machine, developed by [97], SVR is widely used in regression problems due to its greater capacity of generalization [93], [98]. This technic is characterized to map a lowerdimensional data into a high-dimensional feature space using kernel methods to achieve higher accuracies. The most commonly used kernels of this method are linear, polynomial and gaussian or radial 52 basis function. However, there is no evidence about the best kernel to use. In SVR, the train is performed using a symmetrical loss function. During this process, a flexible tube (ԑ-tube) is formed around the estimated function and the main objective of this method is to find the tube with the minimal width (ԑ - epsilon) that approximates the continuous-valued function, penalizing points outside the tube and providing no penalization to the points inside. At the same time, the model complexity and the prediction of the error are balanced. In addition to the great generalization capacity with high accuracy, the dimensionality of the input data does not affect the complexity of the model and, since it is less sensitive to the noise of the inputs, the model is more robust [93]. B. Random Forest Before providing a brief explanation about RF applied in regression problems, it is required to know some operation concepts behind Decision Trees (DT). When using this method (DT), a single prediction is made as a result of questions that are produced based on a splitting method. The decision starts at the root node, on the top of the tree, and it progresses through the tree considering the answers to the questions at each decision node. This process runs until reaching the best prediction, that is found at the terminal node, also known as leaf node, as presented by Figure 17. Figure 17. Example of an architecture of a DT. During the training process, the DT learns the best questions/decisions to ask/perform, as well as the order to ask, with the final purpose of achieving the most accurate estimation. The decision criteria related to the splitting method is normally based on the mean squared error and, thus, according to this value, during the training process, the model decides if it is required to split a node [99]. However, the 53 major problem related with DT is overfitting, because when training, the model tends to adapt itself too much to the training data. Another negative aspect of this method is the instability, producing a model with high variance. In these cases, small variations in the data imply a generation of a different DT. These issues can be overpassed using RF. This regression method aggregates many trees that are trained with different random parts of the same training dataset, in order to reduce the high variance, using a technique called bagging. This technique enables better predictions, since the different trees are not correlated. In this sense, this means that the average performance of many trees is less sensitive to noise, while the prediction of a single tree is considerably sensitive [99]. C. Artificial Neural Networks: Multilayer Perceptron ANN are composed by three layers: input, hidden and the output layer, containing independent variables, activation functions and dependent variables, respectively. The input layer is connected to the hidden layer that is composed by hidden neurons. These neurons receive the information from the input layer, they calculate the weights of the variables using activation functions (such as, hyperbolic tangent, sinusoid, logistic, binary step or identity functions) and predict the final output [100]. If the output of one layer serves as input in the next layer and all nodes are fully connected, the network is called feedforward neural network and the information never fed back in the neural network because there are no loops in its architecture [101]. Figure 18 represents an example of an architecture of a feedforward neural network with two hidden layers. Figure 18. Feedforward neural network architecture with two hidden layers. Based on these characteristics, and according to [102]–[104], MLP is the most used ANN in the majority of the fields due to its simplicity. Thus, a MLP neural network was applied in this dissertation. The training process of MLPs has been widely used along with backpropagated algorithms, since the convergence of the neural network is improved [105]. The principle behind MLPs with backpropagation algorithms is based on the error gradient computation with respect to the weights and 54 it comprehends two phases: (i) to process the data from the input layer to the output layer, passing through the hidden layer(s); (ii) having the outputs, a comparison between the estimated and the real signal is done, computing an error. This error is backpropagated through the neural network based on the gradient descendent techniques. Thus, the weights are updated according with the error between the difference of the predicted output and the real signal. This can enable the achievement of predictive values closer to the target [101], [106]. Mathematically, the error/cost function in backpropagation algorithms can be calculated as the sum of squares, using Equation (4). Where C represents the cost function between the target ( T ) and the predicted output ( O ) for each output neuron, N . As reported by [107], for a network with only one layer with one neuron and a rectified linear unit activation, the prediction/target of the model corresponds to the activation of the neuron. In this case, the activation function is described by Equation (5). Where, w and b represent the parameters of the model (weight and bias, respectively) and X represents the model inputs. As a result, the cost function can be reformulated by Equation 6 and 7: Differentiating Equation (7) with respect to the weights, it is possible to update the current weights, at instant t , considering the error gradient descent and the weights of the previous instant t – 1 , obtaining Equation (8). The derivation steps to achieve Equation 8 can be consulted in [107]. Notwithstanding the prediction can achieve better performances, the gradient descent algorithms used to backpropagate the error can stuck in a local minimum, as presented in Figure 19. 𝐶(𝑇,𝑂)= 1 𝑁෍(𝑇𝑘−𝑂𝑘) 2 𝑁 𝑘=1 (4) 𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑖𝑜𝑛(𝑋)=𝑚𝑎𝑥 (0,𝑤∙𝑋+𝑏) (5) 𝐶(𝑇,𝑂,𝑤,𝑏)= 1 𝑁෍(𝑇𝑘−𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑖𝑜𝑛(𝑋𝑘)) 2 𝑁 𝑘=1 (6) C(T,O,w,b)= 1 N෍(Tk−max (0,w ∙ Xk+b))2 N k=1 (7) 𝑤𝑡=𝑤𝑡−1 − 𝛿𝐶 𝛿𝑤𝑡−1 (8) 55 Figure 19. Gradient descent algorithm getting stuck in a local minimum. To solve this problem, two new parameters can be considered, namely learning rate , 𝜼 and momentum , 𝜶 , as represented by Equation (9). With these parameters, the effect of the error gradient on the weights update is controlled. Moreover, the performance of the neural network and its time of convergence are also dependent on these parameters. Regarding the learning rate, it is required to consider that if this parameter is set too small, the algorithm will take too much time to converge, because the gradient descent algorithm will work slowly. In these cases, the convergence of the model is compromised and it is possible to stop again in a local minimum. On the other hand, if the learning rate is set too high, the algorithm will converge rapidly but with no stability. Thus, low values of learning rate are preferable, also considering the momentum parameter to escape from local minimums, since this parameter adds a percentage to the last update, improving the current update [108]. However, the error on the training set can be small, but when the neural network predicts responses to unknown data, the error is large. This means that the neural network is not generalized and it cannot provide good responses to new situations. According with [109], in MLP, the generalization performance is better using Bayesian regularization. Another regularization technique that provides a generalized neural network is the Levenberg-Marquardt [100]. In this sense, using the neural network toolbox of MATLAB® , in this dissertation, a MLP with backpropagation algorithm is applied, investigating two different training algorithms: trainbr and trainlm , corresponding to the Bayesian regularization method and Levenberg-Marquardt method, respectively. In both cases, the learning rate and momentum parameters were used as default. Moreover, based on wt= wt−1− η δC δwt−1 −αwt−1 (9) 56 [109], the performance of the neural network can increase, obtaining a stable learning if the learning rate is updated during the training process. In this sense, another training algorithm, namely, traingdx was implemented to update the learning rate parameter. D. Artificial Neural Networks: Deep Learning Ankle joint torque is a continuous signal that is dependent on the time and, thus, it can be recognized as a Time-Series signal. In this field, studies reported that the use of Deep Learning methods, such as LSTM and CNN, can provide satisfactory results [110], [111]. LSTM consists of a type of recurrent neural networks. As described in the previous sub-section, the behavior of the activation functions existing on the neurons of the hidden layers of a feedforward neural networks depends on the behavior of the activation functions of neurons of the previous hidden layers. In contrast, with the architecture of recurrent neural networks, the behavior of the activation functions of the hidden neurons depends not only on the behavior of the previous activation functions, but also on the behavior at an earlier time. For this reason, neural networks where the behavior of the activation functions is dependent of the time are called by recurrent neural networks and, thus, Time-Series data is learned in an efficient way [101]. Normally, these architectures are used to predict the future using past information. However, if the interval in time between the current prediction and the previous information is too large, the predictions of the recurrent neural network are not promising. According to [101], if the time to run of a recurrent neural network is too long, the gradient will become tremendously unstable and the capacity to learn will be minimal. To solve this problem, LSTM neural networks were proposed. These kind of neural networks are a conjugation of recurrent neural networks with gradient descendent learning algorithms, making easier to obtain good results with unstable gradient problems under control [112]. Information relative to the train of the LSTM neural network can be found in [112]. CNN are another type of neural networks that is trained based on backpropagation algorithms. The architecture of these neural networks was inspired in the mammalian visual cortex, since according to [113], the first processing stages in the visual cortex consist in the detection of simple features, such as edges and bars. In the following processing stages, more complex associations are done and the object that is being seen is recognized. The behavior of CNN is identical, since these neural networks are composed by three layers (convolutional, pooling and fully-connected layers) and during the training process, the objective is to reduce the dimensionality of the representation based on feature extraction methods, in order to capture the most relevant spatial and temporal dependencies of an image [113], [114]. Thus, this neural network was used to extract the most relevant features to achieve the most proximal ankle joint torque. 57 The convergence and generalization performance of neural networks can be a difficult task to accomplish due to the difficulty into find the best learning rate and momentum value. Moreover, if only one learning rate and momentum value is applied to all weights, the weight update is the same to all neural network, causing the convergence performance less efficient. To improve the convergence performance, some gradient descendent algorithms have been emerged, such as, the Adaptive Moment Estimation (ADAM). With this algorithm, the weights of the neural network are updated using adaptive learning rates, avoiding the existence of a global one. Thus, the convergence performance is increased, while the learning error is reduced, obtaining a versatile neural network [115]. In Deep Learning algorithms explored in this dissertation, ADAM was used. In the neural network toolbox of MATLAB® is not possible to apply this algorithm when using MLP neural network and thus, the most proximal algorithm found was traingdx . 5.2.2. Data Preparation The data collected in the Chapter 4 was used to train the implemented machine learning models. To train all the models, the data were randomly divided int blocks, where 60% of the data were used for training, 20% for validation and 20% for testing, corresponding to 10, 3 and 3 subjects, respectively. The inputs of the models were the ankle joint angles, angular velocity, angular acceleration, body height and walking speed. The output of the models was the ankle joint torque normalized by the body mass. A k - fold cross-validation algorithm was implemented to analyze the robustness of the models. The number of folds was set to 4. Table 13 presents the maximum range and the units revealed by the input and output features. In accordance to these values, it is possible to infer that there is a discrepancy in the magnitude order of each feature. Table 13. Units and Maximum Variation Range of each model feature Feature Unit Maximum Range Ankle Angle degrees 76.0 Ankle Angular Velocity degrees/s 3.75×102 Ankle Angular Acceleration degrees/s2 3.68×103 Body Height m 0.320 Walking Speed km/h 3.00 Ankle Torque N·m/kg 2.18 64 Table 21. Key parameters for LSTM neural network Parameter Value Maximum number of epochs 10000 Maximum validation failures 10 Gradient Descendent Algorithm ADAM Initial Learning Rate 0.01 Learning Rate Drop Period 50 Learning Rate Drop Factor 0.2 Batch Size 64 The results of the LSTM neural network with the parameters defined in Table 21 are presented in Table 22. Table 22. LSTM results with default parameters, where the worst and best results are colored in red and green, respectively Batch Size Hidden Neurons GOF (%) NRMSE (%) Validation mean (std) Test Validation mean (std) Test 64 10 78.1 (0.400) 77.7 4.76 (0.121) 4.70 70 77.2 (2.71) 78.4 4.96 (0.634) 4.55 110 79.4 (0.195) 79.6 4.92 (0.117) 4.31 Considering the regression performance achieved with LSTM, an increment of the number of neurons does not offer significative improvements. The best results were achieved for a LSTM neural network with 110 hidden neurons. The effect of the batch size was studied in this neural network, increasing and the decreasing its value, in order to find the architecture that provided the best results. Due to the high time consumption encountered during the initial developments of this neural network, the batch size was only varied for the LSTM with 110 hidden neurons, since that was the architecture that provided the best performance in default parameters. The results achieved are presented in Table 23, where the worst and the best performances are colored in red and green, respectively. 65 Table 23. LSTM results with batch size variation Batch Size Hidden Neurons GOF (%) NRMSE (%) Validation mean (std) Test Validation mean (std) Test 128 110 79.3 (0.200) 79.5 4.95 (0.125) 4.57 64 110 79.4 (0.195) 79.6 4.92 (0.117) 4.31 32 110 77.4 (0.352) 76.8 4.92 (0.155) 4.87 According with Table 23, with a decreasing of the batch size, the performance of the neural network started to decrease. On the other side, considering a larger batch size, the performance of the neural network remains similar. Thus, an LSTM with 110 hidden neurons and trained with a batch size of 64 can provide good performances. 5.3.5. Convolutional Neural Network Generally, CNN are used for image processing and thus, this neural network operates with 3 – dimensional matrices. In this sense, the input data were treated as images, having suffered a transformation. Once there are five input features and several gait cycles, the data was organized in 3 – dimensional matrices, as follows: width – number of each gait cycle samples; height – number of input features; depth – number of gait cycles. Concerning the training process, this neural network also trained while the number of maximum epochs or the number of maximum validation failures were not reached. At the beginning, the initial learning rate value, the drop period, drop factor and the batch size were set according with the default values. The defined parameters are presented in Table 24. 66 Table 24. Key parameters for CNN Parameter Value Maximum number of epochs 10000 Maximum validation failures 10 Gradient Descendent Algorithm ADAM Initial Learning Rate 0.01 Learning Rate Drop Period 50 Learning Rate Drop Factor 0.2 Batch Size 64 Number of Convolutional Layers 3 (with 8, 16 and 32 filters applied) Kernel Size 5×5 Pooling (Pool Size) Average (2) Stride 2 Fully Connected Layers (Number of outputs) 1 (1) According to [118], with smaller kernel sizes, the computational performance is more efficient than with larger kernel sizes and the neural network is able to learn more complex non-linear features. In this sense a kernel size of 3 was also experienced and the results are presented in Table 25. Table 25. CNN results with kernel size variation Kernel Size GOF (%) NRMSE (%) Validation mean (std) Test Validation mean (std) Test 5×5 81.1 (0.721) 80.5 4.10 (0.122) 4.10 3×3 81.0 (1.08) 80.7 4.13 (0.188) 4.06 Based on the achieved results, the performance of the neural network with both kernel sizes was identical, achieving smaller better results in the test dataset with a kernel size of 3. Since the last layer of CNN is a fully connected layer identical to a MLP and considering that an increment in the number of hidden neurons in MLP caused an improvement in the regression performance, another fully connected layer was added to CNN. Since at the end of the last convolutional layer, a vector with a length of 32×3 is created and considering the desired output torque value, the 67 number of neurons of the fully connected layer added was set to 32. The results achieved with this new architecture are presented in Table 26. Table 26. CNN results with two fully connected layers Kernel Size GOF (%) NRMSE (%) Validation mean (std) Test Validation mean (std) Test 3×3 80.6 (0.596) 80.0 4.20 (0.0814) 4.22 In contrast to the expectable, the performance of CNN decreases with the addition of another fully connected layer. Thus, only one fully connected layer should be considered. In this neural network, the batch size was also varied. The performances achieved are exhibited in Table 27, where the worst and best performances are colored in red and green, respectively. Table 27. CNN results with batch size variation Kernel Size Batch Size GOF (%) NRMSE (%) Validation mean (std) Test Validation mean (std) Test 3×3 128 81.6 (0.533) 81.2 3.99 (0.137) 3.97 64 81.0 (1.08) 80.7 4.13 (0.188) 4.06 32 79.9 (0.548) 79.7 4.36 (0.137) 4.27 By analyzing Table 27, a reduction of the batch size is not favorable, since the regression performances started to decrease. On the other side, with an increment of the batch size, better results were achieved. Thus, a CNN with 3 convolutional and 1 fully connected layer, a kernel size of 3×3 and a batch size of 128 was found as the architecture that provides better results. 5.4. Discussion and General Conclusions Machine learning algorithms have been used in the last years to model nonlinear relationships of the walking. However, to the best of the author’s knowledge, there are no literature evidences on the using of machine learning to predict the ankle joint torque oriented to the subject. In this sense, the most 68 used regression models applied to model nonlinear relationships, namely SVR, RF, MLP, LSTM and CNN, were used to predict the reference ankle joint torque based on five inputs: ankle angle, angular velocity, angular acceleration, body height and walking speed. Table 28 presents the best results obtained per each machine learning algorithm. Table 28. Best results of the implemented regression models where the best model performances achieved are colored in green, while the worst are colored in red Method GOF (%) NRMSE (%) Validation mean (std) Test Validation mean (std) Test SVR 69.4 (9.52) 68.7 5.97 (2.01) 6.59 RF 73.9 (1.95) 73.6 5.68 (0.470) 5.56 MLP 67.3 (1.57) 70.9 7.10 (0.380) 6.13 LSTM 79.4 (0.195) 79.6 4.92 (0.117) 4.31 CNN 81.6 (0.533) 81.2 3.99 (0.137) 3.97 In addition to the best results summarized and presented by Table 28, Figure 20 exhibits the ground truth ankle joint torque and the predictions achieved with each one of the models for a random selected range of the test dataset. Figure 20. Predictions of the SVR, RF, MLP, LSTM and CNN machine learning models in comparison with the real ankle joint torque. 69 In accordance to Figure 20, it is possible to infer that the predictions made by LSTM and CNN models produced an ankle joint torque closer to the expected one, when compared to the remaining regression models. This fact is confirmed by the higher GOF and lower NRMSE value presented by these models, in Table 28. Nonetheless, performing a deeper analysis centered on the gait cycle, during the stance phase, it was verified that all the models present capacity to model the ankle joint torque. However, in the swing phase, irregularities and sudden peaks were verified for the predictions achieved by the SVR and MLP. This phenomenon can be confirmed by the lowest evaluation metrics presented by these two models. Thus, considering the obtained results, the CNN was the model chosen to predict the reference ankle joint torque oriented to the subject. 70 CHAPTER 6 – AAN EMG-BASED CONTROL STRATEGY This chapter describes the proposed AAN EMG-based control strategy presented on Chapter 2. The importance of the regression models developed in Chapter 4 and Chapter 5 is investigated. Additionally, it is developed a method to estimate the real ankle joint torques based on EMG signals and joint angles. The performance of the ankle joint torque estimation is evaluated and the chapter ends with a discussion of the achieved results. 6.1. Introduction The literature analysis in Chapter 2 demonstrated that the EMG-based control strategies integrated into WPADs only follow the intentions of the user and, consequently, it may be possible that individuals with impairments of the lower limbs and whose EMG signals are weak do not receive the required support to walk. AAN control strategies integrating EMG signals represent a recent alternative to overcome these limitations, taking advantage of the anticipatory performance of these signals. Thus, this dissertation aims to construct an AAN EMG-based control strategy for a future integration at SmartOs. To advance with the proposed strategy, it is required to determine two parameters: a reference and a real joint torque, as represented in Figure 4. Regarding the reference ankle joint angle determined in Chapter 4, its impact is determinant on the prediction of the reference ankle joint torque. According to [77], the joint torque about the ankle joint is dependent on the angular acceleration. Since this parameter corresponds to the second derivative of the joint angle, the prediction of the reference ankle 71 joint torque is dependent on the reference ankle joint angle prediction. In this sense, through this chapter, the connection between the Chapter 4 and Chapter 5 is evaluated: i.e., the prediction of the reference ankle joint torques with the best model found in Chapter 5 fed with the predicted ankle joint kinematics determined at Chapter 4. Considering the collected works responsible for converting the EMG signals into joint torque values, the study developed by [64] was chosen to perform this conversion due to present the shortest conversion time, offering promising results. Furthermore, the toolbox to perform this estimation is open source, programmed in C++ language and it is available on [119]. Thus, in this chapter, the code of the toolbox is adapted to the architecture of SmartOs, in order to estimate the real ankle joint torque in real-time. 6.2. Reference Ankle Joint Torque Prediction The prediction of the reference ankle joint torque ( τref(t) ) was based on the prediction of kinematic data for each subject. Five inputs were involved, namely, BH, WS, 𝜃 ref(t), ѡref(t) and αref(t) , as presented in Figure 2. To evaluate the performance of the reference ankle joint torque prediction, two cases are analyzed: (i) the worst; and (ii) the best ankle joint angle prediction. Based on the results achieved in Chapter 4, the worst prediction was verified for a subject with a body height of 1.79 m, walking at 1 km/h, while the best results were achieved for a subject with a body height of 1.65 m, walking at 4 km/h. For each case, the body mass was 81.9 and 60.0 kg, respectively. Thus, for both cases it was predicted the reference ankle joint torque using the best regression model found in Chapter 5, namely, CNN with a kernel size of 3×3, 3 convolutional and 1 fully connected layers, trained with a batch size of 128. The results of the predictions are represented in Figures 21. 72 Figure 21. Reference ankle joint torque prediction based on the worst (a)) and best ((b)) reference ankle joint angle prediction. Although the ankle joint angle prediction for the first case ((i)) has been less favorable, the prediction of the torque for this joint is acceptable and comparable with the real joint torque, presenting a GOF of 74.7 % and a NRMSE of 8.02 %. Based on Figure 15 – a), (that represents the worst ankle joint angle prediction), the ankle joint angle predicted presents a wider range of motion (ROM) when compared with the real ROM. Hence, the angular acceleration also presents a wider range and, since the joint torque is directly dependent on this parameter, it would be expectable that the predicted joint torque could present higher values than the real one. In fact, this phenomenon occurred, as it is proven by Figure 21 – a). Concerning the second case ((ii)), the prediction of the joint torque is quite similar to the real joint torque, presenting a GOF of 89.8% and a NRMSE of 3.16%. 73 Overall, even the predicted reference ankle joint kinematics present less favorable fits, the implemented machine learning algorithm (CNN) can model the reference ankle joint torques with high accuracy. 6.3. EMG-based Real Joint Torque Estimation Once the prediction of the reference ankle joint torque has been achieved, it was required to estimate the real ankle joint torque of a subject, in real-time. Along this section, all the algorithms implemented are exhibited, ending the chapter with the validation of the proposed estimation. 6.3.1. Model Presentation Figure 22 presents the block diagram to estimate the real ankle joint torque, based on real EMG signals and real ankle joint angles. Figure 22. Block diagram to estimate the real ankle joint torque, τreal . The toolbox available on [119] performs the conversions illustrated in the Blocks 1, 3, 4 and 5 of Figure 23. However, it does not calculate the musculotendon lengths (LMTs), neither the moment arms (MAs) illustrated in Block 2, both required to estimate the real ankle joint torque. To solve this problem, an algorithm presented by [120] and adapted by [121] was used to obtain the LMTs and MAs based on real ankle joint angles. To determine these two parameters, a calibration step is required, in order to identify six calibration parameters, namely, Fmax , vmax , L0 m , optimal length of the tendon ( Lslack ), ɸ and type I fibers percentage. In the work advanced by [121], these six parameters (presented in Table 29) were found for a healthy subject with 1.80 m and 80 kg. 80 performed to identify the EMG-based and AAN EMG-based control strategies already developed, as well as the most relevant methodologies to estimate the joint kinetics based on EMG signals, since this conversion is performed in the majority of the control strategies using EMG signals. Proportional myoelectric, musculoskeletal and empirical models were investigated as methods to convert the EMG signals into torque values. Empirical models may be more appropriated to perform the desired conversion for disable persons, since calibration steps are not required. An EMG system was projected, implemented and validated with a healthy subject walking at 1 km/h. The results achieved when compared with a commercial system and with literature findings, revealed a good performance. In this dissertation, a regression model was explored to predict the reference ankle joint kinematics trajectory for a specific subject in the sagittal plane, based on the walking speed and body height. The results achieved were satisfactory and similar to the results obtained in [82]. However, improvements related to (i) the data collection; and (ii) the regression model must be done. Regarding the first point, during the data collection, it was verified that at slow walking speeds, the postural stability of most of the subjects was compromised. During the regression model training, in the presence of slow walking speeds, the regression model learned irregular ankle joint trajectories. Regarding the second point, it was verified that the proposed regression model attributes the same reference ankle joint angle for subjects with the same body height and walking at the same walking speed. However, the inter-subject variation is considerable and, for this reason, it is recommended to develop a model oriented to the user, considering more data, such as body mass. Based on ankle joint kinematics, body height of the subject and walking speed, machine learningbased methods were explored to generate the reference ankle joint kinetics. The performance of five machine learning models was explored, namely: SVR, RF, MLP, LSTM and CNN, where the best results were achieved for a CNN. The results demonstrated that CNN can accurately predict the reference ankle torque trajectories, even considering reference ankle joint kinematics with less favorable predictions. The correct prediction of the reference ankle joint torque oriented to the user based on predicted reference ankle joint kinematics, body height and walking speed is utmost importance in the AAN EMG-based control strategy, in order to compare the reference ankle joint torque with the real one. In this field, considering the best acknowledge of the author, there is no evidence on the literature regarding the ankle joint torque prediction oriented to the subject. 81 To estimate the real ankle joint torque, an EMG-based torque estimation was performed, based on a musculoskeletal model presented in [64]. Only two muscles were used to perform the estimation, in contrast to the six muscles used in [64]. Thus, EMG data from TA and GASL were used with real ankle joint angles to estimate the real ankle joint torque. Since the model depends on calibration parameters that were found for a subject with 1.80 m and 80 kg, the validation of this methodology was performed only considering one subject with the referred physical characteristics. Based on the obtained results, it was verified that the magnitude of the torque provided by the model was inferior to the real magnitude, indicating that only two muscles are not enough to estimate the ankle joint torque. Thus, it is necessary to include more EMG contributions from GASM, SOL, PT and PL muscles. Aiming the future integration of the proposed AAN EMG-based control strategy into SmartOs and considering that the EMG-based torque estimation was developed as a standalone framework, a TCP/IP communication was created to simulate the protocol between the EMG-based torque estimation model and the CCU of SmartOs. Furthermore, with the developed master thesis, the goals that were established in Chapter 1 were all achieved and the RQ can be answered: • RQ1: Which are the contributions and the main differences of the EMG-based control and the AAN EMG-based control strategies? This RQ was addressed in Chapter 2. Both strategies contributed to muscular activity improvement and endurance, avoiding muscle atrophy. Most of the EMG-based control strategies verified a decreased muscular activity while the joint angles were controlled. However, these reviewed strategies were only tested in healthy subjects. Thus, the joint angles were controlled given the motor ability of the healthy subjects. Nonetheless, it is not expectable that these strategies have the potential to assist the lower limbs of disabled subjects. In contrast, the AAN EMG-based control strategies compare, in real-time, the user’s motor performance with the desired user’s motion to ensure a sufficient level of assistance oriented to the user’s needs. Thus, AAN EMG-based control strategies have the potential to assist and rehabilitate the lower limbs, such that the user can achieve autonomy to perform their daily life activities. • RQ2: Is it possible to obtain joint torque measures only using EMG signals? This RQ was addressed in Chapter 2. Based on the reviewed information, it was concluded that most of the control methods which use EMG signals to control WPADs convert these EMG signals into torque values. Nonetheless, there are literature evidences that only EMG 82 readings are not enough to obtain the accurate values of joint torques, being necessary to fuse EMG data with biomechanical data, such as, joint angles, velocities, accelerations. • RQ3: Is it possible to predict reference walking kinematics and kinetics trajectories relying exclusively on the walking speed and anthropometric data? This RQ was approached in Chapters 4, 5 and 6. Based on the results achieved in Chapter 4, it is possible to predict reference walking kinematics using exclusively walking speed and anthropometric data. The results of this dissertation show the needed for adding more anthropometric data (such as, the body mass) to avoid the attribution of the same ankle joint angle to subjects with the same body height, walking at the same walking speed. However, based on the results achieved in Chapter 5 and Chapter 6, this is not a limitation to predict the reference walking kinetics, since good performances were obtained. • RQ4: Can EMG-based torque estimation strategy present a good performance? This RQ was approached in Chapter 6. Based on the results achieved in the literature [64], the EMG-based torque estimation with joint angles can be performed in real-time with good performances, when six muscles are considered. In this dissertation, only EMG data from two muscles (TA and GASL) and ankle joint angles were considered. Based on the achieved results, the magnitude of the estimated ankle joint torque was inferior to the expectable. With the addition of one more muscle (GASM), higher magnitudes were achieved. Thus, more muscles must be considered to estimate the real ankle joint torques, considering EMG signals and real ankle joint angles. 7.1. Future Work Future work in the scope of this dissertation include: (1) the improvement of the regression model to predict reference ankle joint angles, in order to avoid the attribution of the same trajectory for subjects with the same body height and walking speed; (2) the improvement of the quality of the data acquisition by performing walking test in a treadmill with force platforms to avoid loss of the balance, specially at slow walking speeds; (3) the inclusion of more muscles to estimate the real ankle joint torque based on EMG signals and joint angles; (4) the exploration of machine learning algorithms operating in real-time to estimate the real ankle joint torque based on EMG signals and joint angles, since these empirical methods may 83 be more appropriated to perform the conversion in motor impaired persons, because calibration steps are not required. 84 REFERENCES [1] W. Johnson, O. Onuma, M. Owolabi, and S. Sachdev, “Bulletin of World Health Organization,” Stroke: a global response is needed , 2016. [Online]. Available: https://www.who.int/bulletin/volumes/94/9/16-181636/en/. [Accessed: 05-Jun-2019]. [2] F. A. Loterio, “Análise do Padrão de Ativação Muscular de Indivíduos Hemiparéticos Pós-AVC em Marcha Assistida por Andador Robótico,” Universidade Federal do Espírito Santo, 2015. [3] S. H. Jang, “The recovery of walking in stroke patients: a review,” Int. J. Rehabil. Res. , vol. 33, no. 4, pp. 285–289, Dec. 2010. [4] D. G. da Saúde, “Programa Nacional de Prevenção e Controlo das Doenças Cardiovasculares,” Dgs , p. 28, 2006. [5] A. H. NeuroHealth, “A Parent’s Quick Guide to Hemiplegia and Hemiparesis,” 2017. [Online]. Available: https://neurohealthah.com/guide-to-hemiplegia-hemiparesis/. [Accessed: 05-Jun2019]. [6] J. Cao, S. Q. Xie, R. Das, and G. L. Zhu, “Control strategies for effective robot assisted gait rehabilitation: The state of art and future prospects,” Med. Eng. Phys. , vol. 36, no. 12, pp. 1555– 1566, Dec. 2014. [7] G. Colombo, M. Joerg, R. Schreier, and V. Dietz, “Treadmill training of paraplegic patients using a robotic orthosis.,” J. Rehabil. Res. Dev. , vol. 37, no. 6, pp. 693–700, 2000. [8] A. Kaelin-Lang, L. Sawaki, and L. G. Cohen, “Role of Voluntary Drive in Encoding an Elementary Motor Memory,” J. Neurophysiol. , vol. 93, no. 2, pp. 1099–1103, 2004. 85 [9] M. Lotze, “Motor learning elicited by voluntary drive,” Brain , vol. 126, no. 4, pp. 866–872, Apr. 2003. [10] L. Marchal-Crespo and D. Reinkensmeyer, “Review of control strategies for robotic movement training after neurologic injury,” J. Neuroeng. Rehabil. , 2009. [11] K. Gui, H. Liu, and D. Zhang, “A Practical and Adaptive Method to Achieve EMG-Based Torque Estimation for a Robotic Exoskeleton,” IEEE/ASME Trans. Mechatronics , vol. 24, no. 2, pp. 483– 494, Apr. 2019. [12] C. Ma, N. Chen, Y. Mao, D. Huang, R. Song, and L. Li, “Alterations of Muscle Activation Pattern in Stroke Survivors during Obstacle Crossing,” Front. Neurol. , vol. 8, no. MAR, Mar. 2017. [13] P. N. Fernandes et al. , “EMG-based Motion Intention Recognition for Controlling a Powered Knee Orthosis,” in 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) , 2019, pp. 1–6. [14] P. Artemiadis, “EMG-based Robot Control Interfaces: Past, Present and Future,” Adv. Robot. Autom. , vol. 1, no. 2, pp. 1–3, 2012. [15] P. R. Cavanagh and P. V. Komi, “Electromechanical delay in human skeletal muscle under concentric and eccentric contractions,” Eur. J. Appl. Physiol. Occup. Physiol. , vol. 42, no. 3, pp. 159–163, Nov. 1979. [16] S. Olney and C. Richards, “Hemiparetic gait following stroke. Part I: Characteristics,” Gait Posture , vol. 4, pp. 136–148, 1996. [17] C. Fleischer and G. Hommel, “A Human-Exoskeleton Interface Utilizing Electromyography,” IEEE Trans. Robot. , vol. 24, no. 4, pp. 872–882, Aug. 2008. [18] S. Jezernik, G. Colombo, and M. Morari, “Automatic Gait-Pattern Adaptation Algorithms for Rehabilitation With a 4-DOF Robotic Orthosis,” IEEE Trans. Robot. Autom. , vol. 20, no. 3, pp. 574–582, Jun. 2004. [19] R. M. Singh, S. Chatterji, and A. Kumar, “Trends and challenges in EMG based control scheme of exoskeleton robots—a review,” Int. J. Sci. Eng. Res , vol. 3, no. 8, pp. 933–940, 2012. [20] L. L. M. e Silva, C. E. M. de Moura, and J. R. P. de Godoy, “A marcha no paciente hemiparético,” Univ. Ciências da Saúde , vol. 3, no. 2, pp. 261–273, Apr. 2008. [21] B. Balaban and F. Tok, “Gait Disturbances in Patients With Stroke,” PM&R , vol. 6, no. 7, pp. 635– 642, Jul. 2014. [22] C. A. Johnson, J. H. Burridge, P. W. Strike, D. E. Wood, and I. D. Swain, “The effect of combined use of botulinum toxin type A and functional electric stimulation in the treatment of spastic drop 86 foot after stroke: a preliminary investigation,” Arch. Phys. Med. Rehabil. , vol. 85, no. 6, pp. 902– 909, Jun. 2004. [23] J. Perry, Gait Analysis: Normal and Pathological Function . Thorofare: SLACK Incorporated, 1992. [24] K. P. Parekh and J. B. Vyas, “Myoelectric Leg for Transfemoral Amputee,” Int. J. Sci. Res. Dev. , vol. 1, no. 5, pp. 12–15, 2013. [25] B. S Rajaratnam, “A Comparison of EMG Signals from Surface and Fine-Wire Electrodes During Shoulder Abduction,” Int. J. Phys. Med. Rehabil. , vol. 02, no. 04, 2014. [26] D. L. Waite, R. L. Brookham, and C. R. Dickerson, “On the suitability of using surface electrode placements to estimate muscle activity of the rotator cuff as recorded by intramuscular electrodes,” J. Electromyogr. Kinesiol. , vol. 20, no. 5, pp. 903–911, Oct. 2010. [27] M. Solomonow et al. , “Surface and wire EMG crosstalk in neighbouring muscles,” J. Electromyogr. Kinesiol. , vol. 4, no. 3, pp. 131–142, Jan. 1994. [28] R. Bogey, K. Cerny, and O. Mohammed, “Repeatability of Wire and Surface Electrodes in Gait,” Am. J. Phys. Med. Rehabil. , vol. 82, no. 5, pp. 338–344, May 2003. [29] R. A. Bogey, J. Perry, E. L. Bontrager, and J. K. Gronley, “Comparison of across-subject EMG profiles using surface and multiple indwelling wire electrodes during gait,” J. Electromyogr. Kinesiol. , vol. 10, no. 4, pp. 255–259, Aug. 2000. [30] C. J. De Luca, “Surface electromyography: Detection and recording,” DelSys Inc. , vol. 10, no. 2, pp. 1–10, 2002. [31] B. Gerdle, S. Karlsson, S. Day, and M. Djupsjöbacka, “Acquisition, Processing and Analysis of the Surface Electromyogram,” in Modern Techniques in Neuroscience Research , Berlin, Heidelberg: Springer Berlin Heidelberg, 1999, pp. 705–755. [32] D. A. Gabriel, “Effects of monopolar and bipolar electrode configurations on surface EMG spike analysis,” Med. Eng. Phys. , vol. 33, no. 9, pp. 1079–1085, Nov. 2011. [33] R. Merletti and P. Parker, Electromyography: Physiology, Engineering and Noninvasive Applications . New Jersey: John Wiley & Sons, Inc, 2004. [34] A. Suresh, “Body swarm interface ( BOSI ): Controlling Robotic Swarms Using Human BioSignals,” Boston University, 2016. [35] L. Shaw and S. Bagha, “Online EMG Signal Analysis for diagnosis of Neuromuscular diseases by using PCA and PNN,” Int. J. Eng. Sci. Technol. , vol. 4, pp. 4453–4459, 2012. [36] C. ADInstruments, “Trigno EMG Sensors.” [Online]. Available: https://www.adinstruments.com/products/trigno-emg-sensors. [Accessed: 02-Sep-2019]. 87 [37] Delsys Incorporated, Trigno TM Wireless Biofeedback System . 2018. [38] Noraxon, UltiumTM EMG System: Sensor and Receiver User Manual . . [39] I. Motion Lab Systems, Motion Lab Systems: User Guide . 2004. [40] BITalino: Biomedical Equipment, MuscleBAN BE Data Sheet . 2015. [41] BITalino: Biomedical Equipment, OpenSignals: User Manual . 2012. [42] D. P. Ferris and C. L. Lewis, “Robotic lower limb exoskeletons using proportional myoelectric control,” in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society , 2009, pp. 2119–2124. [43] H. Kawamoto and Y. Sankai, “Comfortable power assist control method for walking aid by HAL3,” in IEEE International Conference on Systems, Man and Cybernetics , 2003. [44] D. Ao, R. Song, and J. Gao, “Movement Performance of Human–Robot Cooperation Control Based on EMG-Driven Hill-Type and Proportional Models for an Ankle Power-Assist Exoskeleton Robot,” IEEE Trans. Neural Syst. Rehabil. Eng. , vol. 25, no. 8, pp. 1125–1134, Aug. 2017. [45] Rong Song, Kai-yu Tong, Xiaoling Hu, and Le Li, “Assistive Control System Using Continuous Myoelectric Signal in Robot-Aided Arm Training for Patients After Stroke,” IEEE Trans. Neural Syst. Rehabil. Eng. , vol. 16, no. 4, pp. 371–379, Aug. 2008. [46] A. V. Hill, “The heat of shortening and the dynamic constants of muscle,” Proc. R. Soc. London. Ser. B - Biol. Sci. , vol. 126, no. 843, pp. 136–195, Oct. 1938. [47] M. G. Hoy, F. E. Zajac, and M. E. Gordon, “A musculoskeletal model of the human lower extremity: The effect of muscle, tendon, and moment arm on the moment-angle relationship of musculotendon actuators at the hip, knee, and ankle,” J. Biomech. , vol. 23, no. 2, pp. 157–169, Jan. 1990. [48] D. G. Lloyd and T. F. Besier, “An EMG-driven musculoskeletal model to estimate muscle forces and knee joint moments in vivo,” J. Biomech. , vol. 36, no. 6, pp. 765–776, Jun. 2003. [49] M. Sartori, D. G. Lloyd, M. Reggiani, and E. Pagello, “A stiff tendon neuromusculoskeletal model of the knee,” in 2009 IEEE Workshop on Advanced Robotics and its Social Impacts , 2009, pp. 132–138. [50] M. Sartori, M. Reggiani, A. J. van den Bogert, and D. G. Lloyd, “Estimation of musculotendon kinematics in large musculoskeletal models using multidimensional B-splines,” J. Biomech. , vol. 45, no. 3, pp. 595–601, Feb. 2012. [51] N. Karavas, A. Ajoudani, N. Tsagarakis, J. Saglia, A. Bicchi, and D. Caldwell, “Tele-impedance based assistive control for a compliant knee exoskeleton,” Rob. Auton. Syst. , vol. 73, pp. 78–90, 88 Nov. 2015. [52] Suncheol Kwon, Yunjoo Kim, and Jung Kim, “Movement Stability Analysis of Surface Electromyography-Based Elbow Power Assistance,” IEEE Trans. Biomed. Eng. , vol. 61, no. 4, pp. 1134–1142, Apr. 2014. [53] H. He and K. Kiguchi, “A Study on EMG-Based Control of Exoskeleton Robots for Human Lowerlimb Motion Assist,” in 2007 6th International Special Topic Conference on Information Technology Applications in Biomedicine , 2007, pp. 292–295. [54] T. Yan, M. Cempini, C. M. Oddo, and N. Vitiello, “Review of assistive strategies in powered lowerlimb orthoses and exoskeletons,” Rob. Auton. Syst. , vol. 64, pp. 120–136, Feb. 2015. [55] K. Kiguchi and Y. Imada, “EMG-based control for lower-limb power-assist exoskeletons,” in 2009 IEEE Workshop on Robotic Intelligence in Informationally Structured Space , 2009, pp. 19–24. [56] W. Hassani, S. Mohammed, H. Rifaï, and Y. Amirat, “Powered orthosis for lower limb movements assistance and rehabilitation,” Control Eng. Pract. , vol. 26, pp. 245–253, May 2014. [57] C. Fleischer and G. Hommel, “Torque Control of an exoskeletal knee with EMG signals,” in Proceedings of the joint conference on robotics , 2006. [58] C. Fleischer and G. Hommel, “Calibration of an EMG-Based Body Model with six Muscles to control a Leg Exoskeleton,” in IEEE International Conference on Robotics and Automation , 2007, pp. 2514–2519. [59] W. Hassani, S. Mohammed, and Y. Amirat, “Real-Time EMG Driven Lower Limb Actuated Orthosis for Assistance As Needed Movement Strategy,” in Robotics: Science and Systems IX , 2013. [60] W. Hassani, S. Mohammed, H. Rifai, and Y. Amirat, “EMG based approach for wearer-centered control of a knee joint actuated orthosis,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2013, pp. 990–995. [61] M. Moltedo, T. Baček, T. Verstraten, C. Rodriguez-Guerrero, B. Vanderborght, and D. Lefeber, “Powered ankle-foot orthoses: the effects of the assistance on healthy and impaired users while walking,” J. Neuroeng. Rehabil. , vol. 15, no. 1, p. 86, Dec. 2018. [62] C. Fleischer, C. Reinicke, and G. Hommel, “Predicting the intended motion with EMG signals for an exoskeleton orthosis controller,” in 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2005, pp. 2029–2034. [63] L. Liu, M. Luken, S. Leonhardt, and B. J. E. Misgeld, “EMG-driven model-based knee torque estimation on a variable impedance actuator orthosis,” in IEEE International Conference on Cyborg and Bionic Systems (CBS) , 2017, vol. 2018–Janua, pp. 262–267. 89 [64] G. Durandau, D. Farina, and M. Sartori, “Robust Real-Time Musculoskeletal Modeling Driven by Electromyograms,” IEEE Trans. Biomed. Eng. , vol. 65, no. 3, pp. 556–564, Mar. 2018. [65] M. Chandrapal, X. Chen, W. Wang, B. Stanke, and N. Le Pape, “Investigating improvements to neural network based EMG to joint torque estimation,” Paladyn, J. Behav. Robot. , vol. 2, no. 4, pp. 185–192, Jan. 2011. [66] M. E. Hahn, “Feasibility of estimating isokinetic knee torque using a neural network model,” J. Biomech. , vol. 40, no. 5, pp. 1107–1114, Jan. 2007. [67] P. A. Huijing and G. C. Baan, “Stimulation level-dependent length-force and architectural characteristics of rat gastrocnemius muscle,” J. Electromyogr. Kinesiol. , vol. 2, no. 2, pp. 112– 120, Jan. 1992. [68] K. N. An, K. Takahashi, T. P. Harrigan, and E. Y. Chao, “Determination of Muscle Orientations and Moment Arms,” J. Biomech. Eng. , vol. 106, no. 3, pp. 280–282, 1984. [69] W. Meng, Q. Liu, Z. Zhou, Q. Ai, B. Sheng, and S. S. Xie, “Recent development of mechanisms and control strategies for robot-assisted lower limb rehabilitation,” Mechatronics , vol. 31, pp. 132–145, Oct. 2015. [70] Y. H. Yin, Y. J. Fan, and L. D. Xu, “EMG and EPP-Integrated Human–Machine Interface Between the Paralyzed and Rehabilitation Exoskeleton,” IEEE Trans. Inf. Technol. Biomed. , vol. 16, no. 4, pp. 542–549, Jul. 2012. [71] J. S. Campos Figueiredo, “Smart Wearable Orthosis to Assist Impaired Human Walking,” University of Minho, 2019. [72] STMicroelectronics, “UM1472: STM32F407VG MCU - User Manual.” 2017. [73] J. Wang, L. Tang, and J. E Bronlund, “Surface EMG Signal Amplification and Filtering,” Int. J. Comput. Appl. , vol. 82, no. 1, pp. 15–22, Nov. 2013. [74] C. J. De Luca, “The Use of Surface Electromyography in Biomechanics,” J. Appl. Biomech. , vol. 13, no. 2, pp. 135–163, May 1997. [75] A. Abdelouahad, A. Belkhou, A. Jbari, and L. Bellarbi, “Time and frequency parameters of sEMG signal — Force relationship,” in 2018 4th International Conference on Optimization and Applications (ICOA) , 2018, no. 1, pp. 1–5. [76] F. D. Farfán, J. C. Politti, and C. J. Felice, “Evaluation of EMG processing techniques using Information Theory,” Biomed. Eng. Online , vol. 9, no. 1, p. 72, 2010. [77] D. A. Winter, Biomechanics and Motor Control of Human Movement . Hoboken, NJ, USA: John Wiley & Sons, Inc., 2009.