Modeling the temperature dependence of dynamic mechanical properties and visco-elastic behavior of thermoplastic polyurethane using artificial neural network

polymers Article Modeling the Temperature Dependence of Dynamic Mechanical Properties and Visco-Elastic Behavior of Thermoplastic Polyurethane Using Artificial Neural Network Ivan Kopal 1,2,*, Marta Harniˇcárová1,3, Jan Valíˇcek 1,3,4 and Milena Kušnerová1,3 1 Institute of Physics, Faculty of Mining and Geology, Vysokáškola báˇnská—Technical University of Ostrava, 17. Listopadu 15, 708 33 Ostrava, Czech Republic; [email protected] (M.H.); [email protected] (J.V.); [email protected] (M.K.) 2 Department of Numerical Methods and Computing Modelling, Faculty of Industrial Technologies in Púchov, Alexander Dubˇcek University of Trenˇcín, Ivana Krasku 491/30, 020 01 Puchov, Slovakia 3 Regional Materials Science and Technology Centre, Vysokáškola báˇnská—Technical University of Ostrava, 17. Listopadu 15, 708 33 Ostrava, Czech Republic 4Technical Faculty, Slovak University of Agriculture in Nitra, Tr. A. Hlinku 2, 949 76 Nitra, Slovakia *Correspondence: [email protected]; Tel.: +420-597-323-559 Received: 11 September 2017; Accepted: 15 October 2017; Published: 18 October 2017 Abstract: This paper presents one of the soft computing methods, specifically the artificial neural network technique, that has been used to model the temperature dependence of dynamic mechanical properties and visco-elastic behavior of widely exploited thermoplastic polyurethane over the wide range of temperatures. It is very complex and commonly a highly non-linear problem with no easy analytical methods to predict them directly and accurately in practice. Variations of the storage modulus, loss modulus, and the damping factor with temperature were obtained from the dynamic mechanical analysis tests across transition temperatures at constant single frequency of dynamic mechanical loading. Based on dynamic mechanical analysis experiments, temperature dependent values of both dynamic moduli and damping factor were calculated by three models of well-trained multi-layer feed-forward back-propagation artificial neural network. The excellent agreement between the modeled and experimental data has been found over the entire investigated temperature interval, including all of the observed relaxation transitions. The multi-layer feed-forward back-propagation artificial neural network has been confirmed to be a very effective artificial intelligence tool for the modeling of dynamic mechanical properties and for the prediction of visco-elastic behavior of tested thermoplastic polyurethane in the whole temperature range of its service life. Keywords: thermoplastic polyurethanes; visco-elastic properties; dynamic mechanical analysis; stiffness-temperature model; artificial neural networks 1. Introduction Thermoplastic polyurethanes (TPUs) belong to the family of thermoplastic elastomers, which combine mechanical properties of rubber based materials with the good processability and recyclability of thermoplastics. They represent linear segmented semi-crystalline multi-block copolymers, composed of an alternation of short stiff hard crystalline segments and rather long soft amorphous flexible chains. The hard segments are made from di-isocyanate, while the soft segments consist of long and flexible polyether or polyester chains, which mutually connect two hard segments. In particular, the hard segments act as multi-functional tie points working as both Polymers 2017,9, 519; doi:10.3390/polym9100519 www.mdpi.com/journal/polymers Polymers 2017,9, 519 2 of 17 physical cross-links and reinforcing fillers. The domain structure formed by micro-phase separation of TPUs, due to the thermodynamic immiscibility or incompatibility between the hard and soft phase, endows these polymeric materials with their elastomeric properties, such that can be seen in cross-linked rubber networks [ 1 , 2 ]. The unique chemical structure of the TPUs results in a combination of interesting properties. They are characterized by a wide service temperature range, excellent tear and abrasion resistance, good resistance to non-polar solvents, high compression, tensile strength, and low-temperature elasticity. Therefore, TPUs are of great importance not only from an industrial but also from an academic point of view [3]. Similarly as of all polymeric materials, most of the TPUs properties are strongly dependent on temperature, which at different values induces different conformational changes in the polymer chains, leading to the corresponding transition processes that determine the behavior of polymers at various conditions [ 4 ]. In this sense, the influence of temperature on dynamic mechanical properties and visco-elastic behavior of TPUs plays a considerably important role and it is a longstanding subject of interest for many researchers. Nowadays, one of the most sensitive and most accurate methods used for study of dynamic mechanical properties, as well as of visco-elastic behavior of polymeric materials under various conditions (such as temperature, time, frequency of dynamic mechanical loading, and many others [5]), is represented by the dynamic mechanical analysis (DMA). 1.1. Dynamic Mechanical Analysis DMA is a testing and analytical technique that is commonly used to characterize the mechanical responses of materials subjected to periodic dynamic stress in a cyclic manner, mostly the sinusoidal deformation force with very small amplitude. This technique helps engineers and scientists to understand and subsequently to predict the long-term material behavior under cyclic loading conditions. In the DMA test of polymeric material, a weak sinusoidal stress is applied at various frequencies and the strain inside a specimen of the known geometry is recorded as the temperature increases, which ultimately allows determination of the storage modulus E 0 , loss modulus E 00 , and the damping factor tan δ , as well as the identification of relaxation events that occurred in the tested polymer sample following the transition processes due to the conformational changes of its macromolecular chains. The storage modulus (stiffness) provides a measure of elastic energy stored in the material, the loss modulus (energy absorption or damping) refers to the amount of energy dissipated in the form of heat in each cycle of the sinusoidal deformation, while the ratio of the loss modulus to the storage modulus gives the damping factor. Due to the visco-elastic nature of the polymeric material, the sinusoidal strain lags the applied stress by a phase angle δ , which is a measure of the visco-elastic damping or internal friction. The results obtained by investigations of dynamic mechanical properties or of visco-elastic behavior of polymeric materials using the DMA method are typically provided as a graphical representation of E 0 ,E 00 and tan δ versus temperature or time at one or more frequencies of dynamic mechanical loading (Figure 1). It is then possible to identify in those diagrams all of the relaxation transitions that occurred in the tested polymer at corresponding transition temperatures [6]. 1.2. Relaxation Transitions in Polymers A key relaxation process in polymers is a glass transition at a temperature, T g , which is frequently referred to as an α -transition, α -relaxation or as a primary relaxation. The T g or T α point is defined as a temperature, at which the amorphous portion of polymeric materials experiences a physical change from a hard and brittle glass-like condition to very flexible rubbery-like state. Due to the fact that the glass transition event covers a range of temperatures without discontinuity in the measured, T g , glass transition temperature is generally taken as a mid-point of this range. For amorphous portion of polymers, besides the glass transition, also one or more sub-T g phenomenon often referred to as secondary relaxations, can be observed. They are referred to as β -, γ - and δ -transitions at the temperatures T β ,T γ and T δ and they appear in order of the descending temperature. These transition Polymers 2017,9, 519 3 of 17 processes are the result of local segmental motions that occur in a glassy state. By ‘local’, it is meant that only a small group of atoms are involved in the sub-T g processes, the pure existence of which proves that material in a glassy state is a dynamic one. The glass transition occurs when large segments of polymer chains start moving, β -transition is related to either localized movement in the main chain or the large side movement, and γ -transition relates with the movements occurring in the side chain of polymers. The δ-transition is related to very small motions within macromolecule [7]. The authors Huh and Cooper in the work [ 8 ] have reported that TPUs may exhibit two separate glass transitions in the hard-and-soft-segment domains, respectively. The lower glass transition temperature T αII (T γ ) is associated with a change from stiff glassy behavior of the soft segments to their soft rubbery behavior and the higher one T αI (T α ) is due to the breakdown of hydrogen bonded interactions of Van der Waal’s forces between the rigid and flexible segments of TPUs. Above the glass transition temperature, upon reaching the flow temperature T f , the complete macromolecular chains can slip against each other. The amorphous structures of the polymer start to melt without chemical degradation of macromolecules. When the crystallite melting temperature T m is reached, the crystallites also start to melt. Due to the fact that the temperature range of crystallite melting exceeds the flow temperature of the amorphous state, the entire polymeric material will be plasticized. As long as no thermal degradation will occur in the molten phase at the decomposition temperature T d , the polymer can reversibly get back into the solidified state by cooling and crystallite phases will again be generated. However, the size and distribution of crystallites can be now differing from the original status [ 9 ]. The crystalline and amorphous fraction of the semi-crystalline polymer below the T g temperature are present in the glassy state, within the temperature range between T g and T m the amorphous fraction will be in the rubbery state and above the temperature T m the crystalline fraction of the polymer will also already pass into the rubbery state, while the highly crystalline polymers do not have the rubbery area. The higher the amorphous fraction in the polymer, the stronger is its rubbery area. At a melting point of the crystalline phase, which is lower than the flow temperature of the amorphous phase T f and only in exceptional cases it is identical, the polymer suddenly passes from the rubbery state to the plastic state. The interval between the temperatures T m and T f depends on the molecular mass, the degree of crystallinity, as well as on several other parameters. In the case of a high share of the crystalline phase the importance of T g disappears because the polymer retains the properties of elastic body with the low deformation ability (glass) till the phase transition at T m temperature. However, the glass transition, melting of crystallites and thermal degradation of macromolecular chains are typical phase transitions virtually of all semi-crystalline polymers, including the TPUs [10]. Polymers 2017, 9, 519 6 of 16 Dewar flask (Perkin Elmer, Baesweiler, Germany). The average values of E', E′′, and tan δ for each temperature T were computed. The measurement uncertainty was approx. 2%. The experimental results of DMA tests of the examined material in the form of temperature dependencies of the average storage modulus E′(T), loss modulus E′′(T), and damping factor tan δ(T) at a single constant frequency of dynamic mechanical loading of 1 Hz in the whole temperature range of its service life are presented in Figure 1. The frequency of 1 Hz was chosen due to the fact that the secondary transitions and other structural features of the tested material could be easily detected, and the obtained results could be in the case of need more easily related to the experimental data acquired with the use of other techniques (e.g., by DSC method [27]). Figure 1 shows also highlighted critical values of the storage modulus E′1 – E′4 and temperatures of relaxation events TαII, Tβ, TαI occurred in the tested TPU samples. In connection with the above described stiffness-temperature analysis of experimental data, the temperatures in the diagram in Figure 1 are presented in the absolute scale. Figure 1. Temperature dependences of the average storage modulus E′(T), loss modulus E′′(T), and damping factor tan δ(T) of tested thermoplastic polyurethanes (TPU) at a frequency of 1 Hz in the whole temperature range of its service life. Artificial Neural Network Modeling This work aimed at an investigation of temperature dependencies of the observed visco-elastic parameters E′(T), E′′(T), and of tan δ(T) of the investigated TPU at a given constant frequency of its dynamic mechanical loading, three multi-layer feed-forward artificial neural network models were created utilizing the error back-propagation learning algorithm with an architecture consisting of one input, one hidden and one output layer [28]. The temperature history T(t) of TPU during the DMA test, representing a data set of the input experimental data, was naturally chosen as the sole input parameter of the ANN for all three of the created models, while the corresponding experimental values of the storage modulus, loss modulus, as well as damping factor, were chosen as a target ANN data set for each model individually. The input and output layer has one neuron, which corresponds with the number of the input and output variables of individual models, while it was necessary to optimize the number of neurons in a hidden layer with the use of a trial and error method for each individual model separately. The main criteria for this optimization were the maximum network performance and its trouble-free learning. A single hidden layer was chosen in view of the practical requirement for the maximum network simplicity, its high stability, trouble-free learning, and minimal demands on computing power and computing time. Moreover, a number of mathematical works has shown that virtually any of the above described functions can be approximated with the required precision by means of a well-trained feed-forward ANN with a single hidden layer under Figure 1. Temperature dependences of the average storage modulus E 0 (T), loss modulus E 00 (T), and damping factor tan δ (T) of tested thermoplastic polyurethanes (TPU) at a frequency of 1 Hz in the whole temperature range of its service life. Polymers 2017,9, 519 4 of 17 1.3. Stiffness-Temperature Model of Thermoplastic Urethanes The authors Mahieux and Reifsnider in the work [ 11 ] showed, that the temperature dependence of the storage modulus E 0 (T) at a given constant frequency of dynamic mechanical loading can be quantitatively described with high level of reliability by physically well substantiated unified analytical model based on a Weibull distribution of the breakage of secondary bonds between macromolecular chains during the primary and secondary relaxation processes in any polymeric systems in the form of E0(T)= N ∑ i=1 ∆E0 ie−T Θimi , (1) where E 0 (T) is the temperature dependent storage modulus and Tis the absolute temperature of polymer, ∆ E 0i are the storage modulus magnitudes of particular transition steps, the coefficients Θi represent absolute transition temperatures, the parameters m i are the Weibull moduli corresponding to the statistics of the secondary bond breakage, and Nis the number of relaxation events observed in the investigated polymeric material. This model has been transformed later into a robust physically based analytical model for prediction of the stiffness modulus for a wide range of temperatures and frequencies/strain rates by Richeton et al. in the study [12]. The stiffness-temperature model, presented by the Equation (1), in the form of exponential temperature function E0(T)=E0 1−E0 2e−T Tγmγ +E0 2−E0 3e−T Tβmβ +E0 3−E0 4e−(T Tα)mα+E0 4e−T Tfmf , (2) where E 01 is the instantaneous storage modulus at the beginning of E 0 (T) DMA curve, E 02 is the instantaneous storage modulus immediately after the γ -transition, E 03 is the instantaneous storage modulus immediately after the β -transition and E 04 represents the instantaneous storage modulus at the beginning of the rubbery plateau (Figure 1), was successfully applied in one of our previous works in an analytical description of changes in the storage modulus of the tested TPU over a wide range of temperatures [ 13 ]. Both unknown physical parameters of the stiffness-temperature model (dynamic storage moduli and transition temperatures), as well as its statistical parameters (Weibull moduli) were estimated in the process of parametric fitting of the Equation (2) to the experimental E 0 (T) DMA curve using a trust region algorithm for a non-linear least squares method. However, critical values of the storage modulus E 01− E 04 can be determined from the E 0 (T) DMA curve (Figure 1), the transition temperatures T α ,T β ,T γ from the local maxima of the tan δ (T) curve (Figure 1) and the flow temperature T f from the onset of the constant value of E 0 on the E 0 (T) curve (Figure 1). Accordingly, in the process of the multi-parametric fitting of experimental data set it is enough to estimate the unknown Weibull moduli m α ,m β ,m γ am f , which quantify the width of the secondary bonds failure distribution corresponding to the level of crystallinity (m α ), the width of the secondary bond strengths distribution in the given temperature intervals (m β ,m γ ), and the width of secondary bond strength distribution corresponding to the TPU degree of cross-linking (m f ) [ 14 ]. At the same time, the temperature T αII (T γ ) is associated with the glass transition of TPU soft segments, T αI (T α ) with the glass transition of its hard segments, T β can be attributed to a short-range order translation and to re-orientational motions within the PTU crystals, whereas T f corresponds to global translations of whole polymer molecules between entanglements [ 15 ]. In our work [ 13 ], it has been shown that the stiffness-temperature model presented by the Equation (2) is very successful in describing variations of the elastic behavior of the tested TPU with temperature from its glassy state up to the flow region and within the property transition regions. At the same time, this model makes it possible to predict the stiffness-temperature dependence of different types of TPUs, including the newly proposed ones, without necessarily having to subject them to time demanding and costly DMA testing. Unfortunately, there is currently no physically justified unified analytical model that would be able to reliably describe also the viscous behavior of polymeric systems in the whole temperature Polymers 2017,9, 519 5 of 17 range throughout their service life, including all of the observed relaxation transitions in a way that Equation (1) describes their elastic behavior. However, the authors Ying Li et al. in work [ 16 ] present a predictive multi-scale computational framework to evaluate the visco-elastic properties of linear polymers based on their atomistic constituents. Many different multi-scale computational techniques are described in detail in [ 17 ]. Whereas, the artificial neural networks (ANNs) are, in general, excellent universal approximators not only of linear but also of non-linear functional dependencies between dependent and independent variables [ 18 ], it is quite natural to assume that temperature dependence of the loss modulus E 00 (T) and damping factor tan δ (T), as well as that of the storage modulus E 0 (T), could be successfully modeled right by them. 1.4. Artificial Neural Networks ANNs are intelligent mathematical computational methods inspired by biological neural networks and human brain that can learn from experience and generalize the learned things. Today they are used especially in the cases, where classic techniques fail, as well as other solutions for such problems, which can be formulated in the form of associative tasks [ 19 ]. ANNs are especially efficient in approximation of any continuous function with a finite number of discontinuities by mapping the relationships between vectors of the input (independent) and output (dependent) data [20]. With the right choice of structure and architecture they are capable to learn on the basis of the well-chosen input-output patterns to also model very complicated non-linear relations between the input and output data, for example to model a highly non-linear mechanical behavior of the materials such as polymers and their composites [ 21 , 22 ]. However, the acquired knowledge is, unfortunately, hidden in such ANN structures that cannot readily be extracted and interpreted. ANNs with multi-layer feed-forward architecture are the most commonly used neural networks in engineering applications thanks to their structural simplicity and relatively fast learning abilities [23]. The multi-layer feed-forward ANNs are non-parametric and robust mathematical models that are designed to find non-linear relationships between specified input-output data samples. This type of ANNs consists of at least of three layers of artificial neurons or processing units. The first layer is the input layer and the last layer is the output layer, while all of the layers between the input and output layers are referred to as the hidden layers. In this architecture of the ANNs, each neuron is connected to all of the neurons in the next layer but there are no connections between the neurons on the same layer. The signal propagates from the input layer, through all of the hidden layers, to the output layer in a forward direction. The numbers of neurons in the input, as well as in the output layers, are determined by the formulation of the investigated problem. The number of hidden layers and the number of neurons in these layers are related to the complexity of the relevant information contained in the learning or training data set. At the same time, the number of neurons in the hidden layers defines the capacity of the ANN. The knowledge about the relations between the input and output data is stored in the weights of connection between neurons, the values of which are updated during the supervised training process of the ANN with a set of known and representative values of the input-output data samples [ 24 ]. The training process starts with randomly set weights that are gradually updated by the minimization algorithm of squared difference between the target value and the computed output value on the output of each ANN neuron. Generally, to update the network weights iteratively, a technique called the back-propagation algorithm is used, which involves performing computations backward through the ANN [ 25 ]. The aim of the research described in the present work is exactly the modeling of the temperature dependencies of E 0 (T), E 00 (T), and tan δ (T) of the investigated TPU submitted to DMA tests at a constant single frequency of dynamic mechanical loading in the whole temperature interval of its service life, or from its glassy state up to the flow region, using the multi-layer feed-forward ANNs utilizing the back-propagation learning algorithm. Polymers 2017,9, 519 6 of 17 2. Materials and Methods Most of conventional TPUs are based either on polyester or polyether polyols, 4,4-diphenylmethane diisocyanate as an isocyanate component, and 1,4-butanediol as a chain extender [ 26 ]. In the present work, the synthesis of the tested TPU was performed in one liter of glass reactor by mixing two commercial reactive components—Axson Technologies PX 522/HT POLYOL and Axson Technologies PX 521-522 HT ISO ISOCYANATE (Axson Technologies, Newmarket Suffolk, UK)—under normal pressure and at room temperature. In order to ensure degassing and homogenization before polymerization, the reactive mixture was mixed for approx. 5 min with the use of an ultrasound mixer WELDER (WELDER, Hiroshima, Japan) in a VB vacuum chamber (VB, Prague, Czech Republic). Subsequently, the mixture was poured into a mold and left to cure at 80 ◦ C for 4 h. The post-curing process proceeded at 100 ◦ C for 16 h. Under these conditions the addition of catalyst was not necessary. From the resulting material, ten rectangular shaped samples with dimensions of 30 mm (length) × 6 mm (width) × 1.5 mm (thickness) were cut off using a Gravograph LS100 40 W CO 2 laser cutter (Gravograph, Paris, France). The samples were then subjected to dynamic mechanical analysis over the entire temperature range of the TPU service life at a single constant frequency of dynamic mechanical loading. The DMA test measurements were carried out using a Perkin Elmer Pyris Diamond 8000 Dynamic Mechanical Analyzer (Perkin Elmer, Baesweiler, Germany) in the uniaxial tensile mode at a constant frequency of dynamic mechanical loading of 1 Hz, with an amplitude of 20 µ m from − 125 up to 250 ◦ C and at a constant heating rate of 3 ◦ C · min −1 . The amplitude of applied dynamic stress was 0.1 MPa and the strain rate was 0.1 s –1 . The strain measurements were performed by a linear variable differential transformer in the DMA equipment with a span of 2 mm and a mean resolution of 2 nm. The measurements were performed on ten specimens cooled by liquid nitrogen from a Dewar flask (Perkin Elmer, Baesweiler, Germany). The average values of E 0 ,E 00 , and tan δ for each temperature Twere computed. The measurement uncertainty was approx. 2%. The experimental results of DMA tests of the examined material in the form of temperature dependencies of the average storage modulus E 0 (T), loss modulus E 00 (T), and damping factor tan δ (T) at a single constant frequency of dynamic mechanical loading of 1 Hz in the whole temperature range of its service life are presented in Figure 1. The frequency of 1 Hz was chosen due to the fact that the secondary transitions and other structural features of the tested material could be easily detected, and the obtained results could be in the case of need more easily related to the experimental data acquired with the use of other techniques (e.g., by DSC method [ 27 ]). Figure 1shows also highlighted critical values of the storage modulus E 01 –E 04 and temperatures of relaxation events T αII ,T β ,T αI occurred in the tested TPU samples. In connection with the above described stiffness-temperature analysis of experimental data, the temperatures in the diagram in Figure 1are presented in the absolute scale. Artificial Neural Network Modeling This work aimed at an investigation of temperature dependencies of the observed visco-elastic parameters E 0 (T), E 00 (T), and of tan δ (T) of the investigated TPU at a given constant frequency of its dynamic mechanical loading, three multi-layer feed-forward artificial neural network models were created utilizing the error back-propagation learning algorithm with an architecture consisting of one input, one hidden and one output layer [ 28 ]. The temperature history T(t) of TPU during the DMA test, representing a data set of the input experimental data, was naturally chosen as the sole input parameter of the ANN for all three of the created models, while the corresponding experimental values of the storage modulus, loss modulus, as well as damping factor, were chosen as a target ANN data set for each model individually. The input and output layer has one neuron, which corresponds with the number of the input and output variables of individual models, while it was necessary to optimize the number of neurons in a hidden layer with the use of a trial and error method for each individual model separately. The main criteria for this optimization were the maximum network performance and its trouble-free learning. A single hidden layer was chosen in view of the practical requirement for Polymers 2017,9, 519 7 of 17 the maximum network simplicity, its high stability, trouble-free learning, and minimal demands on computing power and computing time. Moreover, a number of mathematical works has shown that virtually any of the above described functions can be approximated with the required precision by means of a well-trained feed-forward ANN with a single hidden layer under the mild assumptions of the activation function (transfer function), which follows from the so called universal approximation theorem for feed-forward ANNs [29]. Between the input and hidden layer of the ANN a hyperbolic tangent sigmoid transfer function was used in the form of [30] tan sig(n)=1−e−2n 1+e−2n, (3) where nrepresents the net input into the hidden layer transfer function, which keeps the neurons output between certain limits as it is the case in the biological neuron. In order to avoid limitation of the network output by a small range of values, the pure linear transfer function purelin(m)=m, (4) where mis the net input into the transfer function of the output layer, was used between the hidden and output layer [30]. Generally, each neuron of the ANN represents its elementary building and computing unit located in the appropriate network layer. The input layer only transmits the input signal p to the hidden layer neurons without performing any calculations. Neurons of the hidden and output layer contain input, bias, weight, summing junction, transfer function, and output. At the input of each neuron of the hidden layer, each element p i of the input data vector p with the length R 1 is multiplied by its weight iw 1,1 . The weighted inputs of all S 1 neurons of the hidden layer are summed in their summing junctions. This weighted sum of the inputs together with the added bias b 1j of the current neuron create an activation potential n1 j=∑ i,j iw1,1 pi+b1 j(5) leading to the activation of its output, while the bias is just a constant external input with a value of 1 multiplied by its weight, which represents the basic level of activity of the given neuron [ 31 ]. The activation signal, given by the Equation (5), or the net input of the hidden neuron n 1j is then fed as the only argument into the non-linear transfer function, given by the Equation (3), which generates its output a1 j=f1n1 j=tan sign1 j=1−e−2n1 j 1+e−2n1 j . (6) The total output of the hidden layer with S1hidden neurons, represented in a matrix form as a1=f1IW1,1p+b1, (7) is then sent to the output layer. Net input of the output layer n2 j=∑ j lw2,1a1 j+b2 j, (8) created by summing in the summing junction, is then inserted into the pure linear transfer function given by the Equation (4), which returns the output signal a2 j=f2n2 j=purelinn2 j=n2 j, (9) Polymers 2017,9, 519 8 of 17 or the output signal represented in a matrix form a2=f2LW2,1a1+b2. (10) Total output signal of the network ycan be then presented as a matrix in the form y=a2=f2LW2,1f1IW1,1p+b1+b2. (11) The weight matrix IW1,1 represents a matrix of weights between neurons of the hidden and input layer, while LW2,1 represents the weight matrix between neurons of the hidden and output network layers. The upper index in the designation of all of the network elements represents its respective layer, it means that index 1 denotes a hidden layer and index 2 denotes an output layer. The lower index i belongs to the elements of the input vector p , while the lower index j corresponds to the individual neurons in the respective layer (indicated by the upper index). Due to the fact that the output of the hidden layer a1 is also the input of the output layer with the output a2, the hidden layer can be, in general, analyzed also as a single layer network with S1inputs, S2neurons and S2×S1weight matrix W2= w2 1, 1 w2 1, 2 · · · w2 1, S1 w2 2, 1 w2 2, 2 · · · w2 2, S1 w2 S2, 1 w2 S2, 2 · · · w2 S2,S1  . (12) with weights w 2r,q , where the lower index q denotes an element of the input vector a1 , while the lower index r denotes the appropriate neuron, into which the signal from this element is brought [ 32 ]. Each neuron of the hidden layer of the proposed ANN is connected by means of weights with each element of the input vector and with neuron of the output layer, and by bias also with external environment of the network. The ANN architecture for modeling the temperature dependencies of all three observed visco-elastic parameters is in an abbreviated notation [ 32 ], schematically shown in Figure 2. The input vector p in our case for all three generated ANN models represents a vector of the temperature history T of the TPU during its DMA test, while the output signal y of the network for individual models are the vectors of the corresponding values E 0 (T), E 00 (T), or tan δ (T) at temperatures, T(t) predicted by the network. Vectors and matrices are in this work marked in straight bold fonts. Polymers 2017, 9, 519 8 of 16   . 212,122 baLWfa  (10) Total output signal of the network y can be then presented as a matrix in the form    . 211,112,122 bbpIWfLWfay  (11) The weight matrix IW1,1 represents a matrix of weights between neurons of the hidden and input layer, while LW2,1 represents the weight matrix between neurons of the hidden and output network layers. The upper index in the designation of all of the network elements represents its respective layer, it means that index 1 denotes a hidden layer and index 2 denotes an output layer. The lower index i belongs to the elements of the input vector p, while the lower index j corresponds to the individual neurons in the respective layer (indicated by the upper index). Due to the fact that the output of the hidden layer a1 is also the input of the output layer with the output a2, the hidden layer can be, in general, analyzed also as a single layer network with S1 inputs, S2 neurons and S2 × S1 weight matrix . 22 2 2 1 2 2 222 212 2 1 221 211 2 1222 1 1 SSSS S S www www www ,,, , ,, , ,, ... ... ... W (12) with weights w2r,q, where the lower index q denotes an element of the input vector a1, while the lower index r denotes the appropriate neuron, into which the signal from this element is brought [32]. Each neuron of the hidden layer of the proposed ANN is connected by means of weights with each element of the input vector and with neuron of the output layer, and by bias also with external environment of the network. The ANN architecture for modeling the temperature dependencies of all three observed viscoelastic parameters is in an abbreviated notation [32], schematically shown in Figure 2. The input vector p in our case for all three generated ANN models represents a vector of the temperature history T of the TPU during its DMA test, while the output signal y of the network for individual models are the vectors of the corresponding values E′(T), E′′(T), or tan δ(T) at temperatures, T(t) predicted by the network. Vectors and matrices are in this work marked in straight bold fonts. Figure 2. Schematic of the artificial neural networks (ANN) architecture for modeling the temperature dependencies of the storage modulus E′(T), loss modulus E′′(T) and damping factor tan δ(T) of the investigated TPU at a given constant frequency of its dynamic mechanical loading. At first, it was necessary to teach the created ANN to recognize the relationships between the input and output data of the DMA test, then to verify the reliability of the learned recognition and finally to test its ability to generalize the acquired and verified knowledge at prediction of new experimental results of the DMA test. The input and target data of the created ANN were therefore randomly divided to training, validation, and testing data sets in the proportion 0.70:0.15:0.15, Figure 2. Schematic of the artificial neural networks (ANN) architecture for modeling the temperature dependencies of the storage modulus E 0 (T), loss modulus E 00 (T) and damping factor tan δ (T) of the investigated TPU at a given constant frequency of its dynamic mechanical loading. At first, it was necessary to teach the created ANN to recognize the relationships between the input and output data of the DMA test, then to verify the reliability of the learned recognition and finally to test its ability to generalize the acquired and verified knowledge at prediction of new experimental results of the DMA test. The input and target data of the created ANN were therefore randomly Polymers 2017,9, 519 9 of 17 divided to training, validation, and testing data sets in the proportion 0.70:0.15:0.15, respectively. There is no principle limitation for these divisions, but in generally, it is common that the training data set has larger amounts of data than the other sets [ 33 ]. On the other hand, the number of training data can strongly influence the ANN predictive quality [34]. The training data were used for supervised learning of the network [ 35 ], during which the difference between the values y k of the output vector y predicted by the network and the values d k of the vector of the desired (training) data d is minimized with the use of step by step modification of the weights IW1,1,LW2,1 and biases b1,b2. Values of the error vector δ=d−y(13) is appreciated with the use of the mean squared error (MSE) function E(w,b)=1 2l l ∑ k=1 (dk−yk)2, (14) where lis the total number of its elements, which is minimized during the ANN training process [ 36 ], which leads to an adaptation of weights and biases of the network to the training data, or to its optimization of the training data set. The Levenberg–Marquardt back-propagation minimization algorithm [ 37 ], also known as the damped least squares method, based on the modification of gradient Gauss–Newton method [ 38 ], was used for training of all three ANN models. The iterative scheme of the Levenberg–Marquardt algorithm for the modification of weights and biases with the error function MSE has the form: zk+1−zk=JT kJk+λkI−1JT kEk, (15) where zrepresents the weights wand biases bof the network, the product JTkJk is referred to as Hessian matrix approximation for non-linear least square optimization [ 39 ], Jk is the Jacobian matrix that contains first derivatives of the network MSE with respect to the weights and biases [ 40 ], I is the identity matrix [ 41 ], λk is the non-negative learning parameter [ 42 ] and the symbol T { \ displaystyle ˆ{ \ mathsf {T}}}denotes the matrix transpose [ 43 ]. The iterative learning process starts with a random adjustment of the weight and its adaptation continues until reaching the specified maximum number of iterations (epochs), or until the MSE network is minimized to a pre-determined target value. The learning process, during which the ANN seeks through the step-by-step modification of weights and biases the most accurate training data model, must be terminated before the network also begins to adapt to the noise present in each experimental data, which causes a dramatic reduction of its predictive abilities for the unknown data. Therefore, during minimization of the error function, a development of the error function in the validation data set is also monitored. In the event that error of the validation data begins to increase during several iterative cycles, while the error of the training data is still decreasing or is no longer changing, the training process is prematurely terminated in order to prevent the so called overfitting of the network, the cause of which is usually too high a number of neurons in the hidden layer [ 44 ]. The weights and biases, at which the MSE of the validation data set was minimal, are then taken as the resulting values. It means that the training data set is used to fit the model by finding its unknown but correct weights and biases, whereas the validation data set is used to test how well the model performs on the unseen data, or for debugging the ANN architecture (the number of hidden neurons or possibly also hidden layers). The testing dataset contains the data, with which the ANN did not come into contact during the learning process or during validation, and which serve for verification of the level of network learning or for verification of its ability to generalize the acquired knowledge stored in the values of weights and biases values at prediction of new experimental results of the DMA test at new temperatures, unknown to the network. A significant difference in the number of iterations needed to achieve a Polymers 2017,9, 519 16 of 17 13. Kopal, I.; Bakošová, D.; Koštial, P.; Janˇcíková, Z.; Valíˇcek, J.; Harniˇcárová, M. Weibull distribution application on temperature dependence of polyurethane storage modulus. Int. J. Mater. Res. 2016 ,107, 472–476. [CrossRef] 14. Rinne, H. The Weibull Distribution: A Handbook; CRC Press: Boca Raton, FL, USA, 2009. 15. Brinson, H.F.; Brinson, L.C. Polymer Engineering Science and Viscoelasticity, 2nd ed.; Springer: Berlin, Germany, 2014. 16. Li, Y.; Tang, S.; Abberton, B.C.; Kröger, M.; Burkhart, C.; Jiang, B.; Papakonstantopoulos, G.J.; Poldneff, M.; Liu, W.K. A predictive multiscale computational framework for viscoelastic properties of linear polymers. Polymer 2012,53, 5935–5952. [CrossRef] 17. Li, Y.; Abberton, B.C.; Kröger, M.; Liu, W.K. Challenges in multiscale modeling of polymer dynamics. Polymers 2013,5, 751–832. [CrossRef] 18. Fausett, L.V. Fundamentals of Neural Networks, 1st ed.; Prentice Hall: New York, NY, USA, 1994. 19. Zhang, Z.; Friedrich, K. Artificial neural networks applied to polymer composites: A review. Compos. Sci. Technol. 2003,63, 2029–2044. [CrossRef] 20. Sifaoui, A.; Abdelkrim, A.; Benrejeb, M. On the Use of Neural Network as a Universal Approximator. Int. J. Sci. Tech. Control Comput. Eng. 2008,2, 386–399. 21. Bessa, M.A.; Bostanabad, R.; Liu, Z.; Hu, A.; Apley, D.W.; Brinson, C.; Chen, W.; Liu, W.K. A framework for data-driven analysis of materials under uncertainty: Countering the curse of dimensionality. Comput. Methods Appl. Mech. Eng. 2017,320, 633–667. [CrossRef] 22. Le, B.A.; Yvonnet, J.; He, Q.C. Computational homogenization of nonlinear elastic materials using neural networks. Int. J. Numer. Meth. Eng. 2015,104, 1061–1084. [CrossRef] 23. Norgaard, M.; Ravn, O.; Poulsen, N.K.; Hansen, L.K. Neural Networks for Modeling and Control of Dynamic Systems; Springer: Berlin, Germany, 2002. 24. Jin, D.; Lin, A. Advances in Computer Science, Intelligent System and Environment; Springer: Berlin, Germany, 2011. 25. Aliev, R.; Bonfig, K.; Aliew, F. Soft Computing; Verlag Technic: Berlin, Germany, 2000. 26. Bhowmick, A.K.; Stephens, H.L. Handbook of Elastomers, 2nd ed.; CRC-Press: Boca Raton, FL, USA, 2000. 27. Qi, Z.N.; Wan, Z.H.; Chen, Y.P. A comparative DSC method for physical aging measurement of polymers. Polym. Test. 1993,12, 185–192. [CrossRef] 28. Freeman, J.A.; Skapura, D.M. Neural Networks Algorithms, Applications, and Programming Techniques; Addison-Wesley Publishing Company: Boston, MA, USA, 1991. 29. Auer, P.; Burgsteiner, H.; Maass, W.A. A learning rule for very simple universal approximators consisting of a single layer of perceptrons. Neural Netw. 2008,21, 786–795. [CrossRef] [PubMed] 30. Shanmuganathan, S.; Samarasinghe, S. Artificial Neural Network Modelling; Springer: Berlin, Germany, 2016. 31. Du, K.L.; Swamy, M.N.S. Neural Networks and Statistical Learning; Springer: Berlin, Germany, 2014. 32. Beale, M.H.; Hagan, M.T.; Demuth, H.B. Neural Network Toolbox ™ User’s Guide; The MathWorks, Inc.: Natick, MA, USA, 2017. 33. Bowden, G.J.; Maier, H.R.; Dandy, G.C. Optimal division of data for neural network models in water resources applications. Water Resour. Res. 2002,38, 2-1–2-11. [CrossRef] 34. Trebar, M.; Susteric, Z.; Lotric, U. Predicting mechanical properties of elastomers with neural networks. Polymer 2007,48, 5340–5347. [CrossRef] 35. Zhang, Z.; Klein, P.; Friedrich, K. Dynamic mechanical properties of PTFE based short carbon fibre reinforced composites: Experiment and artificial neural network prediction. Compos. Sci. Technol. 2002 ,62, 1001–1009. [CrossRef] 36. Mohri, M.; Rostamizadeh, A.; Talwalkar, A. Foundations of Machine Learning; MIT Press: Cambridge, MA, USA, 2012. 37. Hagan, M.T.; Menhaj, M. Training feed-forward networks with the Marquardt algorithm. IEEE Trans. Neural Netw. 1999,5, 989–999. [CrossRef] [PubMed] 38. Croeze, A.; Pittman, L.; Reynolds, W. Nonlinear Least-Squares Problems with the Gauss-Newton and Levenberg-Marquardt Methods; University of Mississippi: Oxford, UK, 2012. 39. Fairbank, M.; Alonso, E. Efficient Calculation of the Gauss-Newton Approximation of the Hessian Matrix in Neural Networks. Neural Comput. 2012,24, 607–610. [CrossRef] [PubMed] 40. Wackerly, D.; Mendenhall, W.; Scheaffer, R.L. Mathematical Statistics with Applications, 7th ed.; Thomson Higher Education: Belmont, TN, USA, 2008. Polymers 2017,9, 519 17 of 17 41. Barnes, R.J. Matrix Differentiation; University of Minnesota: Minneapolis, MN, USA, 2017. 42. Madi´c, M.J.; Radovanovi´c, M.R. Optimal Selection of ANN Training and Architectural Parameters Using Taguchi Method: A Case Study. FME Trans. 2011,39, 79–86. 43. Nocedal, J.; Wright, S.J. Numerical Optimization, 2nd ed.; Springer: Berlin, Germany, 2006. 44. Zhang, T.; Yu, B. Boosting with Early Stopping: Convergence and Consistency. Ann. Stat. 2005 ,33, 1538–1579. [CrossRef] 45. Syed, A.H.; Munir, A. Distribution of Mean of Correlation Coefficients: Mean of Correlation Coefficients; LAP Lambert Academic Publishing: Saarbrücken, Germany, 2012. 46. Nematollahi, M.; Jalali-Arani, A.; Golzar, K. Organoclay maleated natural rubber nanocomposite. Prediction of abrasion and mechanical properties by artificial neural network and adaptive neuro-fuzzy inference. Appl. Clay Sci. 2014,97–98, 187–199. [CrossRef] 47. Simon, L. An Introduction to Multivariable Mathematics; Morgan & Claypool: St. Louis, CA, USA, 2008. 48. Shi, F.; Wang, X.C.; Yu, L.; Li, Y. MATLAB 30 Case Analysis of MATLAB Neural Network; Beijing University Press: Beijing, China, 2009. 49. Wong, S.S.M. Computational Methods in Physics and Engineering, 2nd ed.; World Scientic Publishing: New York, NY, USA, 1977. 50. Suzuki, K. Artificial Neural Networks—Industrial and Control Engineering Applications; InTech: Rijeka, Croatia, 2011. © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).