Machine learning techniques for the ab initio Bravais lattice determination

ORIGINAL ARTICLE Machine learning techniques for the ab initio Bravais lattice determination Esther-Lydia Silva-Ramírez 1 | Inmaculada Cumbrera-Conde 2 | Rafael Cano-Crespo 3 | Francisco-Luis Cumbrera 3 1 Department of Computer Science and Engineering, University of Cádiz, Puerto Real, Spain 2 Department of Private International Law, Macquarie University, Sydney, New South Wales, Australia 3 Departamento de Física de la Materia Condensada, Universidad de Sevilla, Sevilla, Spain Correspondence Francisco-Luis Cumbrera, Departamento de Física de la Materia Condensada, Universidad de Sevilla, 41012 Sevilla, Spain. Email: [email protected] Abstract Machine learning-based algorithms have been widely applied recently in different areas due to its ability to solve problems in all fields. In this research, machine learning techniques classifying the Bravais lattices from a conventional X-ray diffraction diagram have been applied. Indexing algorithms are an essential tool of the preliminary protocol for the structural determination problem in crystallography. The task of reverting the obtained information in reciprocal lattice to direct space is a complex issue. As an alternative way to afford this problem, different machine learning algorithms have been applied and a comparison between them has been conducted. The obtained accuracy was 95.9% using 10-fold cross-validation (while the best result obtained so far has been 84%). A model based on Bragg positions was our unique predictor, allowing us to obtain the set of the interplanar lattice distances. Our model was successfully checked with a complex example. In addition, our procedure incorporates the following advantages: robustness versus imprecision in data acquisition and reduction of the amount of necessary input data. This is the first time so far that such classification has been carried out in true ab initio condition. KEYWORDS Bravais lattices, crystallography, machine learning 1|INTRODUCTION Machine learning (ML) methods have achieved recently outstanding contributions in the materials research community as well as in other science domains (Agatonovic-Kustrin & Beresford, 2000; Bhadeshia, 1999; Dai et al., 2020; Scott et al., 2007; Sha & Edwards, 2007; Shetty et al., 1999; Woinaroschy et al., 2000; Zhang et al., 2008). ML developments include methods which simulate brain working, for instance, artificial neural networks (ANN), or simulate human experience and draw conclusions as expert systems. Somehow, we can state that we are facing the solution of traditional complex problems with other different paradigm which offers greater speed and efficiency. The goal of our work is concerned to the solution of an outstanding problem of crystallography: the assignation of the Bravais lattice in structural determination. We will approach that key problem from conventional ML methods. Following, we address the task of laying out the fundamentals of such a problem. Crystallography is the science that studies the atomic arrangements of crystalline solids. It is an axiomatic discipline that relies on two postulates: The postulates of Bravais and Schoenflies–Fedorov. First of them, known as micro-periodicity principle, states the foundation for Received: 11 February 2022 Revised: 13 August 2022 Accepted: 25 September 2022 DOI: 10.1111/exsy.13160 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. © 2022 The Authors. Expert Systems published by John Wiley & Sons Ltd. Expert Systems. 2023;40:e13160. wileyonlinelibrary.com/journal/exsy 1of17 https://doi.org/10.1111/exsy.13160 translational symmetry. All possible arrangement of congruent points in a crystalline solid (those which remain invariants under the operations of an algebraic group of translations) are called Bravais lattices. Group theory states that they are only possible 14 Bravais lattices in 3-D space. Figure 1shows the 14 possible Bravais Lattices classified according to the seven crystal systems. Rows represent each one of the seven crystal systems. The second column indicates the geometric cell parameters and successive columns a drawing of the unit cell according to the criteria: •Lattice points only at the vertices (primitive cells). •Lattice points at the vertices and in the center of the cell (body-centered cells). •Lattice points at the vertices and in the center of the basal faces (base-centered cells). •Lattice points at the vertices and in the center of all faces (face-centered cells). The first step in solving an unknown crystal is the determination of the unit cell (the repeating unit), as a precondition for the subsequent determination of the relative positions of atoms within that cell. In fact, two separate kinds of information can be extracted from diffraction spectra: the first one comes from the geometrical arrangement of the reflections, which gives us the information about the crystal lattice and the symmetry of the crystal. This kind of information source is obtained by means of the indexing algorithms (IA in this article). The second one comes from the intensity of reflections and is concerned with the information about the cell content. The so called indexing algorithms are an essential part of every data collection software package. The well-known underlying principle of indexing methods is universal: they are based on matching experimental scattering vectors to some vectors of the reciprocal lattice (the algebraic FIGURE 1 The 14 Bravais lattices in three dimensions. Source: D.V. Anghel, Bravais lattice table, 2003 2of17 SILVA-RAMÍREZ ET AL. 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License dual lattice in the wave vector space). The most extensively used are TREOR90 (Werner et al., 1985), DICVOL91 (Boultif & Louër, 2004) and ITO15 (Visser, 1969), which work on very different approaches to solve the problem. Those current approaches fall into two main categories (Habershon et al., 2004): exhaustive search or deductive recognition of relationship between subsets of data. In practice, the measurement of the positions of the so called Bragg peaks is equivalent to the measurement of the modules of a subset of the vectors of the reciprocal lattice, in fact the shorter ones, without any knowledge about orientation relations. The task of reverting that limited information to direct space, in order to build the correct Bravais lattice, is a complex issue. Figure 2shows a diffraction spectra of Boron Carbide manufactured and obtained in our laboratory. Horizontal axis is the scattering angle (2θ) and vertical axis is the intensity in arbitrary units. The Bragg peaks are superposed on an inelastic smooth background which lacks of crystallographic relevance. By means of any peak-hunter algorithm we obtain the positions of the maxima which can be introduced in Bragg law: λ¼2dhkl sinθ,ð1Þ to obtain the interplanar lattice distances of the (hkl) family of planes. Now, the equation to solve is Qhkl ¼1 d2 hkl ¼fa,b,c,α,β,γ,h,k,lðÞ,ð2Þ where a,b,c,α,βand γare the linear and angular parameters of the unit cell and the hklðÞMiller indices. Miller indices are unknown three integer numbers that specify the orientation of plane lattice satisfying the Bragg law. All the parameters in f a, b, c, α,β,γ,h,k,lðÞare unknown. Further obstacles to the correct indexing are related to geometrical ambiguities because while a unit cell defines the lattice, a lattice can be described in infinite different ways. Proof of lack of satisfaction are the new attempts published in bibliography. As a sign of that, Jacobson (1997) proposed a cosine Fourier transform to obtain direct space cell vectors. Coelho (2017) proposed the same way a new indexing algorithm, independent of peak positions, to overcome the issue of inaccuracy in the peaks measurement. It is tempting to afford the indexing problem by means of ML methods, based on learning (training and validation) from a selected database. Following, we will expose the previous attempts so far. Recently, Liang et al. (2020) made available to public the tool called Crystal Structure Prediction Network (CRYSPNet) which is based on neural networks and can predict several features as Bravais lattice, lattice parameters of inorganic materials based on its chemical composition. The code has been trained and validated over a large number of the Inorganic Crystal Structure Database (ICSD) entries. Despite their results clearly improve other strategies, the higher accuracy reached is about 84% for the metal group set. The limitations were justified on the basis of class imbalance, particularly for the low symmetry compounds. We agree with that observation but we will suggest later that ANN models are by far less efficient than other ML techniques for this kind of classification problem. In any case its algorithm is not universal as it is restricted to groups of inorganic compounds. In the same direction of thought is the work of Ryan et al. (2018). His approach, based on ANN, demonstrates the ability of deep learning to extract meaningful information from large repository data. Although the evaluation of crystalline structures falls within the scope of their work, it was not contemplated the simple but crucial challenge of identifying the Bravais lattice from the only predictor of Bragg positions obtained from the diffraction spectrum. FIGURE 2 Diffraction spectra of boron carbide SILVA-RAMÍREZ ET AL.3of17 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Another outstanding contribution includes the work of Lee et al. (2020), which reports a deep learning protocol for phase identification and quantification in multiphase inorganic compounds in the quaternary Sr Li Al O. Finally, we refer to the relevant work of Wang et al. (2020) and Oviedo et al. (2019). In both contributions, the authors, by starting from a large set of predictors, made crystallographic classification limited to metal organic frameworks or seven space groups, respectively. It is the first time a study related to all sort of materials has been conducted by using the interplanar lattice distances as only predictor. The aim is a fast and reliable method of assignation of Bravais lattice avoiding the traditional indexing algorithms. None of the classic algorithms are conclusive versus complex problems. In fact, practice suggest the combined use of the three algorithms to compare matches and differences. Our work concerns to the bottleneck task in the preliminary treatment of X-ray data. We have focused our attention in that point (Bravais lattice assignation), once this problem has been solved, the determination of lattice parameters becomes trivial. Then, our challenge is to address the problem by using as only predictor the list of interplanar lattice spacing dhkl. In addition, we intend our method to be applicable to all sort of materials and not restricted to a particular subset as in previous attempts. Then, the originality of this work lies in the fact that we afford a crucial problem in crystallography based on a single predictor, without the support of large repository data and applicable to all kinds of materials. Previous attempts were restricted to particular subclasses as metals or ceramics as in references: Lee et al. (2020); Liang et al. (2020), and others. The article sequence is as follows. In Section 2, the applied methodology is described. The main results are presented in Sections 3and 4. The following section explores the reduction of the input data and the selection of the more significant attributes, Section 5. Section 6addresses the issue of robustness face to inaccuracies in data acquisition or face to the presence of intruders or missing data. In section 7, the models were tested face to a very complex case. Finally, Section 8deals with the most prominent conclusions. 2|METHODOLOGY In this Section the methodology followed in this work is described. We try to highlight the challenge that the study carried out in this research represents. The explicit mode of Equation (2) is shown in Equation (3) where the lattice spacings dhkl are related to the unit cell parameters trough the complex relation 1 d2 hkl ¼ h a h=a cosγcosβ k=b 1 cosα l=c cosα1   þ þk b 1h=a cosα cosγk=b cosα cosβl=c1   þ þl c 1 cosγh=a cosγ1k=b cosβcosαl=c   2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 4 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 5  1 cosγcosβ cosγ1 cosβ cosβcosα1   1 ,ð3Þ and in turn these relate to the Bragg position by means of Bragg law, Equation (1). From this equation, we can infer the advantage of choosing lattices spacing as descriptor instead of Bragg positions, since for the latter choice the dependence with lattice parameters is complicated by the presence of an arcsin law. Although in 3-D space there are 14 Bravais lattices, Figure 1, in practice hexagonal unit cell can be described as rhombohedral by means of: ar¼br¼cr¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffi ah 3þc2 h 9 s αr¼βr¼γr¼cos12c2 h3a2 h 2c 2 h3a2 h  ! ,ð4Þ where ar,b r,c rαr,βrand γrare lattice parameters for rhombohedral unit cell while ahand chare the lattice parameters for hexagonal unit cell. In a similar way, rhombohedral lattices can be expressed as hexagonal ones. That is why we have reduced the number of classes to 13 by grouping together the hexagonal and rhombohedral primitive lattices. As a matter of fact, we have established the following correspondence: 4of17 SILVA-RAMÍREZ ET AL. 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Primitive cubic 1 Body centered cubic 2 Face centered cubic 3 Primitive tetragonal 4 Body centered tetragonal 5 Primitive orthorhombic 6 Body centered orthorhombic 7 Base centered orthorhombic 8 Face centered orthorhombic 9 Primitive monoclinic 10 Base centered monoclinic 11 Triclinic 12 Hexagonal and trigonal 13 Our starting set was made from 500 entries, 100 from the ICSD database (Bergerhoff et al., 1983) and the remaining from the crystal open database (COD) data set (Vaitkus et al., 2021). The selection was random so as to avoid any bias, and the only restriction was having at least 3% of each class. The histogram of the starting data set is shown in Figure 3. This graph represents how many instances are classified in each class. For example, for class 1 there are 29 records, 32 instances are classified in class 2, and so on. At this respect, the percentage of each Bravais lattice in the ICSD data set can be read in Liang et al. (2020). Two facts are well known: 1. The suite of lattice distances is monotonously decreasing with the scattering angle. 2. Traditional methods need at least the first 20 reflections as input data, to avoid ambiguities. With regard to both points, all lattice distances were divided by the largest one which is the first. This way we have a sequence of real numbers starting from 1 and monotonously decreasing afterwards. According to the usual methods, we have chosen the first 20 lattice distances, although the normalization procedure reduces by one unit the number of significant input data. In this way, we have 20 relationships with 66 unknown: 6 defining the metric of the unit cell and 20 triples of integer numbers hklðÞ. Once we solved Equation (3), we obtained the unit cell and the set of (hkl) integers which allow us to build the corrected Bravais lattice. In Table 1an extract of data set is shown. The first 19 columns represent the described attributes and the last column represents the target variable for the classification problem. For the particular case of Bravais lattices of the cubic system, Equation (3) reduces to: dhkl ¼a ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi h2þk2þl2 p:ð5Þ For this particular case of cubic symmetry, the sequence of normalized data does not depend on lattice parameters, but only on the integer Miller indexes. This leads to an easily identifiable pattern. For the rest of cases, Equation (3) shows the complexity of the problem and the inability FIGURE 3 Classes distribution in the data set SILVA-RAMÍREZ ET AL.5of17 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License TABLE 1 An extract of data set Attributes Target class 1 0.894 0.748 0.639 0.599 0.573 0.555 0.501 0.498 0.459 0.445 0.445 0.428 0.415 0.413 0.407 0.385 0.382 0.373 7 1 0.863 0.691 0.640 0.540 0.507 0.476 0.448 0.443 0.432 0.410 0.367 0.363 0.358 0.346 0.340 0.322 0.317 0.313 6 1 0.707 0.632 0.500 0.471 0.447 0.426 0.392 0.378 0.354 0.343 0.333 0.324 0.316 0.301 0.283 0.277 0.272 0.263 4 1 0.760 0.667 0.613 0.555 0.496 0.432 0.411 0.392 0.369 0.349 0.329 0.308 0.297 0.285 0.273 0.264 0.255 0.245 13 6of17 SILVA-RAMÍREZ ET AL. 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License to obtain predictable patterns. Another additional consideration to take into account is that every Bravais lattice is associated with a subset of the 230 Space Groups of Schoenflies-Fedorov. For each group of symmetry certain operations leave a trace in the diffraction pattern systematically cancelling certain intensities, following rules called systematic absences. In Figure 4a and b, we show powder X-ray diffraction diagrams for compounds with two different space groups of the same Bravais lattice (primitive orthorhombic). The first is a symmorphic group (P222) whereas the second is Pnma which show systematic absences due to screw axes and glide planes (in fact Pnma is a supergroup of P212121). In both figures, horizontal axis is the scattering angle (2θ) and vertical axis is the intensity in arbitrary units. The small markers at the bottom indicate the Bragg positions. If we remark the Bragg peak positions, regardless its intensities, patterns are very different despite coming from the same Bravais lattice. Therefore, we can state we are facing up a formidable challenge without the hope of any predictable pattern. 3|COMPUTATIONAL TOOLS With regard to ANN we have used the multiple back propagation (MBP) software developed by Lopes and Ribeiro (2010). This program incorporates an architecture multiple feed forward network which allows a partition in sub-spaces for the mapping function relating to inputs and outputs. It also implements neurons with selective activation. Experimental results on benchmarks showed the superiority, in most cases, over classical multilayer perceptron networks (Lopes & Ribeiro, 2011). The other ML models were accessed through the graphical interface Weka (Frank et al., 2016). This software allows you to preprocess a big set of data. It also applies different ML algorithms and compares various outputs. Weka was developed by the University of Waikato in New Zealand, this term stands for Waikato Environment for Knowledge Analysis. The methods used in the experiments were: Naïve Bayes A probabilistic classifier applying Bayes theorem. k-NN A clustering supervised algorithm. Chirp Method based on Hypercube description regions (HDR). Furia A fuzzy rule learner. J48 A decision tree implementing the ID3 algorithm (iterative dichotomiser). Meta-estimators random tree, random forest and extra tree. 4|PRELIMINARY RESULTS Two kind of experiments were done, using different methods for validation in order to measure the ability to predict new data avoiding selection bias and overfitting: •Train/test split: data are shuffled randomly and a split 2/3 is used for training and the other split 1/3 for testing the model. •k-fold cross-validation (CV): the data are split into k folds, so the model is trained on k1 folds and then it is tested on the other fold. The process is repeated until each unique group has been used as the test set. Exhaustive performances were performed with a partition parameter equal to 10. FIGURE 4 Simulated X-ray diffraction pattern for two materials with the same Bravais lattice but different space group SILVA-RAMÍREZ ET AL.7of17 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License The results with the percentages of correctly classified records are shown in Table 2. The first column represents the applied model, the second the percentage of correctly classified instances using the HoldOut validation method versus the 10-fold Cross-Validation technique in the last column. We draw several conclusions: •Superiority of meta-estimators against ANN model for these kinds of problems. From now on we will no longer consider ANN methods. •While ML models are promising, however the test accuracy is not significant for both splitting and cross validation experiments. All of above leads us to make the decision of pre-process the data to overcome the imbalance between classes or overfitting. With such a goal in mind we have used two supervised filters already included in Weka: Smote and ClassBalancer. The first one is a synthetic minority oversampling method. On the other hand, the second one reweights the instances so that each class has the same total weight. After this we have applied again the ML models, the obtained results are described in Table 3, including the results of κ-statistic from 10-fold CV experiments accounting for the possibility of agreement by chance (Smeeton, 1985). In Figure 5a sketch of these results is shown. The x-axis represents the different studied models and the y-axis represents the accuracy (%) of the classification for both splitting and cross validation experiments. It is observed that most of the models present a higher accuracy when working with the 10-fold CV validation method. Except for the Naïve Bayes model, all models present an accuracy greater than 75%, highlighting the Random Tree model, which exceeds 95%. 5|REDUCTIONININPUTSPACE The following Section explores the selection of the more significant attributes and the reduction of the input data. We intend to determine if there is an improvement in the performance of the models when a reduction of the data set is carried out. 5.1 |Improvement by attribute selection For the process of classification, we selected 20 attributes which were eventually reduced to 19 because of the normalization task. Afterwards, we raised the question of its relevance since some of them might be irrelevant or superfluous. In fact, attribute selection is a mining data method TABLE 2 Results from preliminary experiments Models Train/test split 10-fold CV MBP 37.0 22.0 Naïve Bayes 14.1 13.6 k-NN 27.1 32.4 J48 39.5 37.3 Chirp 31.5 29.8 Furia 41.7 29.9 Random Tree 50.2 59.6 Extra Tree 50.3 59.5 Random Forest 50.0 61.3 TABLE 3 Results from experiments with preprocessed data Models Train/test split 10-fold CV k-statistic Naïve Bayes 28.6 32.6 0.19 k-NN 82.4 89.6 0.89 J48 79.3 85.7 0.79 Chirp 76.5 75.2 0.81 Furia 71.8 81.0 0.76 Random Tree 92.5 95.5 0.93 Extra Tree 90.0 83.5 0.90 Random Forest 84.7 90.6 0.92 8of17 SILVA-RAMÍREZ ET AL. 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License to extract the ranking of attributes. The goal is not only to reduce the processing time, but also to improve the metrics of the classification algorithms. The underlying philosophy is very well explained in the article of Gnanambal et al. (2018). Attribute selection is a two-step process, the first one is a searching process and the following one deals with ranking. The PART algorithm was first applied according the recommendations in Gnanambal et al. (2018). For the second step we applied the heuristic Greedy stepwise method (Butterworth et al., 2004). The results are shown in Table 4. We have highlighted those attributes that exceed 55% significance which it is clearly observed in Figure 6. The y-axis represents the percentage of significance, while the x-axis represents each of the attributes. The threshold value has been marked with a horizontal line. All those attributes that exceeded this value were selected. After this task we have repeated the classification based only on the highlighted attributes and the most successful algorithm (Random Tree). Thus, we improved our results from 95.5 to 95.9 in 10-fold CV. While this improvement seems slight, we showed nonetheless effectiveness of attribute selection. TABLE 4 Selected attributes according to acceptance criteria Attributes Significance (%) Acceptance 10 2 100 X 360 X 440 540 680 X 720 860 X 920 10 60 X 11 20 12 20 13 60 X 14 20 15 0 16 60 X 17 60 X 18 0 19 20 20 20 FIGURE 5 Graphic representation of performance from the experiments SILVA-RAMÍREZ ET AL.9of17 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Department of Computer Science and Engineering at the University of Cádiz. Her major research interests include Computational Intelligence, Data Processing and Machine Learning‐based techniques applied to different areas. Inmaculada Cumbrera‐Conde has a Bachelor of Business Administration and law degrees in Spain and Australia (University of Cadiz, Catholic University of Lyon, University of Technology, Sydney and University of Sydney). She is working towards a PhD with the Road to Research Scholarship ‐RTP and MRES Scholarship from Macquarie University. Her research interests include Project Management, AI and Machine Learning, and Private International Law. She is the Chair of Spanish Researchers Abroad in Australia‐Pacific. She has broad experience in law and business and is a Seasonal Teacher at Macquarie University. Rafael Cano‐Crespo received the degree in Physics in 2012, Master's degree in Science and Technology of New Materials in 2014 and PhD in Science and Technology of New Materials from the University of Sevilla in 2018. He is currently an Assistant Professor and researcher in the Department of Condensed Matter Physics of the Faculty of Physics at the University of Sevilla. His research interests include the processing, spark plasma sintering and mechanical behavior of advanced ceramic materials (alumina, zirconia and ceramic composites) using different experimental characterization techniques such as: scanning electron microscopy, transmission electron microscopy, creep at high temperatures, Raman spectroscopy, Vickers hardness, nanoindentation. Francisco-Luis Cumbrera received the degree in Physics from the University of Sevilla. He obtained the PhD in the same university working on the Crystallization of Metallic Glasses. He made a stay of four years in the Laboratoire de Microscopie Electronique in the University of Haute Normandie. He has made long stays in the CNRS laboratory of Bellevue (Paris) and also in the laboratory of Cristallographie et Mineralogie (Université Pierre et Marie Curie, Paris). He is currently Full Professor in the Department of Condensed Matter Physics in the University of Sevilla. His research interests include X‐rays, intermetallic compounds and advanced ceramics. How to cite this article: Silva-Ramírez, E.-L., Cumbrera-Conde, I., Cano-Crespo, R., & Cumbrera, F.-L. (2023). Machine learning techniques for the ab initio Bravais lattice determination. Expert Systems,40(2), e13160. https://doi.org/10.1111/exsy.13160 16 of 17 SILVA-RAMÍREZ ET AL. 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License APPENDIX A An extract of the rules set generated by Random Tree model is shown in this section: campo 13 < 0.29 jcampo 2 < 0.69 jjcampo 6 < 0.28 jjjcampo 6 < 0.18: 6 (2/0) jjjcampo 6 ≥0.18 jjjjcampo 6 < 0.27: 5 (6/0) jjjjcampo 6 ≥0.27: 4 (2/0) jjcampo 6 ≥0.28 jjjcampo 10 < 0.3 jjjjcampo 6 < 0.34 jjjjjcampo 2 < 0.52 jjjjjjcampo 13 < 0.22 jjjjjjjcampo 10 < 0.25: 13 (11/0) jjjjjjjcampo 10 ≥0.25 jjjjjjjjcampo 6 < 0.31: 11 (1/0) jjjjjjjjcampo 6 ≥0.31: 12 (1/0) jjjjjjcampo 13 ≥0.22 jjjjjjjcampo 2 < 0.5 jjjjjjjjcampo 14 < 0.23: 4 (3/0) jjjjjjjjcampo 14 ≥0.23 jjjjjjjjjcampo 8 < 0.31: 3 (3/0) jjjjjjjjjcampo 8 ≥0.31: 9 (2/0) jjjjjjjcampo 2 ≥0.5: 13 (6/0) jjjjjcampo 2 ≥0.52 jjjjjjcampo 8 < 0.31: 3 (18/0) jjjjjjcampo 8 ≥0.31: 13 (2/0) jjjjcampo 6 ≥0.34 jjjjjcampo 2 < 0.6 jjjjjjcampo 10 < 0.29: 13 (22/0) jjjjjjcampo 10 ≥0.29: 11 (1/0) jjjjjcampo 2 ≥0.6 jjjjjjcampo 3 < 0.51: 5 (3/0) jjjjjjcampo 3 ≥0.51: 3 (5/0) jjjcampo 10 ≥0.3 jjjjcampo 17 < 0.25 jjjjjcampo 3 < 0.57 jjjjjjcampo 8 < 0.32: 13 (2/0) jjjjjjcampo 8 ≥0.32 jjjjjjjcampo 2 < 0.65: 13 (1/0) jjjjjjjcampo 2 ≥0.65: 9 (4/0) jjjjjcampo 3 ≥0.57. jjjjjjcampo 10 < 0.31: 12 (4/0). jjjjjjcampo 10 ≥0.31: 4 (2/0). jjjjcampo 17 ≥0.25 jjjjjcampo 3 < 0.55 jjjjjjcampo 10 < 0.33: 7 (6/0) jjjjjjcampo 10 ≥0.33: 13 (3/0) jjjjjcampo 3 ≥0.55: 13 (7/0) SILVA-RAMÍREZ ET AL.17 of 17 14680394, 2023, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/exsy.13160 by Universidad De Sevilla, Wiley Online Library on [23/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License