Automatic Spectral Classification of Stars using Machine Learning: An Approach based on the use of Unbalanced Data

Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 DOI:10.5121/mlaij.2022.9401 1 AUTOMATIC SPECTRAL CLASSIFICATION OF STARS USING MACHINE LEARNING: AN APPROACH BASED ON THE USE OF UNBALANCED DATA Marco Oyarzo Huichaqueo1 and Renato Muñoz Orrego2 1School of Engineering, Rovira i Virgili University, 43007 Tarragona, Spain 2School of Engineering, Technical University of Madrid, 28006 Madrid, Spain ABSTRACT With the increase in astronomical surveys, astronomers are faced with the challenging task of analyzing a large amount of data in order to classify observed objects into hard-to-distinguish classes. This article presents a machine learning-based method for the automatic spectral classification of stars from the latest release of the SDSS database. We propose the combinatorial use of spectral data, derived stellar data, and calculated data to create patterns. Using these patterns as inputs, we develop a Random Forest model that outputs the spectral class of the observed star. Our model is able to classify data into six complex classes: A, F, G, K, M, and Carbon stars. Due to the unbalanced nature of the data, we train our model considering three data use cases: using the original data, using under-sampling, and over-sampling data techniques. We further test our model by using a fixed dataset and a stratified dataset. From this, we analyze the performance of our model through statistical metrics. The experimental results showed that the combinatorial use of data as an input pattern contributes to improve the prediction scores in all data use cases, meanwhile, the model trained with augmented data outperforms the other cases. Our results suggest that machine learning-based spectral classification of stars may be useful for astronomers. KEYWORDS Spectral Classification, Machine Learning, Data Analysis, Astronomy 1. INTRODUCTION Recently, due to the increase in scientific astronomical surveys, there is a need to quickly characterize the data obtained from the observations. Nevertheless, this task is carried out manually, due to the lack of confidence in more sophisticated algorithms, being computationally costly in terms of memory and processing time due to the use of large amounts of data and the long time-consuming analyzes performed by astronomers. For instance, astronomers must perform some challenging tasks such as classifying observed objects into main classes (e.g. galaxy or star) and then their subclasses (e.g. starforming, starburst, B9-type, M0V-type, and others) based on information such as their morphology, recorded spectral data, and calculated stellar parameters. In the particular case of star classification, astronomers must analyze the spectral data of the candidates. The spectral data is obtained by using instruments and filters to capture specific wavelengths such as ultraviolet (u), green (g), red (r), near-infrared (i), and infrared (z) [1]. These light bands provide useful information that allows a classification based on physical characteristics. In this way, by using stellar classification systems the observed candidates can be classified by human experts. A standard in astrophysics is the well-known Morgan-Keenan (MK) Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 2 classification system (Morgan et al., 1943) [2] that is based on both the luminosity and effective temperature of a star. The MK system divides stellar spectra into O, B, A, F, G, K, and M main classes; from the hottest (O-type: Teff > 30,000 K) to the coolest (M-type: 2,200-3,700 K) and then each letter class can be also subdivided using a numeric digit where 0 correspond to the hottest and 9 to the coolest. Nevertheless, depending on the nature of the candidates, the MK system does not present clear results in some cases. For instance, when the MK system results in irresolvable overlaps in terms of spectral types in both effective temperature and luminosity, some candidates can be classified into L-type [3] and methane dwarfs (T-type), by analyzing the infrared spectra or carbon types (type C) [4] by analyzing the swan bands of their spectra. On the other hand, from a practical point of view, an important consideration is that with the advancement of technology and computational techniques, an intelligent machine can do this challenging classification task (for a human) more efficiently. Even more when the use of artificial intelligence (AI) and its supervised methods, such as machine learning (ML) and deep learning (DL), have shown their potential in different fields of scientific and industrial application. In this work, a system for the automatic recognition of spectral classes of stars based on ML from spectroscopic and photometric data is proposed. Our proposed system is capable of recognizing six types of stars: A, F, G, K, M, and C-type. For this, we propose the use of the latest data collected up to 2021 from the Sloan Digital Sky Survey (SDSS) [5]. The main contributions of this work are as follows: • Demonstrating that by using a widely used ML algorithm, it is possible to reach acceptable prediction rates for a challenging task of classification star types under MK spectral class and non-conventional class. • Introducing an approach to improve the prediction rates from the combinatorial use of astronomical data. • Evaluating our method based on different data use cases and different types of test sets. This work is organized into five sections. Section 2 discusses related work on astronomical object classification. In Section 3, we introduce the methodology followed in this work. Section 4 covers the computational experimentation and results obtained. Finally, Section 5 provides research conclusions with some possible directions for future research. 2. RELATED WORK A review of the literature on astronomical objects classification methods using AI techniques was carried out. This review only includes recent articles (up to 4 years ago) and it is split into star classification and stellar object classification, as follows. 2.1. Star Classification Several works have proposed the use of AI techniques for the classification of stars based on the MK system. Sharma et al. (2020)[6] proposed an approach for the classification of stellar spectra into O-B-A-F-G-K-M type stars. In their work, they trained some ML and DL algorithms using different spectral libraries and tested them by using the Indo-U.S. Library of Coud´e Feed Stellar Spectra (CFLIB). Using a convolutional neural network (CNN) model, their best-resulting accuracy was 89%. In the work of Lu et al. (2020)[7], a fully connected artificial neural network (ANN) was proposed for binary classification of stars into F-G and G-K types from 2D spectral images. In their development, the Data Release 6 (DR6) from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) was used for training and testing. Thus, their method Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 3 achieved an accuracy of 80%. Similarly, Dafonte et al. (2020)[8] presented a DL method to classify spectral data into A-B-F-G-K-M type stars. To this, the authors proposed the use of an ANN trained and tested using their database. Therefore, their experimental results showed that their method achieved an accuracy of 83%. We further identify some works in which the Random Forest and XGBoost algorithms have recently been used for variable star classification. Zhang et al. (2020) [9] proposed a method to classify RR Lyrae candidates into A-type and F-type stars. They combined photometric and spectroscopic data obtained from SDSS achieving a completeness of ~97%. Adassuriya et al. (2021) [10] used stellar lights curves observed by the Kepler mission to train a classifier that achieved a sensitivity of 87% during the identification β-Cephei, δ-Scuti, γ-Doradus, Red Giants, RR Lyrae, and RV Tauri star types. In the work of Naydenkin et al. (2020) [11] a ML model to classify data of the Zwicky Transient Facility (ZTF) into β-Cephei, δ-Scuti, and RR Lyrae variable stars is proposed. Their method achieved a 95% of the area under the curve (AUC) metric. Similar works have been performed by Hosenie et al. (2019 & 2020) [12][13] where they proposed a system for classification into β-Cephei, δ-Scuti, and RR Lyrae variable stars with ~98% of accuracy using Catalina Real-time Transient Survey (CRTS) database. 2.2. Stellar Object Classification On the other hand, some authors proposed methods to identify stellar objects from astronomical data. Acharya et al. (2018) [14] presented ML algorithms to classify into star, quasar, and galaxy class from SDSS photometric data. Their best-resulting accuracy was 94.10% achieved using Random Forest. Wierzbinski et al. (2021) [15] used SDSS photometric data and principal component analysis (PCA) to develop some ML models to classify into star, quasar, and galaxy. Thus, Wierzbinski et al. reached an accuracy of 99.16% by using a voting classifier that contains estimators such as quadratic discriminant analysis (QDA), support vector machine (SVM), Decision Tree, Random Forest, XGBoost, Bagging classifier, multilayer perceptron (MLP), Extra Trees, and Naive Bayes classifier. Similarly, Martinazzo et al. (2020) [16] proposed an approach to classify data from Southern Photometric Local Universe Survey (S-PLUS) into star and galaxy using DL models. They demonstrated that using DenseNet-121 was possible to reach an accuracy of 99.2%. A comparison between classical ML models and DL models was proposed by Ethiraj & Kumar (2022) [17] using SDSS data to classify into star, quasar, and galaxy. In their work, the Extra Trees classifier reached an accuracy of 96%, meanwhile, DL models such as EfficientNetB2 and Xception achieved 91% of accuracy. This paper uses an approach similar to that proposed by Sharma et al., Lu et al., Dafonte et al., and Zhang et al. Particularly, in our case, we propose a system that considers a greater classification capacity, also including types of stars not considered in the MK system, such as Ctype stars. 3. MATERIALS AND METHOD 3.1. Proposed Architecture The proposed method in this work consists of the complementary use of spectroscopic and photometric data of observed stars to estimate their spectral class by using supervised learning. As shown in Fig. 1, our architecture includes a processing module and a supervised learning module. The function of the processing module is to prepare the data to be used in the next module, while the function of the supervised learning module is to analyze and classify the data. A summary of the methodology followed in this work is as follows. Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 4 Figure 1. Overview of the proposed spectral classification architecture In the processing module, the raw data from an astronomical survey is processed in order to obtain labeled data samples. To this end, we use data from SDSS, which contain astronomical data for stars previously manually classified by astronomers. Within the information provided by SDSS is the spectral subclass (e.g. A0, F2, and others) of the observed stars, but in our approach, we propose a system designed to estimate the main spectral class (e.g. A, F, and others). Therefore, we perform data processing to obtain the spectroscopic and photometric data of each observed star, thus obtaining labeled data that are denoted as samples for our purpose. We also propose to use the samples as patterns to allow the combinatorial use of the data obtained. The details of both data processing and pattern usage will be explained later in this article. In the supervised learning module, we propose the use of a widely used ML algorithm. In this way, using the patterns as inputs we train our ML model to classify the observed stars into A, F, G, K, M, and C-type. Thus, our system is able to identify which spectral class belongs to each input pattern. Furthermore, due to the unbalanced nature of the data from astronomical surveys, which significantly influences the predictive behaviour of our ML model, the training of the ML model is proposed using the real unbalanced data and balanced data through techniques that will be explained later in this article. Finally, in particular, the spectroscopic data correspond to measured u, g, r, i, and z light bands in a range of 3543-9134 Å. In the photometric data case, this information stands for derived stellar parameters such as redshift, effective temperature (Teff), surface gravity (log(g)), metallicity [Fe/H] of ELODIE star (Prugniel & Soubiran, 2001) [18]. In addition, all implementation of the proposed architecture was carried out using Python codes on a laptop workstation with a 1.6 GHz Intel® Core™ i5-10210U CPU and 8 GB of memory. The detail of each module is described below. 3.2. Experimental Data SDSS is a scientific project where the main aim is to map the universe and identify astronomical objects such as galaxies, quasars, and stars. Since 1998, SDSS has progressed through several phases that involve multiple surveys with interlocking science goals. In this work, we use the Data Release 17 (DR17) (Abdurro’uf et al., 2022) [19] of the fourth phase of the project (SDSS-IV) which contains observations through January 2021 considering a dual hemisphere view of the sky, observing from both Las Campanas Observatory, using the du Pont Telescope (Bowen & Vaughan, 1973) [20] and the Sloan Foundation 2.5m Telescope (Gunn et al. 2006) [21] at Apache Point Observatory. DR17 includes different open access types of data such as images, optical spectra, infrared spectra, integral field unit spectra, stellar library spectra, and catalog data. From our approach, we obtained both spectroscopic and photometric data through the science archive server (SAS) catalogues of the SDSS. In this way, all data is provided Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 5 by using the standard flexible image transport system (FITS) file format widely used in astronomy. 3.3. Data Processing From the data obtained, first we selected the best observations following some recommendations from the SDSS catalogue. These observations correspond to those that satisfy some conditions that are described below. The particular data was selected based on our ML approach. In this way, the data classified as stars by astronomers were selected using the “Type” condition that distinguishes stars based on their morphology. In addition, when the bitmask called “ZWARNING” is equal to zero indicates no problems were identified during the redshift determination. Furthermore, as criteria, only those primary observations of the objects were selected through both “SpecPrimary” and “Mode” conditions set to true for spectroscopic and photometric data respectively. The resulting processed data contains different spectral subclasses previously determined by astronomers that are also unevenly distributed. The data correspond to both u, g, r, i, and z light bands values and some stellar parameters such as Teff, log(g), [Fe/H], and redshift for each sample. For future ML classification, we cluster the data based on its main spectral class as shown in Table 1. Table 1. Distributions of the samples obtained from the SDSS database Original subclasses from SDSS Samples Class O, OB 1,957 O B6, B9 6,719 B A0, A0p 86,623 A F2, F5, F9 337,929 F G0, G2, G5 72,025 G K1, K3, K5, K7 150,543 K M0V, M2V, M1, M2, M3, M4, M5, M6, M7, M8 159,625 M L0, L1, L2, L3, L4, L5, L5.5, L9 4,643 L T2 290 T Carbon, Carbon_lines, CarbonWD, CV 37,315 C 3.4. Machine Learning Approach In our supervised learning approach, we propose the use of the spectral data combined with stellar parameters as input patterns to a meta-estimator based on the ensemble learning method. Therefore, we use the widely used Random Forest algorithm (Breiman, 2001) [22] set as 100trees in the forest and entropy function to measure the information gain of a split. In order to get a robust classifier, we also propose an interesting experimental focus based on both the complementary use of the fairly reddening-insensitive pseudocolor parameter, proposed by Cáceres & Catelan (2008) [23], and the statistical analysis of spectral data and stellar parameters in each sample. In this way, we trained the Random Forest classifier using an input pattern based on the combinatorial use of the data as described below and denoted in Table 2: • Pattern 1: Structured by the spectral data. • Pattern 2 to 5: Structured by adding the stellar parameters. Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 6 • Pattern 6: Structured by adding the fairly reddening-insensitive pseudocolor calculated value, denoted as: 𝐶0 = (𝑢 − 𝑔)0 − (𝑔 − 𝑟)0. • Pattern 7: Structured by adding the provided variance calculated from spectral data, denoted as: 𝑉𝑎𝑟1 = 𝑉𝑎𝑟(𝑢, 𝑔, 𝑟, 𝑖, 𝑧). • Pattern 8: Structured by adding the provided variance calculated from derived stellar parameters data, denoted as: 𝑉𝑎𝑟2 = 𝑉𝑎𝑟(𝑇𝑒𝑓𝑓, log(𝑔) , [𝐹𝑒⁄𝐻], 𝑟𝑒𝑑𝑠ℎ𝑖𝑓𝑡). Table 2. The proposed input pattern for supervised learning Pattern Structure of proposed input pattern 1 u, g, r, i, z 2 u, g, r, i, z, Teff 3 u, g, r, i, z, Teff, log(g) 4 u, g, r, i, z, Teff, log(g), [Fe/H] 5 u, g, r, i, z, Teff, log(g), [Fe/H], redshift 6 u, g, r, i, z, Teff, log(g), [Fe/H], redshift, C0 7 u, g, r, i, z, Teff, log(g), [Fe/H], redshift, C0, Var1 8 u, g, r, i, z, Teff, log(g), [Fe/H], redshift, C0, Var2 In this work, we are concerned that we have a severe class imbalance distribution that will make it difficult to obtain good prediction rates. To face this challenging issue, first, from the processed data shown in Table 1, we do not consider the O, B, L, and T-type classes due to their poor amount of samples, and then we propose the following experimental data use case: the use of the unbalanced original data distribution of the classes, and the use of the balanced data through under-sampling, and over-sampling data techniques. These three proposed cases are denoted as 1, 2, and 3, respectively in Fig. 2. In order to reduce the data, we consider the smallest class as the target. Thus, it is possible to undersample all classes greater than the smallest class. On the other hand, to augment the data the largest class was considered as the target. For this, we propose the use of the Synthetic Minority Over-sampling Technique (SMOTE, Chawla et al. 2002) [24] in order to simplify the issue and oversample the minority classes. Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 7 Figure 2. Data distribution for (A) fixed testing dataset and (B) stratified testing dataset Furthermore, to correctly evaluate our ML model we propose the use of two datasets for testing. Our testing approach is useful for testing our method with different datasets that have been created from randomly shuffled data. In both cases, a seed value was used to initialize the random number generator to ensure that the experiments are reproducible and the same in any case. First, we consider a fixed testing dataset containing the same number of samples per class. This amount is distributed in a balanced way for all classes and corresponds to 30% of the smallest class. In this way, it is possible to guarantee a static and formed testing dataset with original data. Secondly, we consider a stratified testing dataset, which contains the preserved proportion of samples per class. This proportion corresponds to 30% of the samples for each class. In this way, it is possible to guarantee a dynamic and formed testing dataset based on data that can be real or synthetic (e.g. oversampled data). For this purpose, the model_selection.train_test_split method of the scikitlearn library (available on the official website https://scikit-learn.org/stable/) was used. Fig. 2 shows in a simple way the data distribution for each spectral class and each data use case. 3.5. Statistic Metrics and Performance Evaluation In order to evaluate our ML model, we consider some statistic metrics by using some categorical labels obtained from a prediction task such as true positive (TP) and false positive (FP), true Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 8 negative (TN) and false negative (FN). Thus, we propose some widely used statistical evaluation metrics such as accuracy (acc.), precision (prec.), recall, F-score (F), and AUC for multiclass classification task (Sokolova & Lapalme, 2009) [25]. It is important to clarify that these metrics will be calculated using the mean value obtained for each class. As shown in Table 3, the focus of these metrics is to represent the classifier’s ability to both correctly predict and avoid false classification. We further consider some non-conventional statistical metrics in order to get a better performance evaluation. These metrics are described in the following equations. In our approach, to represent the level of agreement between the predictions obtained for our ML models, we propose the Cohen’s Kappa (K) score [26] as shown in Eq. (1), where Pa standing for the observed agreement ratio and Pe is the expected agreement. On the other hand, to indicate how close the prediction probability (p) is to the corresponding outcome (y) for N observations, we propose the use of the Log-loss (L-L) function [27] as denoted in Eq. (2). Table 3. Description of the statistic evaluation metrics 4. RESULTS AND DISCUSSION 4.1. Evaluation of Data use Cases As illustrated in Fig. 2, for the proposed unbalanced data use case, 844,060 original samples were used. On the other hand, for the balanced data use case, we used 223,890 samples obtained by using the under-sampling technique; meanwhile, 2,027,574 samples obtained by using the oversampling technique were used. The data distribution is as follows. For the unbalanced data case, when the fixed testing dataset was used the distribution was: 776,896 samples for training (A: 75,429 - F: 326,735 - G: 60,831 - K: 139,349 - M: 148,431 - C:26,121), and 67,164 samples for testing (11,194 samples per class). Likewise, when the Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 9 stratified testing dataset was used the distribution per class was: 590,842 samples for training (A: 60,636 - F: 236,550 - G: 50,418 - K: 105,380 - M: 111,738 - C: 26,120), and 253,218 samples for testing (A: 25,987 - F: 101,379 - G: 21,607 - K: 45,163 - M: 47,887 - C: 11,195). On the other hand, for the balanced data case from the use of the under-sampling technique, when the fixed testing dataset has used the distribution was: 156,726 samples for training (26,121 samples per class), and 67,164 samples for testing (11,194 samples per class). Likewise, when the stratified testing dataset has used the distribution per class was: 156,726 samples for training (26,121 samples per class), and 67,164 samples for testing (11,194 samples per class). For the balanced data case from the use of the over-sampling technique, when the fixed testing dataset has used the distribution was: 1,960,410 samples for training (326,735 samples per class), and 67,164 samples for testing (11,194 samples per class). Likewise, when the stratified testing dataset has used the distribution per class was: 1,419,300 samples for training (236,550 samples per class), and 608,273 samples for testing (101,379 samples per class). In this way, the Random Forest algorithm was trained using the patterns proposed in Table 2, and its resulting statistic metrics are presented in Table 4. The results obtained from our ML approach are as follows. We can see that the best performance (0.94 of specificity and 0.94 of sensitivity for top prediction) was reached for the model developed from the use of balanced data through over-sampling technique. This is to be expected as the stratified testing dataset inevitably contains synthetic data that is likely to be easier to predict than the real data. On the other hand, we can note that there is no significant improvement between the use of original data (0.88 of specificity and 0.73 of sensitivity for top prediction) and the use of balanced data through the under-sampling technique (0.86 of specificity and 0.80 of sensitivity for top prediction). 4.2. The Effect of the Combinatorial use of Astronomical Data The input pattern number seven outperforms the other input cases in all data use cases. Therefore, by analyzing the resulting metrics presented in Table 4 we can see how much each stellar parameter contributes to the predictive results of the trained model. In this way, we note that the complementary use of the derived stellar parameters helps to strengthen the information provided as input data to the Random Forest algorithm. On the other hand, the contribution of the combinatorial use of astronomical data is interesting. This is because a contribution from the stellar data was demonstrated, contrary to the Pearson correlation [28] analysis between spectral and stellar data presented in Fig. 3. For instance, we can see this in the decrease of the Log-loss rate when the model is trained using patterns that contain more astronomical information. We further can see that both the fairly reddeninginsensitive pseudocolor and the calculated variance from spectroscopic data contribute to slightly improving the prediction capacity of the model. In the same way, when using the calculated variance from stellar parameters the model did not show improvements. Machine Learning and Applications: An International Journal (MLAIJ) Vol.9, No.4, December 2022 16 AUTHORS Marco Oyarzo Huichaqueo holds a B.S. degree in Automation Engineering and Industrial Control from the Technological University of Chile Inacap, since 2015. In 2021, he received his M.Sc. degree in Computational Engineering and Mathematics from the Rovira i Virgili University of Catalonia. Currently he is a member of the Institute of Electrical and Electronics Engineers (IEEE) and affiliated to the International Federation of Automatic Control (IFAC). He has five years of industrial research and development experience and has also published some papers in international journals and conferences. His research interests include robotics and artificial intelligence applications, machine learning and deep learning methods, computer vision, and industrial automation. He can be contacted at email: [email protected]. Renato Muñoz Orrego holds a B.S. degree in Industrial Control and Instrumentation Engineering from the Federico Santa María Technical University, since 2016. He is currently a student of the M.Sc. degree in Robotics and Automation at the Technical University of Madrid, since 2021. He is currently carrying out his research thesis at the European Organization for Nuclear Research (CERN). His research interests include intelligent robotics, human-robot interaction, and environment perception using machine learning and computer vision. He can be contacted at email: [email protected].