A comprehensive bibliometric survey of micro-expression recognition system based on deep learning

Heliyon 10 (2024) e27392 Available online 4 March 2024 2405-8440/© 2024 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Review article A comprehensive bibliometric survey of micro-expression recognition system based on deep learning Adnan Ahmad a , Zhao Li a , ** , Sheeraz Iqbal b , * , Muhammad Aurangzeb c , Irfan Tariq a , Ayman Flah d , e , f , g , Vojtech Blazek h , Lukas Prokop h a Key Laboratory of Underwater Acoustic Signal Processing of Ministry of Education, School of Information Science and Engineering, Southeast University, Nanjing, 210096, China b Department of Electrical Engineering, University of Azad Jammu and Kashmir, Muzaffarabad, 13100, AJK, Pakistan c School of Electrical Engineering, Southeast University, Nanjing, 210096, China d College of Engineering, University of Business and Technology (UBT), Jeddah, 21448, Saudi Arabia e MEU Research Unit, Middle East University, Amman, Jordan f The Private Higher School of Applied Sciences and Technology of Gabes, University of Gabes, Gabes, Tunisia g National Engineering School of Gabes, University of Gabes, Gabes, 6029, Tunisia h ENET Centre, VSB—Technical University of Ostrava, Ostrava, Czech Republic ARTICLE INFO Keywords: Bibliometric analysis Micro expression Scopus Web of science ABSTRACT Micro-expressions (ME) are rapidly occurring expressions that reveal the true emotions that a human being is trying to hide, cover, or suppress. These expressions, which reveal a person’s actual feelings, have a broad spectrum of applications in public safety and clinical diagnosis. This study provides a comprehensive review of the area of ME recognition. A bibliometric and network analysis techniques is used to compile all the available literature related to ME recognition. A total of 735 publications from the Web of Science (WOS) and Scopus databases were evaluated from December 2012 to December 2022 using all relevant keywords. The first round of data screening produced some basic information, which was further extracted for citation, coupling, co-authorship, co-occurrence, bibliographic, and co-citation analysis. Additionally, a thematic and descriptive analysis was executed to investigate the content of prior research findings, and research techniques used in the literature. The year wise publications indicated that the published literature between 2012 and 2017 was relatively low but however by 2021, a nearly 24-fold increment made it to 154 publications. The three topmost productive journals and conferences included IEEE Transactions on Affective Computing (n =20 publications) followed by Neurocomputing (n =17) and Multimedia tools and applications (n =15). Zhao G was the most proficient author with 48 publications and the top influential country was China (620 publications). Publications by citations showed that each of the authors acquired citations ranging from 100 to 1225. While publications by organizations indicated that the University of Oulu had the most published papers (n =51). Deep learning, facial expression recognition, and emotion recognition were among the most frequently used terms. It has been discovered that ME research was primarily classified in the discipline of engineering, with more contribution from China and Malaysia comparatively. * Corresponding author. ** Corresponding author. E-mail addresses: [email protected] (S. Iqbal), [email protected] (M. Aurangzeb), [email protected] (A. Flah), [email protected] (V. Blazek), [email protected] (L. Prokop). Contents lists available at ScienceDirect Heliyon journal homepage: www.cell.com/heliyon https://doi.org/10.1016/j.heliyon.2024.e27392 Received 12 November 2023; Received in revised form 21 February 2024; Accepted 28 February 2024 Heliyon 10 (2024) e27392 2 1. Introduction Micro-expressions (ME) are transient, low-intensity facial expressions that occurs when people try to conceal their genuine feelings, either purposefully or subconsciously [1]. As a result, it is difficult to discern such genuine emotional information. One of the most significant concerns is that the duration of a ME is exceedingly short, lasting approximately 0.04s–0.2s [2], and other literatures have found that the duration of a ME is less than 0.33s and does not surpass 0.5s [3]. The rapid arrival and disappearance of ME makes tracking and identification more difficult. Similar to low-intensity ME, they only apply to a portion of facial expression action units [4]. As a result, even without formal instruction, the human eye can quickly recognize ME and most participants struggle to identify it. Many researchers have been working hard over the past decades to assist computers better interpret human facial ME and emotional engagement. Takalkar et al. [5], documented the literature about face detection and recognition, filtering, facial structure detection and selection, and classification. These methods were categorized into three types: appearance-based methods, motion-based methods, and deep learning-based methods. From which it can be observed that these techniques focused on appearance have garnered greater interest in the literature. These methods describe the dynamics of expression intensity as well as textural details like furrows and wrinkles. To characterize ME in appearance-based methods, a variety of representations have been used, such techniques include Gabor filters, 2D Gabor filter and Sparse Representation (2DGSR), Discriminant Tensor Subspace Analysis (DTSA) and Local Binary Patterns on Three Orthogonal Planes (LBPTOP) [6–12]. Several studies [11,13–16] presented LBPTOP variants to rapidly amplify ME recognition. Guo et al. [13], introduced Three Orthogonal Planes with Centralized Binary Patterns (CBP-TOP). In their work they only considered pixels with the highest weight among their neighbors. Huang et al. [17], proposed Spatio-Temporal Local Binary Patterns with Integral Projection. They employed integral projection to maintain facial image shape attributes and hence improve micro-expression discrimination. Huang et al. [14], went even further, proposing the Spatio-Temporal Completed Local Quantized Patterns (STCLQP). Instead of homogeneous patterns, they used orientation, sign, and magnitude components to construct appropriate pattern codes. The usage of these pattern codes can result in improved performance; however, it is based on the training dataset used to develop classifiers. LBPTOP was used by Zong et al. [15], to detect ME, the data set was preprocessed with Eulerian video magnification (EVM), which involved amplifying ME at low intensity. Liong et al. [16], statically selected different face areas based on the frequency of occurrence of Action Units region-of-interest (ROI-selective). According to Zong et al. [15], the traditional spatial division methods cannot guarantee such acceptable subregions since its grid size is fixed. While using the large subregions may result in noisy data that can interfere with spatiotemporal feature performance, while using small subregions results in a loss of valuable information. They developed a hierarchical division method in which face was divided into subregions of varied dimensions to address this issue. STLBP-IP was used for ME features detection [17]. ME samples was used to improve recognition of the three expressions namely "disgust", "happiness", and "surprise" to address the scarcity of labeled ME instances. A Singular Value Decomposition (SVD) approach was used to implement LBP (reps. LBP-TOP) in order to construct a macro-to-micro transformation model for the detection of face macro-expression (reps. ME) features while Coupled Metric Learning (CML) method was used to represent the common characteristics of ME samples and facial macro-information in their Hot Wheel Patterns (HWP) and HWP-TOP algorithms for modeling face macro and ME [18,19]. The results were then compared to cutting-edge methodologies and it was discovered that using macro-expression data to detect ME, did not result in a substantial boost in efficacy. In past few years, numerous articles have been published which discussed the motion-based approaches for measuring orientations of the facial component and non-rigid movement [20–24]. To compute the facial motion its adjustments among consecutive frames the usage of the brightness conservation idea, The Histogram of Oriented Optical Flow (HOOF) and its variants were commonly used in motion-based automated algorithms for ME detection. To detect movement changes in ME, Li et al. [24], developed the Histogram of Image Gradient Orientation on Three Orthogonal Planes (HIGO-TOP). Magnification on frames was used in their proposed approaches to make the feature more useful. From last few years to overcome these problems, deep learning (DL) based method has been propose which achieved high-level feature identification and pattern recognitions simultaneously, the proposed approaches were assessed utilizing a range of deep neural network (DNN) topologies, including the Convolutional Neutral Network (CNN) and the Recurrent Neural Network (RNN). For this instance, Kim et al. [25], employ DL based algorithms to recognize ME. In their proposed approach, CNN was used to identify spatial feature space on select video frames at different expression levels (onset, apex, offset). Further, RNN-derived Long short-term memory (LSTM) network was used to recognize time-dependent features in video sequences. Similarly, Peng et al. [26], came up with a new type of neural network called the "DTSCNN" which stands for Dual Temporal Scale Convolutional Neural Network for ME recognition. In their method of calculating the high-dimensional spatiotemporal space of features, they incorporated optical-flow sequences over various temporal scales into the DTSCNN. In addition, Wang et al. [27], and Reddy et al. [28], proposed an architecture of 3D-CNNs derived from video sequences for the purpose of convolutional and feature identification based on the 3D kernel. Takalkar et al. [29], recently presented a hybrid technique to recognizing ME based on handmade and deep features. The handmade features, which were based on the LBP-TOP, depicted the face’s spatio-temporal motions, whilst the deep features were derived using CNN. The process of recognition was performed in a “black box” using Deep Learning (DL) algorithms. Although the development of DL based algorithms and outstanding classifiers in ME identification [30,31], their dependability in ME detection remains a challenge due to the limited number of ME datasets. To address the issue of insufficient samples in the ME dataset, Zhi et al. [32], and Wang et al. [33], used transfer learning. They use the ME dataset to train their model. The model was then transferred and fine-tuned to recognize ME. The fundamental issue with Transfer learning-based methods is about performance which suffers as a result of noise such as brightness, misaligned face, and the relatively brief duration and subtle movement of ME [34]. A. Ahmad et al. Heliyon 10 (2024) e27392 3 The enormous size of the feature space is a significant drawback of appearance-based approaches. For instance, Huang et al. [14], used more than 23,000 features while the method that Wang et al. [11], developed, contained about 4425 features. Wang et al. [34], used Facial Action Coding System (FACS) [35] to establish 16 ROIs in order to solve this issue. Additionally, ROIs were also used for ME detection by Liu et al. [20], and Zong et al., [15]. They demonstrated that ME changes the appearance of only a small number of local and local subregions of the face. Studies have shown that a model’s ME efficiency can be increased by including ROI-based features. The pyramid of uniform local binary patterns was then used as the basis for a model developed by Abdallah et al., [36]. It is noteworthy that even after ROIs are used to identify the feature space of appearance-based methods, their effectiveness in differentiating ME is still limited when compared to deep learning-based methods and motion-based methods. We believe that this is due to confusion in the classification of identical ME, which is one of the most difficult challenges for all researchers in this field. In light of the previously discussed literature, there was a sudden offshoot in research and understanding in the ME recognition domain. As a result, the bibliometric analysis is conducted, which aids in a full understanding and grasp of this topic. Bibliometrics is described as a field of study in which scientists evaluate bibliographic data (i.e., published literature) using various statistical and mathematical techniques to discover significant trends and patterns [37–40]. Bibliometric studies assess and measure the influence and impact of publications. These researchers commonly seek out patterns in the production, dissemination, and reception of information, alongside examining the relative significance and impact of diverse bibliographic elements such as citations, authors, institutions, and journals, among other factors [37,38]. A comprehensive and unbiased evaluation of the status of a specific field is a fundamental objective of bibliometrics research. This objective is accomplished by quantifying the volume and caliber of relevant publications, as well as the frequency with which they are acknowledged and utilized by other scholars. Such assessments can be utilized by researchers, policymakers, and funding organizations to identify areas of excellence and areas in need of improvement, enabling them to make well-informed decisions regarding the allocation of resources and support. Additionally, bibliometrics research can be used to identify new trends and potential areas of expansion in a particular field by examining the shift in publishing and citation practices over time. This can help researchers uncover new prospects and subjects while also staying up to date on developments in their respective fields. Bibliometric studies can also be used to examine the reach and influence of specific publications, authors, and institutions. These researches track the number of citations, the journals in which the publications were published, and the nations and institutions that cite the papers. Such evaluations can help researchers evaluate their own work while also uncovering prospective collaborations and networking opportunities with other scholars. Furthermore, corporations can use this type of analysis to make educated decisions and assess the effectiveness of specific organizations, countries, and publications. Some of the prominent software programs used by researchers to undertake bibliometric analysis include VOSviewer [41], BibExcel [42], CiteSpace [43], and Bibliometrix [44]. Thus, providing data on an open access portal for bibliometric analysis is a solid practice for supporting new researchers. It is imperative for new researchers to remain abreast of developments in their field by providing pertinent analyses, evidence-based descriptions, and precise representations derived from data obtained from the Scopus and Web of Science databases [45]. The purpose of this analysis is to track the evolution of research trends in ME overtime. This includes the identification of emerging topics, shifts in focus, changes in methodologies. Analyzing bibliometric data can reveal the most prominent journals, conference, authors or another ME research is disseminated. This information can be valuable for researchers looking to target specific topic in ME recognitions. This study’s findings can be utilized to identify current trend and hot topics of published articles for further study and research. This also provide in-depth understanding of a research topic by inducing and mapping authors’ regions. To gain research knowledge, it is necessary to be aware of the existing constraints [46]. This helps discovering new trends as well as exposing the future research, which is the fundamental motivation for this study. To the best of our knowledge, no previous work evaluated the extensive bibliometric literature evaluation and software-based thematic analysis on micro-expression recognition. Furthermore, this study presented a framework for thematic analysis that most researchers and industrial experts can employ in future research. The goal of this study was to provide answers to the following questions. •What are the research hotspots for ME recognition in the existing literature? •Which countries, ME-related articles, authors, and/or journals have achieved remarkable results? •What are the key topics and advancements in the ME recognition research? •What are the limitations and research priorities in consideration of ME recognition? In order to carry out an in-depth bibliometrics literature review, R-based tool was used in this study as well as the VOSViewer software. As a result, this study was attempted to improve the following areas of the literature now in existence. •Our work has established a thematic framework through the utilization of software-based thematic analysis, which has facilitated the investigation of main research hotspot. •In this study, we additionally classified the preceding research investigations based on the employed methodology. The result provided an insight into current research approaches as well as approaches that have been overlooked. The article is structured as follows, Section 2 outlines the methodology, data sources, and tools used in the bibliometrics analysis research. Section 3 presents the results and observations, which provides a complete bibliometric study in terms of performance analysis and science mapping. Section 4 presents some limitations of the study. Section 5 presents the conclusions of the research and summarizes its findings. A. Ahmad et al. Heliyon 10 (2024) e27392 4 2. Materials and methods 2.1. Research methodology The goal of this study was to perform a bibliometric analysis about ME recognition. The proposed study evaluated published articles by extracting quantitative information from graphical data using a range of statistical techniques (VOSviewer, an R-based tool). Data was gathered using various search engines and analyzed to estimate the number of published articles each year as well as relevant study subjects. It also included information about prominent authors, countries, organizations, and publications in the field of ME recognition. Furthermore, depending on the statistics, the most influential subject areas and articles were highlighted. As a result, our planned bibliometric study has included works on micro facial expression and recognition since the commencement of this survey. 2.1.1. Data collection In order to obtain bibliometric data, search terms were used in the WOS and Scopus databases on Jan 10, 2023, and the retrieved results were further ranked based on the title, abstract, and other inclusion/exclusion criteria suggested by various investigators. 2.1.2. Identification of search terms To discover the relevant terms, several publications from the previous literature were reviewed. To locate relevant phrases, Google Scholar was also searched for "micro-expression" and "facial micro-expression." The keywords "facial microexpression" OR "microexpression" OR "micro expression" OR "facial-microexpression ". A preliminary search was then conducted using the title, abstract and keyword of the publication. This resulted in the detection of 782 articles in Scopus database while the same keywords were also used in WOS and a total of 428 publications were retrieved. The search period was then limited to 2012-22 with only included Englishlanguage and peer-reviewed open access works and the final results obtained from Scopus were 712 publications while that of WoS were 404 publications. The search terms were then employed to acquire research and review articles from both WOS and Scopus databases as of January 10th, 2023. 2.1.3. Article selection The selection of an appropriate database is critical in conducting efficient searches. We made considerable use of the Scopus and WOS data bases in this respect. Scopus, in particular, gives academics greater visibility than other databases, and its sophisticated search functionality tools allow them to organize references and collect citations from papers. Furthermore, the Scopus database includes journals from a wide range of publishers, including Springer, Elsevier, Taylor & Francis, Emerald Insight, and IEEE [43,47]. In addition, WOS was used to fill in the missing articles and journals in Scopus. The first step was to search both databases for relevant Fig. 1. Article selection procedure used in this study. A. Ahmad et al. Heliyon 10 (2024) e27392 5 phrases. Abstracts, titles, and keywords yielded 712 total results for Scopus and 404 for WOS. Fig. 1 depicted the article segment process employed in this investigation. 712 Scopus articles and 404 WOS articles were extracted in two different formats. After combining the two datafiles, R-studio was used to locate and remove identical articles. As a result, 735 articles in Comma Separated Values (CSV) format were exported for bibliometric analysis. 3. Results and discussion 3.1. Bibliometric analysis This section addresses the bibliometric analysis of 735 selected publications’ author keywords, author partnerships, journals, citations, institutions, theme evaluation, and bibliographic coupling. 3.1.1. Publication by year Fig. 2 depicted a significant increase in the publication of research articles over the last ten years, indicating increased interest in micro-expression recognition in academic communities. Fig. 2 showed that the published literature was relatively low between 2012 and 2017. In 2012, there were only 12 published papers; by 2021, a nearly 24-fold increase made it to 154 publications. We anticipated that the publication of micro-expression recognition will grow exponentially in 2023 and subsequent years. 3.1.2. Publication by journals, authors and countries Fig. 3 showed the top five journals, it is imperative to consider the cumulative quantity of research papers that have been published within the past decade. The three most productive journals and conferences, as shown in Fig. 3a, were IEEE Transactions on Affective Computing (20 papers), Neurocomputing (17 papers), Multimedia tools and applications (15 papers). It was observed that the top three journals together published 41% of the overall literature. Fig. 3b illustrated the top four authors with the most articles published between 2012 and 2022. In addition, the figure revealed that Zhao G was the most proficient author (48 publications) followed by Wang S and NA N with 47 and 37 publications. Likewise, the top influential country as shown in Fig. 3c was China (620 publications) followed by Malaysia (91), and Finland (60) in terms of scientific production. 3.1.3. Publications by citations The present study evaluated the aggregate number of citations while gathering data and insights on noteworthy authors in the domain of micro-expression recognition. The top ten most frequently cited articles from the Scopus and WOS databases were summarized in Fig. 4. Fig. 4 illustrated that each of the authors acquired citations ranging from 100 to 1225. It is important to note that due to variations in indexing algorithms and time periods employed by different databases, the total number of citations obtained from Google Scholar and other databases may differ. According to Fig. 4, the author Zhao Guoyang’s works obtained the most citations (1225). She is currently a full Professor at the University of Oulu’s Center for Machine Vision and Signal Analysis (IAPR Fellow, IEEE Senior Member). The documents of author Fu Xiaolan come next, with a total of 1030 citations. She is a Professor of Psychology and the Fig. 2. Publication of micro expression recognition per year. A. Ahmad et al. Heliyon 10 (2024) e27392 6 Director of the Chinese Academy of Sciences’ Institute of Psychology. She is a pioneer in China’s micro-expression study. She has created three open-access micro-expression databases: CASME, CASME II, and CAS (MEU2). Her research interests include lie detection, emotional computing, learning, perception and attention, and perception and attention. Another author with a high number of citations is Wang S. His work has been cited 1006 times, and he is now an Assistant Researcher at the Institute of Psychology of the Chinese Academy of Sciences. He is the author of approximately 40 scholarly papers. At the 2011 International Joint Conference on Biometrics, he is among the ten doctoral consortium participants. Therefore, it is acceptable to believe that all the publications depicted in Fig. 4 were among the most well-known studies regarding recognition of micro-expressions in literature. 3.1.4. Publication by institutions Fig. 5 depicted that the University of Oulu has the most published papers (51), among the top nine the top nine academic institutions. The Institute of Psychology (the Chinese Academy of Sciences) was revealed to be the second ranked with total of 47 publications while the Multimedia University (Malaysia) was ranked third with 39 number of publications. According to the analyzed results, seven of the top ten most productive institutions were from China and actively pursued research into micro-expression recognition. 3.1.5. Common words in title and abstract Fig. 6 illustrated the Common words used in abstracts and titles of all the publications. The most common term used was microexpressions with 208 times in title-abstracts, followed by facial expressions (84 times), face recognition (74 times), and deep learning (73 times). 3.1.6. Common words in keywords The words most commonly employed by researchers in the keywords section were also assessed, given their tendency to convey the central focus of the research. Fig. 7a depicted that deep learning, facial expression recognition, and emotion recognition were among the most frequently used terms. The substantial and conspicuous font visibility of the keywords is apparent, as demonstrated by their large and bold font. Fig. 7b illustrated the corresponding analysis and clustering dendrogram for the aforementioned keywords. A dendrogram is a diagram that depicted the word’s hierarchical relationship. A dendrogram is primarily used to determine the best way Fig. 3. (A) Publication by Journals, (B) Most influential author based on total publications, (C) productive countries. Fig. 4. Publication by citations. A. Ahmad et al. Heliyon 10 (2024) e27392 7 to assign words to clusters. Fig. 7b depicted the hierarchical clustering of word observations using a dendrogram. Focusing on the height at which any two words were connected together was essential for dendrogram comprehension. Fig. 7b showed that micro expression and facial micro expression recognition were more similar in the brown cluster, with the shortest link connecting them. The height of the blue and green clusters, on the other hand, was greater, indicating the difference between the words. 3.1.7. Network analysis This bibliometrics study was conducted using Bibliometrix (R-based software for mapping literature), for quantitative bibliometric and scientometric research, the bibliometrix R-package (http://www.bibliometrix.org) offers a number of tools. It is written in R, an open-source language, environment, and ecosystem. The best reasons to choose R over other languages for scientific computing may be found in its extensive, powerful statistical algorithms, easy access to excellent numerical routines, and integrated data visualization tools. Similarly, VOSViewer [41] is a visualization tool specifically designed for constructing and visualizing bibliometric networks. It is widely used in the academic and research community for analyzing relationships between scholarly entities such as authors, keywords, and publications. The mainstream of the data analysis was conducted with the help of Bibliometrics, based on Bibliographic Cooccurrence Analysis (BCA), Bibliographic Coupling, Co-Authorship, Quotation, and Co-Citation Analysis. To analyze the data and report the results, the software manuals of Guleria and Kaur [48] were used. In network visualizations of scientific data, various entities such as authors, keywords, nations, and organizations have been depicted by emphasizing prominent nodes [49]. It is a rare occurrence to observe two entities, such as authors and publishing, being portrayed in a single map. The size of the node (circle) in network mapping represents the measured value of these elements. The significance of the node is directly proportional to its numerical value, which primarily indicates the number of citations or occurrences in an article. The linkages (edges) that connect the nodes reflect their relationship. The strength (weight) value of connections determines the association between two nodes, with a higher total link strength (TLS) value indicating a stronger association. Consequently, if a node represents the frequency of appearance of an article, the links between nodes represent the number of references exchanged. Therefore, as the number of nodes increases, the TLS value (weight) also increases. Additionally, the color and position of nodes in a network map serve as relevant indicators. When two articles (nodes) are in close proximity, they appear linked and share a greater number of references. The usage of the same color signifies that the articles belong to the same category. 3.1.8. Co-occurrence analysis (documents) The keyword sections of the author, for example, provide information in a distinctive manner. The term "co-occurrence" refers to how frequently a term appeared in particular publications [50]. The strength of a word’s overall length determines how many times it appeared in a text. The size of the node is directly proportional to the frequency of the words; for instance, a larger node size indicates a higher prevalence of the term. A thicker line linking two or more terms indicates their proximity to a particular cluster. To better grasp the intellectual terms cited by diverse professors, co-occurrence analysis was used. In total, 1410 author keywords and 1989 index keywords were discovered. Table 1 displays the TLS value of the ten top authors and index keywords. "Micro-expression recognition, micro-expression," and "deep learning" were identified as the most essential terms in both groups. Fig. 8a depicted the network visualization of all keywords. Micro-expression recognition, micro-expressions, and deep learning were the most prominent nodes. Keywords were also investigated between the years of 2012 and October 2022. The overlay visualization of all keywords is shown in Fig. 8b. A close examination of the graph revealed that the keywords used most frequently in late 2022 were expression recognition, convolutional neural network, micro-expression analysis, task analysis, and feature extractions. Fig. 5. Top nine institutions numbers of publication. A. Ahmad et al. Heliyon 10 (2024) e27392 8 Fig. 6. Tree-map of the most commonly used words in ME recognition publications’ titles and abstracts. A. Ahmad et al. Heliyon 10 (2024) e27392 9 Fig. 7. (a). Word cloud for the most relevant keywords in micro expression recognition publications (b). Dendrogram of Authors keywords. Table 1 The outcomes derived from the co-occurrence analysis conducted on author and index keywords. Author Keyword TLS Index Keyword TLS Micro-expression recognition 203 Micro-expression 1128 Deep learning 174 Face recognition 445 Micro-expression 169 Facial expressions 434 Optical flow 125 Deep learning 381 Emotion recognition 84 CNN 282 Features Extraction 81 Computer vision 203 Recognition 65 Classification 178 Affective computing 33 Learning system 181 Transfer learning 49 Image enhancement 55 Task analysis 46 semantics 47 A. Ahmad et al. Heliyon 10 (2024) e27392 16 Fig. 12. References based Co-citation analysis. A. Ahmad et al. Heliyon 10 (2024) e27392 17 available in databases. Additionally, the use of citation networks as a gauge of effect and quality may be subjected to criticism. Through the comprehensive and systematic inclusion of all document kinds and languages, the comparative utilization of various databases, and the use of bibliometric mapping tools, these constraints could serve as further research fields for bibliometric analysis on similar studies. 5. Conclusion This research endeavor constituted a comprehensive examination of published articles pertaining to the domain of micro expression recognition. It encompassed fundamental data derived from antecedent literature, encompassing the quantity of articles, yearly publications, and topical domains, and scrutinized the most impactful authors, articles, journals, and countries through the utilization of VOSviewer and R-based tools. The proposed study revealed that the majority of articles were about micro expression and facial micro-expression recognition. In terms of the subject domain, the research hotspots were micro-expression recognition and deep learning, followed by a core application of micro-expression recognition (optical flow, emotion recognition, features extractions, affective computing, transfer learning). Furthermore, the author’s keywords included the fundamental terms of ME recognition, whereas the index keywords included broad terms. When it comes to journals, IEEE Transactions on Affective Computing has published the most articles as compared to the other journals. In terms of countries, China published the most articles and was also leading in terms of authorship and co-citation analysis using TLS and citation values. It was also noticed that majority of the authors and institutions were from China and Malaysia, with micro expression recognition being the leading research area. In future work, an algorithm which take less training time when train on micro expression datasets can be developed. Secondly, the micro expression dataset is highly imbalanced so over coming this problem an algorithm can be improved to overcome this issue. CRediT authorship contribution statement Adnan Ahmad: Writing – original draft, Software, Conceptualization. Zhao Li: Investigation, Formal analysis. Sheeraz Iqbal: Writing – review & editing, Project administration, Investigation. Muhammad Aurangzeb: Formal analysis, Data curation. Irfan Tariq: Visualization, Validation. Ayman Flah: Investigation, Funding acquisition, Data curation. Vojtech Blazek: Visualization, Validation. Lucas Prokop: Investigation, Formal analysis. Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:This work Aknowldgment: This paper was supported by the following project TN02000025 National Centre for Energy II. The authors declare there is no conflict of interest. Acknowledgment This article has been produced with the financial support of the European Union under the REFRESH – Research Excellence For Region Sustainability and High-tech Industries project number CZ.10.03.01/00/22_003/0000048 via the Operational Programme Just Transition and paper was supported by the following project TN02000025 National Centre for Energy II. References [1] W. Zheng, S. Lu, Z. Cai, R. Wang, L. Wang, L. Yin, PAL-BERT: an improved question answering model, Comput. Model. Eng. Sci. (2023), https://doi.org/ 10.32604/cmes.2023.046692. [2] X. Liu, G. Zhou, M. Kong, Z. Yin, X. Li, L. Yin, W. Zheng, Developing multilabelled corpus of twitter short texts: a semi-automatic method, Systems 11 (8) (2023) 390, https://doi.org/10.3390/systems11080390. [3] W.-J. Yan, Q. Wu, J. Liang, Y.-H. Chen, X. Fu, How fast are the leaked facial expressions: the duration of micro-expressions, J. Nonverbal Behav. 37 (2013) 217–230. [4] S. Porter, L. Ten Brinke, Reading between the lies: identifying concealed and falsified emotions in universal facial expressions, Psychol. Sci. 19 (5) (2008) 508–514. [5] M. Takalkar, M. Xu, Q. Wu, Z. Chaczko, A survey: facial micro-expression recognition, Multimed. Tool. Appl. 77 (15) (2018) 19301–19325. [6] P. Zhang, X. Ben, R. Yan, C. Wu, C. Guo, Micro-expression recognition system, Optik 127 (3) (2016) 1395–1400. [7] S.-J. Wang, H.-L. Chen, W.-J. Yan, Y.-H. Chen, X. Fu, Face recognition and micro-expression recognition based on discriminant tensor subspace analysis plus extreme learning machine, Neural Process. Lett. 39 (2014) 25–43. [8] S.-J. Wang, W.-J. Yan, X. Li, G. Zhao, X. Fu, Micro-expression recognition using dynamic textures on tensor independent color space, in: 2014 22nd International Conference on Pattern Recognition, IEEE, 2014, pp. 4678–4683. [9] W.-J. Yan, et al., Casme II: an improved spontaneous micro-expression database and the baseline evaluation, PLoS One 9 (1) (2014) e86041. [10] Y. Wang, J. See, R.C.-W. Phan, Y.-H. Oh, Lbp with six intersection points: reducing redundant information in lbp-top for micro-expression recognition, in: Computer Vision–ACCV 2014: 12th Asian Conference on Computer Vision vol. 12, Springer, Singapore, Singapore, 2015, pp. 525–537. November 1-5, 2014, Revised Selected Papers, Part I. [11] Y. Wang, et al., Effective recognition of facial micro-expressions with video motion magnification, Multimed. Tool. Appl. 76 (2017) 21665–21690. [12] F. Qu, S.-J. Wang, W.-J. Yan, H. Li, S. Wu, X. Fu, Cas (ME) $^ 2$: a database for spontaneous macro-expression and micro-expression spotting and recognition, IEEE Transactions on Affective Computing 9 (4) (2017) 424–436. [13] Y. Guo, C. Xue, Y. Wang, M. Yu, Micro-expression recognition based on CBP-TOP feature with ELM, Optik 126 (23) (2015) 4446–4451. [14] X. Huang, G. Zhao, X. Hong, W. Zheng, M. Pietik¨ ainen, Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns, Neurocomputing 175 (2016) 564–578. A. Ahmad et al. Heliyon 10 (2024) e27392 18 [15] Y. Zong, X. Huang, W. Zheng, Z. Cui, G. Zhao, Learning from hierarchical spatiotemporal descriptors for micro-expression recognition, IEEE Trans. Multimed. 20 (11) (2018) 3160–3172. [16] S.-T. Liong, J. See, R.C.-W. Phan, K. Wong, S.-W. Tan, Hybrid facial regions extraction for micro-expression recognition system, Journal of Signal Processing Systems 90 (2018) 601–617. [17] X. Liu, S. Wang, S. Lu, Z. Yin, X. Li, L. Yin, W. Zheng, Adapting feature selection algorithms for the classification of Chinese texts, Systems 11 (9) (2023) 483, https://doi.org/10.3390/systems11090483. [18] X. Jia, X. Ben, H. Yuan, K. Kpalma, W. Meng, Macro-to-micro transformation model for micro-expression recognition, Journal of computational science 25 (2018) 289–297. [19] X. Ben, X. Jia, R. Yan, X. Zhang, W. Meng, Learning effective binary descriptors for micro-expression recognition transferred by macro-information, Pattern Recogn. Lett. 107 (2018) 50–58. [20] Y.-J. Liu, J.-K. Zhang, W.-J. Yan, S.-J. Wang, G. Zhao, X. Fu, A main directional mean optical flow feature for spontaneous micro-expression recognition, IEEE Transactions on Affective Computing 7 (4) (2015) 299–310. [21] F. Xu, J. Zhang, J.Z. Wang, Microexpression identification and categorization using a facial dynamics map, IEEE Transactions on Affective Computing 8 (2) (2017) 254–267. [22] H. Lu, K. Kpalma, J. Ronsin, Motion descriptors for micro-expression recognition, Signal Process. Image Commun. 67 (2018) 108–117. [23] S.-T. Liong, J. See, K. Wong, R.C.-W. Phan, Less is more: micro-expression recognition from video using apex frame, Signal Process. Image Commun. 62 (2018) 82–92. [24] X. Li, et al., Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods, IEEE transactions on affective computing 9 (4) (2017) 563–577. [25] D.H. Kim, W.J. Baddar, Y.M. Ro, Micro-expression recognition with expression-state constrained spatio-temporal feature representations, in: Proceedings of the 24th ACM International Conference on Multimedia, 2016, pp. 382–386. [26] M. Peng, C. Wang, T. Chen, G. Liu, X. Fu, Dual temporal scale convolutional neural network for micro-expression recognition, Front. Psychol. 8 (2017) 1745. [27] S.-J. Wang, et al., Micro-expression recognition with small sample size by transferring long-term convolutional neural network, Neurocomputing 312 (2018) 251–262. [28] S.P.T. Reddy, S.T. Karri, S.R. Dubey, S. Mukherjee, Spontaneous facial micro-expression recognition using 3D spatiotemporal convolutional neural networks, in: 2019 International Joint Conference on Neural Networks (IJCNN), IEEE, 2019, pp. 1–8. [29] M.A. Takalkar, M. Xu, Z. Chaczko, Manifold feature integration for micro-expression recognition, Multimed. Syst. 26 (2020) 535–551. [30] Y. Huang, Y. Yan, S. Chen, H. Wang, Expression-targeted feature learning for effective facial expression recognition, J. Vis. Commun. Image Represent. 55 (2018) 677–687. [31] Z. Yu, G. Liu, Q. Liu, J. Deng, Spatio-temporal convolutional features with nested LSTM for facial expression recognition, Neurocomputing 317 (2018) 50–57. [32] R. Zhi, H. Xu, M. Wan, T. Li, Combining 3D convolutional neural networks with transfer learning by supervised pre-training for facial micro-expression recognition, IEICE Trans. Info Syst. 102 (5) (2019) 1054–1064. [33] C. Wang, M. Peng, T. Bi, T. Chen, Micro-attention for micro-expression recognition, Neurocomputing 410 (2020) 354–362. [34] S.-J. Wang, et al., Micro-expression recognition using color spaces, IEEE Trans. Image Process. 24 (12) (2015) 6034–6047. [35] P. Ekman, W.V. Friesen, Facial Action Coding System, Environmental Psychology & Nonverbal Behavior, 1978. [36] T. Ben Abdallah, R. Guermazi, M. Hammami, Towards micro-expression recognition through pyramid of uniform temporal local binary pattern features, in: International Conference on Intelligent Systems Design and Applications, Springer, 2018, pp. 629–640. [37] N. Donthu, S. Kumar, D. Mukherjee, N. Pandey, W.M. Lim, How to conduct a bibliometric analysis: an overview and guidelines, J. Bus. Res. 133 (2021) 285–296. [38] N. Donthu, W. Reinartz, S. Kumar, D. Pattnaik, A retrospective review of the first 35 years of the international journal of research in marketing, Int. J. Res. Market. 38 (1) (2021) 232–269. [39] I. Danvila-del-Valle, C. Est´ evez-Mendoza, F.J. Lara, Human resources training: a bibliometric analysis, J. Bus. Res. 101 (2019) 627–636. [40] B. Moyle, C.-l. Moyle, L. Ruhanen, D. Weaver, A. Hadinejad, Are we really progressing sustainable tourism research? A bibliometric analysis, J. Sustain. Tourism 29 (1) (2020) 106–122. [41] N.J. Van Eck, L. Waltman, VOS: a new method for visualizing similarities between objects, in: Advances in Data Analysis: Proceedings of the 30th Annual Conference of the Gesellschaft für Klassifikation eV, Freie Universit¨ at Berlin, Springer, 2007, pp. 299–306. March 8–10, 2006. [42] O. Persson, R. Danell, J.W. Schneider, How to use Bibexcel for various types of bibliometric analysis, Celebrating scholarly communication studies: A Festschrift for Olle Persson at his 60th Birthday 5 (2009) 9–24. [43] N. Maflahi, M. Thelwall, When are readership counts as useful as citation counts? S copus versus M endeley for LIS journals, Journal of the Association for information Science and Technology 67 (1) (2016) 191–199. [44] C. Chen, CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature, J. Am. Soc. Inf. Sci. Technol. 57 (3) (2006) 359–377. [45] P. Tupe-Waghmare, R.R. Joshi, Bibliometric Survey on Multipurpose Face Recognition System Using Deep Learning, vol. 2021, Library Philosophy and Practice, 2021. [46] M.-L. Tseng, T.P.T. Tran, H.M. Ha, T.-D. Bui, M.K. Lim, Sustainable industrial and operation engineering trends and challenges toward Industry 4.0: a data driven analysis, Journal of Industrial and Production Engineering 38 (8) (2021) 581–598. [47] Z. Liu, C. Wen, Z. Su, S. Liu, J. Sun, W. Kong, Z. Yang, Emotion-semantic-aware dual contrastive learning for epistemic emotion identification of learnergenerated reviews in MOOCs, IEEE Transact. Neural Networks Learn. Syst. (2023), https://doi.org/10.1109/TNNLS.2023.3294636. [48] D. Guleria, G. Kaur, Bibliometric analysis of ecopreneurship using VOSviewer and RStudio Bibliometrix, 1989–2019, Libr. Hi Technol. 39 (4) (2021) 1001–1024. [49] M. Akinlolu, T.C. Haupt, D.J. Edwards, F. Simpeh, A bibliometric review of the status and emerging research trends in construction safety management technologies, International Journal of Construction Management 22 (14) (2022) 2699–2711. [50] M. Callon, J.-P. Courtial, W.A. Turner, S. Bauin, From translations to problematic networks: an introduction to co-word analysis, Soc. Sci. Inf. 22 (2) (1983) 191–235. [51] F.J. Acedo, C. Barroso, C. Casanueva, J.L. Gal´ an, Co-authorship in management and organizational studies: an empirical and network analysis, J. Manag. Stud. 43 (5) (2006) 957–983. [52] L. Cisneros, M. Ibanescu, C. Keen, O. Lobato-Calleros, J. Niebla-Zatarain, Bibliometric study of family business succession between 1939 and 2017: mapping and analyzing authors’ networks, Scientometrics 117 (2018) 919–951. [53] D. Crane, Social structure in a group of scientists: a test of the “invisible college” hypothesis, in: Social Networks, Elsevier, 1977, pp. 161–178. [54] W. Merghani, A. Davison, M. Yap, Facial micro-expressions grand challenge 2018: evaluating spatio-temporal features for classification of objective classes, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), IEEE, 2018, pp. 662–666. [55] M.E. Kemeny, et al., Contemplative/emotion training reduces negative emotional behavior and promotes prosocial responses, Emotion 12 (2) (2012) 338. [56] I. Zupic, T. ˇ Cater, Bibliometric methods in management and organization, Organ. Res. Methods 18 (3) (2015) 429–472. [57] B.C.A. Petroni, E.M. de Moraes, R.F. Gonçalves, Big data analytics for logistics and distributions using blockchain, in: IFIP International Conference on Advances in Production Management Systems, Springer, 2018, pp. 363–369. A. Ahmad et al.