A framework for the Comparative analysis of text summarization techniques

i A framework for the Comparative analysis of text summarization techniques Trijit Ghosh Dissertation presented as partial requirement for obtaining the master’s degree in Data Science and Advanced Analytics 2 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa A FRAMEWORK FOR THE COMPARATIVE ANALYSIS OF TEXT SUMMARIZATION TECHNIQUES by Trijit Ghosh (M20170009) Dissertation presented as partial requirement for obtaining the master’s degree in Data Science and Advanced Analytics Advisor / Co Advisor: Professor Ricardo Rei; Professor Roberto Henriques July 2021 3 ACKNOWLEDGEMENTS I would first like to thank my thesis advisor Professor Doctor Ricardo Rei of the NOVA Information Management School at Universidade NOVA de Lisboa as he was the one who challenged me to a theme as innovative as the one that gave motto to this master’s thesis on text summarization. I want to thank him for encouraging me and motivating me even when the time to devote to this master’s thesis was not what was wanted and expected. Finally, a big thank you to my parents for encouraging me and giving me the chance to do this master’s in areas as interesting and exciting as advanced analytics are. That gave me the possibility to have a career that I have dreamed of. It would not be possible without their support. 4 5 Contents 1. Introduction ...................................................................................................................................... 8 1.1. Background ............................................................................................................................... 8 1.2. Motivation ................................................................................................................................. 8 1.3. Objective .................................................................................................................................... 8 2. EXTRACTIVE SUMMARIZATION ............................................................................................... 9 2.1. Intermediate Representation ............................................................................................ 9 2.2. Sentence Score ......................................................................................................................... 9 2.3. Summary Sentences Selection ........................................................................................... 9 3. TOPIC REPRESENTATION APPROACHES .......................................................................... 11 3.1. Topic Words .......................................................................................................................... 11 3.2. Frequency-driven Approaches ....................................................................................... 11 3.3. Latent Semantic Analysis ................................................................................................. 14 3.4. Bayesian Topic Models ...................................................................................................... 14 3.5. BERT ......................................................................................................................................... 15 4. THE IMPACT OF CONTEXT IN SUMMARIZATION ........................................................... 20 4.1. Web Summarization ........................................................................................................... 20 4.2. Scientific Articles Summarization ................................................................................. 20 4.3. Email Summarization ........................................................................................................ 21 5. METHODOLOGY ........................................................................................................................... 35 5.1. Design Search Research .................................................................................................... 35 5.2. Strategy ................................................................................................................................... 37 6. PROPOSAL of a framework on scenarios of text summarization techniques ...... 38 6.1. PROPOSAL .............................................................................................................................. 38 6.2. VALIDATION .......................................................................................................................... 38 7. CONCLUSIONS ............................................................................................................................... 46 8. References ...................................................................................................................................... 47 6 List of Tables Table 1 ............................................................................................................................................................... 21 Table 2 ............................................................................................................................................................... 27 Table 3 ............................................................................................................................................................... 38 Table 4 ............................................................................................................................................................... 42 Table 5 ............................................................................................................................................................... 43 7 List of figures Figure 1 : Weighted Terms v/s Specificity .......................................................................................... 13 Figure 2 : Bert Embeddings ...................................................................................................................... 16 Figure 3 : Architecture of BERT .............................................................................................................. 17 Figure 4 : Encoders and Decoders ......................................................................................................... 17 Figure 5 : Overall pre-training and fine-tuning procedures for BERT ..................................... 18 Figure 6 : Fine Tuning phase .................................................................................................................... 18 Figure 7 : Accuracy of BERTbase on Masked LM and Left-to-Right ............................................. 19 Figure 8 : Precision and Recall of different text files ...................................................................... 40 Figure 9 : Precision, Recall and F-Measure for different values of k applying LSA ............ 41 Figure 10 : Overall Comparison of the methods ............................................................................... 44 8 1. INTRODUCTION 1.1. BACKGROUND We see that with the boom of information technology and IOT (Internet of things), the size of information which is basically data is increasing at an alarming rate. This information can always be harnessed and if channeled into the right direction, we can always find meaningful information. But the problem is this data is not always numerical and there would be problems where the data would be completely textual, and some meaning has to be derived from it. If one would have to go through these texts manually, it would take hours or even days to get a concise and meaningful information out of the text. This is where a need for an automatic summarizer arises easing manual intervention, reducing time and cost but at the same time retaining the key information held by these texts. In the recent years, new methods and approaches have been developed which would help us to do so. These approaches are implemented in lot of domains, for example, Search engines provide snippets as document previews, while news websites produce shortened descriptions of news subjects, usually as headlines, to make surfing easier. Broadly speaking, there are mainly two ways of text summarization – extractive and abstractive summarization. Extractive summarization is the approach in which important sections of the whole text are filtered out to form the condensed form of the text. While the abstractive summarization is the approach in which the text as a whole is interpreted and examined and after discerning the meaning of the text, sentences are generated by the model itself describing the important points in a concise way. 1.2. MOTIVATION As the Internet has grown in popularity, a vast amount of information has become available. Summarizing vast amounts of text is challenging for humans. In this age of information overload, automatic summarizing technologies are in high demand. We will try to focus on various extraction methodologies for single and multi-document summarization in this thesis. Some of the most often used methods, such as topic representation approaches, frequencydriven methods, graph-based and machine learning techniques, will be described. Even though it is difficult to thoroughly explain all of the many algorithms and approaches in this thesis, we will try to give a good overview of recent trends and breakthroughs in automatic summarizing methods, as well as discuss the state-of-the-art and compare various ways. 1.3. OBJECTIVE The objective of this paper is to provide comparative analysis of different techniques of text summarization used in different scenarios, methodology - to define a set of analysis parameter that can allow us to classify different techniques e.g., complexity, accuracy and speed. 9 2. EXTRACTIVE SUMMARIZATION Extractive summarization, as mentioned above, chooses pertinent subsets from the text given, based on some metrics and is thus combined to a condensed form. To understand how summarization systems work, we describe three, fairly, independent tasks which all summarizers perform: 1) Build a transitional depiction of the input text which communicates the most important aspects of the text. 2) Score the sentences supporting the representation. 3) Choose a summary consisting of variety of texts. 2.1. INTERMEDIATE REPRESENTATION All these summarization techniques will develop some intermediate representations of the given input text supported certain metrics and discern the important sentences supported those metrics. There are two forms of approaches supported the representation: topic representation and indicator representation. Topic representation methods remodel the text into a transitional characterization and examine the topic(s) given within the text. This method differs in terms of formulation and thereby, complexity, and are divided into frequency-driven approaches, topic word approaches, latent semantic analysis and Bayesian topic models. We take a deeper look into topic representation approaches within the following sections. Indicator representation approaches describe every sentence as a listing of features (indicators) of importance like sentence length, position within the document, having certain phrases, etc. 2.2. SENTENCE SCORE When the input text is transformed into a form which the model interprets, a score is assigned to every sentence based on that metric. This score is just a representation of how important a sentence is. These indicators (metric) are derived from mathematical or machine learning models. Once the score of every of the sentences are obtained, they are, then, aggregated or fed into a function which ranks these sentences based on the scores obtained. they're also referred to as indicator weights. 2.3. SUMMARY SENTENCES SELECTION After the sentences are scored and ranked supported the various metrics or indicators, the important sentences are filtered out. These processes can use different algorithms to separate the sentences supported the ranks and a few other scores as an example redundancy score. Some methods use greedy algorithms to search out the simplest sentences representing the essence of the text. This method will not always be the most effective approach which ignores 16 Figure 2 : Bert Embeddings The input text is first fed into the text processing embeddings namely: - Position Embeddings Segment Embeddings Token Embeddings 17 Figure 3 : Architecture of BERT The output of these embedding is then fed to the BERT layer which consists of transformers. Figure 4 : Encoders and Decoders A transformer can consist of 12/24 blocks of encoders with 12/16 attention heads and 110/340 million parameters, namely BERTbase/BERTlarge respectively. If looked closely, each transformer can be a set of encoders and decoders as shown above in the diagram. It is here that the output of the transformer is fed to the summarization layer. Pre-training has been quite crucial when it came to language models. There have been 18 applications of these models such as natural language inference and paraphrasing. This paper (Devlin, Chang, Lee, & Toutanova, 2018), mainly talks about fine tuning approach with the proposal of BERT as described earlier. The reason it is called bidirectional is because unlike other models where the text is read from left to right or from right to left, the BERT approach reads the sentences in both the directions and tries to understand the context of the words both to its left and right. Figure 5 : Overall pre-training and fine-tuning procedures for BERT In the above diagram, the architecture of the BERT is shown where apart from the output layers, both use the same architecture. Figure 6 : Fine Tuning phase Source: (Devlin, Chang, Lee, & Toutanova, 2018) The above figure shows that by working on the fine-tuning phase of the model, the model was able to achieve an accuracy by an overwhelming margin of 4.5% and 7% prior to its state of the art. 19 Figure 7 : Accuracy of BERTbase on Masked LM and Left-to-Right Source: (Devlin, Chang, Lee, & Toutanova, 2018) Although it is to be mentioned that this bi-directional approach takes longer than unidirectional approach which is what has been shown in the above diagram where the BERT MLM method converges slower than left to right but accuracy wise, it is well ahead of the other approach. This bi-directional approach is, definitely, useful although a bit slow and makes the application of BERT to a wider range of fields. 20 4. THE IMPACT OF CONTEXT IN SUMMARIZATION It is quite evident that a context is very useful when it comes to understanding the information presented in a text. In the same way, if models can be empowered with the information of context, then a summarizer system would be able to prune the correct information to summarize the text. For example, according to (Allahyari, et al., 2017), when summarizing blogs, the debates or comments that follow the blog post are useful resources for determining which portions of the blog are critical and intriguing. There is a significant quantity of information available in scientific paper summaries, such as published articles and conference information, that can be used to highlight essential sentences in the original work. 4.1. WEB SUMMARIZATION If we look at the web pages, we will see that they have lot of objects which is not always possible to summarize for example, pictures, gifs, and some unwanted materials like advertisements which is so not relevant to the information given in the pages. For those kinds of situations, it will be helpful to use the links which direct us to the page to be summarized. These links would give the model a knowledge of the context and would be helpful to provide improved summary for the page. In (Amitay & Paris, 2000), where they talk about website summurization for the first time, they came up with a concept called “THE INCOMMONSENSE SYSTEM”. This model incorporates a web crawling system which crawls all the links that link back to the current page and this is how the context is derived. Since then, a lot of different algorithms have been developed which was based on the above principle but has beeen improved. 4.2. SCIENTIFIC ARTICLES SUMMARIZATION In (Mei & Zhai, 2008), a summurization problem related to scientific papers is studied and presented. Needless, to say with each passing year, new discoveries are being made and new research papers are being published every year. So with this, the daunting task of briefing a scientific papers becomes really challenging. Specially when it comes to including a context in the summary which references a multitude of different research papers. In order to solve this problem, they came with a concept of impact based summurization as mentioned in (Mei & Zhai, 2008). This method leverages on the importance of sentence score which has been used in the original paper using the KL divergence method (i.e., finding the similarity between a sentence and the language model). The conclusions made in this paper are substantial porving that this method was useful and could be used for future breifing models for scientific papers.They proposed a language model that gives a probability to each word in the citation context sentences. They then score the importance of sentences in the original paper. . 21 4.3. EMAIL SUMMARIZATION When it comes to email summarization, the texts become a bit different. In order to find a context, a whole chain mail threads are to be processed to determine the story. In (Nenkova & Bagga, 2003), they discuss a few methods of how the conversational nature of the text can be used to gather context. Their methods does provide a conclusive evidence as to how useful this method could be and this further, this method could be enhanced by using some visualization techniques as well. While in (Rambow, Shrestha, Chen, & Lauridsen, 2004), they take a bit of a different approach where they determine important features for summarizing the email, where each feature could be one thread of several threads of email. (Newman & Blitzer, 2003) discusses a whole new approach for summarizing an email. They leverage the clustering algorithm to solve the problem of email summarization. In their paper they talk about few clustering algorithms which help them see all the threads of the email as a whole and post application of the mentioned algorithms, an overview is formed. Below, Table 2. cite the journals or the references from which some techniques were analyzed and furthermore, their benefits or defects are given. Table 2. Illustrates the various methods which were explained in the articles mentioned in Table 1 22 21 Table 1 Journal/Conference Sub Topic Years (xxxxxxyy) # Articles #Articles Techniques Benefits #Articles Techniques Drawbacks 2017 International Conference on Computing Methodologies and Communication (ICCMC) Text Summarization 2017 Automatic text summarization by local scoring and ranking for improving coherence It has made use of sentence feature metrics to score sentences like “sentence to sentence cohesion”, “word frequency”, it makes use of metrics which might ignore valuable information and thereby making the summary less meaningful. For example, metric like “length of sentence” is used to avoid selecting too short or too long sentences of the document. Doing this, at times, the model might overlook some information which might have been present in those sentences but were not taken into account while summarizing the text. 22 2017 International Conference on Big Data, IoT and Data Science (BID) Text Summarization 2017 Automatic text summarization of news articles The lexical chain generation proposed by Silber and McCoy algorithm has linear run time complexity. Further, certain issues were resolved in both algorithms by implementing pronoun resolution and enhanced sentence scoring to leverage the structure of news articles. One of the lexical chain generation algorithm adopted was proposed by (Barzilay & Elhadad, 2000) has exponential run time complexity. Artificial Intelligence Review archive Volume 47 Issue 1, January 2017 Pages 166 Text Summarization 2017 Recent automatic text summarization techniques: a survey This paper talks about a variety of techniques which have their own benefits. 1. Trained Summarizer and Latent Sematic Analysis – uses a modified corpus-based approach and a TRM technique based on latent semantic analysis. The summarizer is based on a function that assesses main sentences/words for things like location, keyword, likeness to title, and centrality in order to generate summaries. The score function is optimized using stochastic techniques such as genetic algorithms. 2. Information retrieval performance has greatly improved There are some drawbacks of the approaches used which are as follows:Trained Summarizer and Latent Sematic analysis – the summaries generated are not very consistent with topic and the sentences don’t correlate so much at times. Feature weights of score function produced by GA fail to consistently give the best results for the test corpus. Obtaining the appropriate dimension reduction ratio and explaining LSA effects are tough in the LSA+TRM technique. Moreover, the time complexity to compute SVD is quite high. 2. Using a sentence-based abstraction technique to extract data – In this approach, only casual coherence is considered whereas 23 because of the use of a sentence-based abstraction technique. Sentences that represent the central notion are linked. 3. Understanding and summarizing documents using a document concept lattice – In comparison to existing sentence grouping and sentence scoring algorithms, the suggested approach performs exceptionally well. 4. Sentence extraction using text summarization based on context and statistics – This space and time which are inter-related links which makes sense out of the document as a whole are also required for representing behavioral context. 3. Through the use of a document concept lattice, it is possible to comprehend and summarize text – time complexity for generating a DCI is high because it considers all possible combinations. 4. Sentence extraction through contextual and statistical based summarization text. 30 document summaries through non - negative matrix factorization this approach could not naturally catch the meaning of semantic features that are highly sparse and have a limited view of meaning. As a result, summarization systems based on LSA are unable to choose meaningful phrases. As a result, elements of semantic feature vectors in the suggested method exclusively contain non-negative values and are also extremely sparse, allowing semantic characteristics to be easily read. A sentence can be represented by a linear combination of certain significant semantic elements. As a result, subtopics in a document can be quickly identified, and there's a better probability of extracting relevant lines. A method for picking phrases to construct general document summaries is suggested using NMF, in which a content is first pre-processed and then summarized. To generate a non-negative semantic feature matrix, NMF is used to a term-by-sentence matrix. For each sentence, generic relevance is calculated, which indicates how much a sentence explains. Query based summarization of multiple documents by applying regression models 2011 Ouyang Y Ouyang, Li, Li, & Lu, 2011 (Ouyang, Li, Li, & Lu, 2011) suggested a method for ranking phrases in query-based summarization of numerous manuscripts using regression models. Three querydependent features, such as named-entity matching, word-matching, and semantic matching, and four query-independent features, such as sentence position, named entity, word TF-IDF, and stop-word penalty, are used in this methodology to choose main sentences in query-based summarization of multiple documents. To begin with, human summaries generate "false" training data. Then, using different methods based on the N-gram methodology that calculate "nearly true" relevance ratings of phrases are created and analyzed using this training data and their collection of texts, and a mapping function is learned using this training data via a collection of previously specified features of sentences. Then, using this learned function, the significance of sentences in the test data is estimated. An efficient data collection of training data for learning regression models requires two things: (a) an appropriate group of topics with correctly handwritten summaries, and (b) a good approach for computing the relevance of words. The Maximal Marginal Relevance (MMR) technique is used to remove redundancy from the summary. Automatic text summarization using MR, GA, 2009 Mohamed Abdel Fattah, Fuji Ren (Fattah and Ren 2009) With the use of a few statistical features, (Fattah & Ren, 2009) suggested an approach to improve content selection in automatic text summarization. As a trainable summarizer, this method relies on distinct statistical aspects in each sentence to generate summaries. Position of Sentence (Pos), +ve keyword, -ve keyword, +ve 31 FFNN, GMM and PNN based models keyword, +ve keyword, +ve keyword, +ve keyword, +ve keyword, +ve keyword, +ve keyword, +ve keyword, +ve keyword, R2T, Centrality of Sentence (Cen), Presence of Name Entity in Sentence (PNE), Presence of Numbers in Sentence (PN), Bushy Path of Sentence (BP), Relative Length of Sentence (RL), and Aggregate Similarity (AS) are all measures of sentence similarity. Genetic Algorithm (GA) and Mathematical Regression (MR) models have been trained to acquire an optimal mix of feature weights by mixing all of these features. For sentence categorization, feed forward neural networks (FFNN) and probabilistic neural networks (PNN) are utilized. Some text features, such as the +ve and -ve keywords, are language-dependent, while eight others are not. All of the above-mentioned variables are taken into account when calculating a sentence's weighted score function. All sentences in a document are ranked in decreasing order of their scores, and a highly scored cluster of sentences is utilized to generate a summary of the content using various compression rates (10, 20, 30 percent used here). Conclusion: The results demonstrate that feature BP is the most essential text feature since it produces the best results, while feature PND produces the worst results because statistical data is absent from religious and political pieces. Because it could model arbitrary densities, the GMM approach produced the best results of all the strategies. Maximum coverage and minimum redundancy in summarization of text 2011 Alguliev, R. M Alguliev, Aliguliyev, Hajirahimova, & Mehdiyev, 2011 (Alguliev, Aliguliyev, Hajirahimova, & Mehdiyev, 2011) introduced an unsupervised summarizing model for generic text as an Integer Linear Programming problem (ILP) that immediately detects essential sentences from the article as well as the full article's relevant information. Maximum Coverage and Minimum Redundancy is the name of this strategy (MCMR). This method aims to improve three key aspects of a summary: (a) relevance, (b) redundancy, and (c) length. A subset of sentences from the document collection's relevant text is picked. Then, using NGD-based similarity (Normalized Google Distance) and cosine similarity, similarity between the summary and the document collection is computed, and this similarity must be maximized. An objective function is developed and must be maximized to ensure that the summary contains the important content found in the document collection and that the summary does not contain a significant number of phrases that communicate the same information. At the same time, the length of the summary must be limited. Lastly, an empirical function is created by linearly combining the cosine similarity- 32 based and NGD-based similarity empirical functions, and this combined empirical function must be maximized. This technique to summarizing is incorporated as an optimization problem that aims to find a global solution to the problem. The Branch & Bound algorithm (B&B) and the Binary Swarm Optimization method are the algorithms used to address the ILP problem. Conclusion: This method, which combines MCMR with the B&B algorithm, surpasses all others. It demonstrates that summarizing outcomes are dependent on similarity measurements. It is also proved through tests that using cosine similarity and NGD-based similarity metrics together produces better results than using them separately. Summarization of documents through a progressive technique for selection of sentences 2013 Ouyang Y Ouyang, Li, Zhang, Li, & Lu, 2013 (Ouyang, Li, Zhang, Li, & Lu, 2013) proposed a new progressive technique for generating a summary based on the selection of "novel and salient" sentences. Subsuming relationship between two sentences, i.e., an irregular relationship between sentences that shows the level of recommendation of one phrase by another. In order to ascertain the link between two phrases, the relationship between their concepts must be found. The association between concepts is then discovered by using a coverage-based measure to discover the relationship between words. A Direct Acyclic Graph (DAG) is used to organize all of the words that appear in the found word relations. A progressive strategy for sentence selection is created on the basis of an asymmetric relationship between sentences, in which a sentence is either picked as a novel general statement or as a supporting sentence. The following two methods are used to choose new and relevant sentences in this method: (a) discovered concepts are only included during the assessment of sentence relevance to assure sentence originality, and (b) for now, the relationship between sentences is used to improve the saliency measure. To implement this strategy, a random walk on the DAG from the central node to its nearby nodes is performed, with the goal of covering the central words first and then reaching the greatest amount of words via word relations. Redundancy is eliminated by punishing repetitive words, resulting in fresh concepts being introduced each time a new phrase is chosen. Conclusion: In terms of generating summaries with improved saliency and coverage, the Progressive system surpasses the traditional Sequential approach. Evaluation of 2013 Ferreira, Rafael; Cabral, Luciano de Ferreira, et al., In the recent decade, (Ferreira, et al., 2013) incorporated fifteen scoring techniques that had been referenced in the research. ROUGE (Lin 2004) is used for quantitative 33 sentence scoring methods for extractive summarization of text Souza; Lins, Rafael Dueire; Silva, Gabriel Pereira e; Freitas, Fred; Cavalcanti, George D.C.; Lima, Rinaldo; Simske, Steven J.; Favaro, Luciano 20132013 evaluation, while the number of sentences that are similar among the machinegenerated and human-made summaries is counted for qualitative evaluation. The processing time of each algorithm is taken into account. Word scoring, sentence scoring, and graph scoring approaches are used to pick relevant sentences. The most essential terms are given scores in the word scoring approach. Word frequency, TF/IDF, upper case, proper noun, word co-occurrence, and lexical similarity are among the approaches used to score words. The properties of sentences are examined in the sentence scoring approach. The existence of cues, numerical data, sentence length, sentence position, and sentence centrality are all factors in sentence scoring. Scores are determined using the graph scoring approach by looking at the relationships between sentences. Text rank, bushy path of the node, and aggregate similarity are all graph scoring approaches. The six common concerns of stop words, structural transformation, comparable semantics, ambiguity, redundancy, and coreference are then explored, along with some suggestions for advancing sentence score outcomes. Exploring correlations among multiple terms through a graph-based summarizer, GRAPHSUM 2013 Baralis, Elena; Cagliero, Luca; Mahoto, Naeem; Fiori, Alessandro Baralis, Cagliero, Mahoto, & Fiori, 2013 GRAPHSUM, a new graph-based, general-purpose summarizer for summarizing numerous documents, was proposed by (Baralis, Cagliero, Mahoto, & Fiori, 2013). This method investigates and applies association rules, a data mining methodology for finding connections between several terms. It is not reliant on sophisticated semantic models (like taxonomies or ontologies). The document collection is organized as a transactional dataset after preprocessing so that association rule mining may be conducted on it. Then, from the transactional dataset, frequently recurring itemsets with high correlations among the terms are identified, and a correlation graph is constructed from these terms, which will aid in the selection of significant lines for the summary. The Apriori algorithm is used to mine frequently recurring itemsets, and the support measure is employed for this job. The lift measure indicates the intensity of relationship between two terms and is used to evaluate positive or negative connections between commonly used words. A variation of the classic PageRank graph ranking algorithm is used to determine the relevance of the graph nodes. The graph nodes that have a significant number of positive correlations are placed first, while those that have a negative connection with the adjacent nodes are penalized. For summary creation, the sentences that are the most appropriate for the correlation 34 graph and have a high relevance score are picked. The greedy algorithm is employed to select sentences in this case. GRAPHSUM performs better over a wide range of state-of-the-art techniques, some of which rely heavily on highly developed semanticbased models or complicated language processes. Incorporating various levels of language analysis for tackling redundancy in text summarization 2013 Elena Lloret, Manuel Palomar Lloret & Palomar, 2013 (Lloret & Palomar, 2013) provided a method for detecting redundant information based on three layers of language analysis: lexical, syntactic, and semantic. Cosine similarity is utilized in the lexical based technique to detect similarity between sentences in two sources. Those sentences that have a cosine similarity greater than a certain threshold are considered repetitive, and they are all eliminated. In a syntacticbased method, entailment relations are computed between pairs of phrases to determine whether the meaning of one sentence can be deduced from the meaning of the other sentence. If a positive entailment is obtained, the second sentence is deemed superfluous and eliminated. Sentence alignment is determined at the document level between a set of linked documents using a open source available Champollian Tool Kit in a semantic-based manner. Syntactic and semantic techniques are preferable than lexical approaches that rely on cosine similarity. Text summarization can be done in two ways. Before the material is summarized, unnecessary sentences are deleted in the first technique. The set of useful sentences is then given to the summarization system, which uses statistical (term frequency) and linguistic (code quantity principle) factors to select essential sentences, as well as a summary. 35 5. METHODOLOGY This study will be conducted through a qualitative research, based on a well-structured parameter for comparing the different techniques adopted for text summarization techniques. This is the basis to compare different techniques and shed light on which techniques would be more useful in (if any) particular situations. What are their drawbacks and advantages? 5.1. DESIGN SEARCH RESEARCH Design Science Research is a form of investigation that entails building or improving something in a novel way in response to a specific challenge. The quest for a solution based on extensive scientific investigation ensures that the final proposed artifact is coherent and credible. A crucial phase that should not be overlooked is good communication of the finished product (Hevner, March, Park, & Ram, 2004). Each of the six key stages of DSR methodology, as shown in Figure 5, will be discussed in greater detail right away. Figure 3. DSR Method Adaptation (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007) Identify problem and motivation Define the research challenge in detail and justify the importance of a solution. Begin by establishing a testable theory that leads to a research problem by demonstrating to stakeholders the value of an effective solution and what they will gain from its result (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). Define objectives and a solution 36 Clearly define goals (quantitative or qualitative) to establish the foundation for a solution based on the problem characterization and what can and cannot be done (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). Design and Development The goal of the design and development stages is to create knowledge through the design and development of the artifact itself (Gregor & Hevner, 2013). This could be accomplished by breaking down the major scientific problem into smaller components (Hevner, March, Park, & Ram, 2004). To have an effective/ clear structure in the next phase, it is necessary to have a clear grasp of the solution value and to defend it with some theoretical foundation (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). A solution that must meet business requirements (Hevner, March, Park, & Ram, 2004). To gain the appropriate theoretical basis, it is essential to do research and collect knowledge about the present status of the problem and existing solutions, as well as to analyze direct and indirect solutions and their efficacy (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). With the knowledge, it is possible to develop a solution to meet research and, as a result, business objectives, as well as to debate the usefulness of the suggested artifact (Hevner, March, Park, & Ram, 2004). Evaluation To certify an artifact's efficacy, it must be put to use or presented to stakeholders (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007), which must be supported by a clear specification of evaluation methodologies that are suitable for the situation at hand and are based on industry requirements. Because the majority of claims on the final solution are related to performance issues, alignment with business needs is critical (Hevner, March, Park, & Ram, 2004). Comparing what falls under the purview of the master's thesis with what could be observed in its practical implementation is one technique to evaluate how the answer matches the initial challenge (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). Although it is critical to emphasize that the primary goal is to "identify how well an artifact works" rather than "theorize or prove anything about why the artifact works" (Hevner, March, Park, & Ram, 2004). At the end of this phase, it should be determined whether the artifact is ready to be shared with the rest of the world, or whether more effort should be spent improving it to make it more effective/aligned with the original problems (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). Communication While releasing the final artifact to the public is a step in the right direction, it's also critical to let people know how unique and successful the artifact is in solving the highlighted problems (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). It is critical to discuss how the artifact was created and the review process that led to its validation throughout this communication (Hevner, March, Park, & Ram, 2004). 37 It should be conveyed to technical and management audiences in order to gather input to enhance the solution, both in terms of business and technology, for future implementations (Hevner, March, Park, & Ram, 2004). 5.2. STRATEGY Problem There are numerous text summarizing approaches, each with its own set of benefits and drawbacks. Some are more computationally complex than others, while others have only been implemented in specific languages. Some utilize more statistical measures to quantitatively address summarizing problems, while others more extractive in nature. Despite all of these possibilities, there is no single approach or methodology that can be used on any type of text. We need to know which strategy or technique to use in various situations. Objective After stating the topic, our goal in this paper will be to research and assess several strategies, as well as to describe their benefits and drawbacks, as well as the situations and circumstances in which they might be employed. In the same case, not all methods would perform the same. As a result, we would do our best to produce a fair comparison and highlight the techniques' or methodology' limitations. Design and Development Initially, a number of research publications on text summarization approaches were examined. Some of the strategies for examining its algorithm, time complexity, the data it was implemented on, how efficient the algorithm is, how useful the generated summary is, and whether it was an abstractive or extractive based methodology have been detailed in depth above. 38 6. PROPOSAL OF A FRAMEWORK ON SCENARIOS OF TEXT SUMMARIZATION TECHNIQUES 6.1. PROPOSAL Although text summarization has a vast number of techniques to offer, it was not possible to cover all of those here and a such only few were selected, which were studied here aforementioned in the above tables. The following table below compares those above techniques in terms of accuracy and time complexity, applicability. Although this table does not give a fair comparison since, all these techniques were not applied on the same document and for the same situations. Table 3 Techniques Parameters Accuracy Speed Applicability on different language Scenarios applicable The lexical chain generation Accuracy is better Has linear run time complexity For example, Bengali Although this method can be applied to multiple situations, most research papers state its main applicability in World Wide Web. Latent Semantic Analysis Certain combinations show different accuracy mentioned below Linear Time complexity For example, Bengali, Hindi LSA now scales to ca. 100 million-word corpora by larger computer memory and new algorithms. Query based summarization of multiple documents by Results demonstrate that for computing the importance of In The speed varies with documents explained in detail Although, any paper related to this technique has not summarizing research papers of a specific domain, biomedical documents for better accuracy 39 applying regression models comparison to training to score and classifying models, regression models perform better. below yet been applied to other language. But this technique should not have any issues (technical) if applied to other language. In summarizing text, maximum scope and desired minimal repetition Accuracy is 97% according to (HoudaOufaida, OmarNouali, & PhilippeBlache, 2014). Although this is just one sample. Computational time is proportional to O(X*Y) where X and Y are different terms in the distance matrix used to discern the similarity. Tested in languages like Arabic, Czech, English, French, Greek, Hebrew and Hindi singleand multi-document summarization. In both tasks, documents are split into sentences in preprocessing Evolutionary optimization algorithm for summarizing multiple documents Accuracy is usually good if the algorithm is run making sure that the whole search space is covered and not stuck at local maxima Time complexity for these algorithms is usually pretty high as it has to make sure that the whole search space is covered during the run-time. This proposed method has not yet been applied in other languages. Digital archives of governmental documents The lexical chain generation - Word Sense Disambiguation (WSD) accuracy is better. The algorithm proposed by Silber and McCoy has linear run time complexity. Tested in different languages apart from English. For example, Bengali. Although this method can be applied to multiple situations, 46 7. CONCLUSIONS As the Internet has grown in popularity, a vast amount of information has become available. Summarizing vast amounts of text is challenging for humans. In this age of information overload, automatic summarizing technologies are in high demand. Various extraction methodologies for single and multi-document summarization were highlighted in this research. Topic representation approaches, frequency-driven methods, graphbased and machine learning techniques were described as some of the most often utilized methodologies. Although it is impossible to elucidate all of the many methods and approaches in my thesis, it does provide a good overview of recent trends and advancements in automatic summarizing methods and describes the current state-of-the-art in this field. Limitations One of the main limitations of this report is that it wasn’t validated by lot of people given the fewer number of experts in this field. With that goes the unsaid, that this paper doesn’t document all the NLP techniques, which is quite a broad field. 47 8. REFERENCES A., N., & K., M. (2012). A Survey of Text Summarization Techniques. Em A. C., & Z. C., Mining Text Data (pp. 43-76). Boston, MA: Springer, Boston, MA. Alguliev, R. M., Aliguliyev, R. M., Hajirahimova, M. S., & Mehdiyev, C. A. (2011). MCMR: Maximum coverage and minimum redundant text summarization model. Expert Systems with Applications , 14514-14522. Allahyari, M., Pouriyeh, S., Assefi, M., Safaei, S., Trippe, E. D., Gutierrez, J. B., & Kochut, K. (2017). Text Summarization Techniques: A Brief Survey. Amitay, E., & Paris, C. (2000). Automatically Summarising Web Sites - Is There A Way Around It? CIKM00: Proceedings of the ninth international conference on Information and knowledge management (pp. 173–179). Association for Computing MachineryNew YorkNYUnited States. Baralis, E., Cagliero, L., Mahoto, N., & Fiori, A. (2013). GraphSum: Discovering correlations among multiple terms for graph-based summarization. Em Information Sciences (pp. 96-109). Barzilay, R., & Elhadad, M. (2000). Using Lexical Chains for Text Summarization. Carenini, G., Ng, R. T., & Zhou, X. (2008). Summarizing Emails with Conversational Cohesion and Subjectivity. (pp. 353–361). Association for Computational Linguistics. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science . Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Fattah, M. A., & Ren, F. (2009). GA, MR, FFNN, PNN and GMM based models for automatic text summarization. Computer Speech & Language , 126-144. Ferreira, R., Cabral, L. d., Lins, R. D., Silva, G. P., Freitas, F., Cavalcanti, G. D., . . . Favaro, L. (2013). Assessing sentence scoring techniques for extractive text summarization. Em Expert Systems with Applications (pp. 5755-5764). Gong, Y., & Liu, X. (2001). Generic text summarization using relevance measure and latent semantic analysis. Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 19-25). SIGIR '01. HoudaOufaida, OmarNouali, & PhilippeBlache. (2014). Minimum redundancy and maximum relevance for single and multi-document Arabic text summarization. Journal of King Saud University - Computer and Information Sciences , 450-461. III, H. D., & Marcu, D. (2006). Bayesian Query-Focused Summarization. Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics (pp. 305-312). ACL-44. 48 Jones, K. S. (2004). A statistical interpretation of term specificity. Journal of Documentation Volume 60 Number 5 , 493-502. Ko, Y., & Seo, J. (2004). Learning with Unlabeled Data for Text Categorization Using a Bootstrapping and a Feature Projection Technique. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04) , (pp. 255–262). Ko, Y., & Seo, J. (2008). An effective sentence-extraction technique using contextual information and statistical approaches for text summarization. Em Pattern Recognition Letters (pp. 1366-1371). Lee, J.-H., Park, S., Ahn, C.-M., & Kim, D. (2009). Automatic generic document summarization based on non-negative. Information Processing and Management , 20-34. Lloret, E., & Palomar, M. (2013). Tackling redundancy in text summarization through different levels of language analysis. Em Computer Standards & Interfaces (pp. 507-518). Luhn, H. P. (1958). The Automatic Creation of Literature Abstracts. IBM JOURNAL APRIL 1958 . Mei, Q., & Zhai, C. (2008). Generating Impact-Based Summaries for Scientific Literature. Proceedings of ACL-08: HLT (pp. 816–824). Columbus: Association for Computational Linguistics. Nenkova, A., & Bagga, A. (2003). Facilitating email thread access by extractive summary generation. Recent Advances in Natural Language Processing III: Selected papers from RANLP 2003 , pp. 287-. Newman, P. S., & Blitzer, J. C. (2003). Summarizing Archived Discussions: A Beginning. Proceedings of the 8th international conference on Intelligent user interfaces (pp. 273– 276). IUI '03. Ouyang, Y., Li, W., Li, S., & Lu, Q. (2011). Applying regression models to query-focused multidocument summarization. Information Processing & Management , 227-237. Ouyang, Y., Li, W., Zhang, R., Li, S., & Lu, Q. (2013). A progressive sentence selection strategy for document summarization. Information Processing & Management , 213-221. Rambow, O., Shrestha, L., Chen, J., & Lauridsen, C. (2004). Summarizing Email Threads. Proceedings of HLT-NAACL 2004: Short Papers (pp. 105–108). HLT-NAACL-Short '04. Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing and Management . W.K.Chan, S. (2006). Beyond keyword and cue-phrase matching: A sentence-based abstraction technique for information extraction. Em Decision Support Systems (pp. 759-777). Ye, S., Chua, T.-S., Kan, M.-Y., & Qiu, L. (2007). Document concept lattice for text understanding and summarization. Em Information Processing & Management (pp. 1643-1662). Yeh, J.-Y., Ke, H.-R., Yang, W.-P., & Meng, I.-H. (2005). Text summarization using a trainable summarizer and latent semantic analysis. Em Information Processing & Management (pp. 75-95). Page | i