Climate Change Representation in IPCC Reports and Wikipedia: A Comparative Analysis Through Natural Language Processing

Master Technologies des Langues 2023-2025 Lucas PRÉVOT Master’s Thesis Under the supervision of Pablo RUIZ FABO Senior Lecturer Climate Change Representation in IPCC Reports and Wikipedia: A Comparative Analysis Through Natural Language Processing La représentation du changement climatique dans les rapports du GIEC et sur Wikipédia : Une analyse comparative à l’aide du traitement du langage naturel Cette étude présente une analyse comparative via traitement automatique du langage naturel des résumés à l’intention des décideurs (SPM) du Groupe de travail III du Groupe d’experts intergouvernemental sur l’évolution du climat (GIEC) et des versions correspondantes de l’article « Atténuation du changement climatique » en anglais sur Wikipédia. En utilisant un large éventail de techniques de TAL, notamment la lexicométrie, la stylométrie, l’évaluation de la lisibilité, l’analyse de la modalité, la similarité sémantique (Sentence BERT), le topic modeling (BERTopic), la détection des sentiments et des émotions, ainsi que la reconnaissance d’entités nommées, l’étude examine comment l’atténuation du changement climatique est représentée dans ces deux sources entre 1990 et 2022. Les résultats montrent que les SPM du GIEC conservent un niveau de technicité élevé, tandis que Wikipédia, initialement plus accessible et centré sur les événements et les personnalités, s’est progressivement aligné, tant sur le plan sémantique que stylistique, avec le GIEC. L’analyse ne révèle aucune preuve de biais délibéré dans la représentation de l’atténuation du changement climatique sur Wikipédia. Les différences de cadrage et de mise en avant s’expliquent plutôt par son rôle de ressource éditée par le public et destinée à un large public. L’étude conclut que Wikipédia joue un rôle important et en constante évolution dans la compréhension des enjeux du changement climatique chez le grand public. Le contenu de Wikipédia est de plus en plus fidèle aux rapports du GIEC. Mots-clés : Changement climatique, GIEC, Wikipédia, traitement du langage naturel, TAL, analyse, topic modeling, reconnaissance des entités nommées, communication scientifique Climate Change Representation in IPCC Reports and Wikipedia: A Comparative Analysis Through Natural Language Processing This thesis presents a comparative Natural Language Processing analysis of the Intergovernmental Panel on Climate Change (IPCC) Working Group III Summaries for Policymakers and corresponding versions of Wikipedia’s "Climate Change Mitigation" (CCM) article. Using a comprehensive range of NLP techniques, including lexicometry, stylistic, readability assessments, modality analysis, semantic similarity (Sentence BERT), topic modelling (BERTopic), sentiment and emotion detection, and Named Entity Recognition, the study explores how CCM is portrayed across these two sources between 1990 and 2022. The findings show that IPCC SPMs consistently maintain a high level of technicality and Wikipedia, while initially more accessible and focused on events and personalities, has gradually aligned both semantically and stylistically with the IPCC. The analysis reveals no evidence of deliberate bias in Wikipedia’s representation of climate change mitigation. Instead, differences in framing and focus reflect its role as a publicly edited resource intended for a general audience. The study concludes that Wikipedia plays an important and evolving role in supporting public understanding of climate change, increasingly reflecting the IPCC’s scientific assessments. Keywords: Climate Change, IPCC, Wikipedia, Natural Language Processing, NLP, Analysis, Topic Modeling, Named Entity Recognition, Science Communication Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 i Acknowledgements This master’s thesis has felt like a journey. While this document represents the final tangible outcome of my work, the journey truly began in 2023 when I enrolled in Technologies des Langues. From that point onward, I learned so much each day that I feel compelled to express my gratitude for every moment. Each day was a stepping stone towards the completion of this work. First and foremost, I am deeply grateful to my supervisor, Dr Pablo Ruiz Fabo, for his teachings, dedication and passion for natural language processing. Even though I asked far too many questions in class, he always took the time to answer every single one of them and never once appeared bothered. I also wish to thank Dr Amalia Todirascu and Dr Delphine Bernhard for the knowledge and support they shared with us. Alongside Dr Pablo Ruiz Fabo, they each played a crucial role in the programme, and the Master Technologies des Langues would not have been the same without them. It is also important for me to thank everyone else involved, from the individual course lecturers who did an excellent job to the reactive administrative staff, as each person contributed in their own way to this thesis. Even the coffee machine in the Patio’s cafeteria deserves a thank you. Thank you, coffee machine. However, as wonderful as the Université de Strasbourg may be as a place to study, it represents only one half of a much larger picture. What would we be without the support of friends and family? I’d like to thank my friends, especially my very good friend Antonin, and others who have come and gone from my life, but who, at one point or another, were there for me. Most importantly, I wish to thank my mother for her unwavering, never-ending support, as well as the people I consider family in Bischwiller and Diebolsheim. Thank you for believing in me. Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 ii Table of Contents Introduction ..................................................................................................................................... 1 1 Previous Works ...................................................................................................................... 3 1.1 Previous Works on the IPCC ....................................................................................... 3 1.1.1 Presentation .................................................................................................... 3 1.1.2 Previous Works ............................................................................................... 4 1.2 Previous Works on Wikipedia ...................................................................................... 6 1.2.1 Presentation .................................................................................................... 6 1.2.2 Previous Works ............................................................................................... 7 1.3 Section Summary: Previous Works ............................................................................. 7 2 Technical State of the Art ....................................................................................................... 8 2.1 Text Representation .................................................................................................... 8 2.1.1 Encoding ......................................................................................................... 8 2.1.2 Format ............................................................................................................. 9 2.1.3 Raw Text ......................................................................................................... 9 2.2 Corpus Availability ...................................................................................................... 9 2.2.1 The IPCC Reports ........................................................................................... 9 2.2.2 The Wikipedia Climate Change Portal ........................................................... 10 2.3 Statistical Analysis of Textual Data ........................................................................... 11 2.3.1 Implementing Tokenization ............................................................................ 12 2.3.2 The Python Programming Language ............................................................. 12 2.3.3 The spaCy Pipeline ....................................................................................... 13 2.3.4 Text Analysis Platforms ................................................................................. 14 2.3.5 What is Textometry? ..................................................................................... 15 2.3.6 Lexicometry ................................................................................................... 15 2.4 Classification ............................................................................................................. 18 2.4.1 Sentiment Analysis and Emotion Detection ................................................... 19 2.4.2 Tools for Sentiment Analysis ......................................................................... 19 2.5 Topic Modelling ......................................................................................................... 20 2.5.1 Tools for Topic Modelling .............................................................................. 20 2.6 Other Techniques ..................................................................................................... 20 2.6.1 Sentence-BERT and Cosine Similarity Score ................................................ 20 2.6.2 Soft-cosine Similarity ..................................................................................... 21 2.6.3 Named Entity Recognition ............................................................................. 21 2.7 Section Summary: Technical State of the Art ............................................................ 23 3 Corpus and Methods ............................................................................................................ 24 3.1 Corpus ...................................................................................................................... 24 3.1.1 Selected Article Revisions ............................................................................. 25 3.1.2 Corpus Size ................................................................................................... 25 3.2 Preprocessing ........................................................................................................... 26 3.2.1 Data Acquisition and Cleaning....................................................................... 26 3.2.2 Preprocessing and Linguistic Annotations ..................................................... 29 3.3 Methodology ............................................................................................................. 30 3.3.1 Lexicometry, Stylistic, and Readability Analysis ............................................ 30 3.3.2 Modality Analysis........................................................................................... 31 3.3.3 Semantic Similarity Comparison .................................................................... 33 3.3.4 Topic Modelling ............................................................................................. 34 3.3.5 Sentiment Analysis and Emotion Detection ................................................... 35 3.3.6 Named Entity Recognition Analysis ............................................................... 37 3.4 Section Summary: Corpus and Methods ................................................................... 39 4 Results and Discussion ........................................................................................................ 40 4.1 Lexicometry, Stylistic, and Readability Analysis ........................................................ 40 4.1.1 Initial TXM Exploration .................................................................................. 41 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 iii 4.1.2 Word Count ................................................................................................... 42 4.1.3 Lexical Diversity ............................................................................................ 42 4.1.4 Lexical Density .............................................................................................. 43 4.1.5 Relative Frequency of Function Words .......................................................... 44 4.1.6 Sentence Count, Length, and Word Length ................................................... 45 4.1.7 Readability .................................................................................................... 46 4.1.8 POS Tags Distribution ................................................................................... 47 4.1.9 TF-IDF scores ............................................................................................... 48 4.2 Modality Analysis ...................................................................................................... 50 4.2.1 Verb and Adverbs.......................................................................................... 50 4.2.2 Likelihood and Confidence ............................................................................ 52 4.3 Semantic Similarity Comparison ............................................................................... 53 4.4 Topic Modelling ......................................................................................................... 54 4.5 Sentiment and Emotions Analysis ............................................................................. 55 4.5.1 Sentiment Analysis with VADER ................................................................... 55 4.5.2 Transformer-based Sentiment Analysis ......................................................... 56 4.5.3 Emotion Detection ......................................................................................... 56 4.6 Named Entity Recognition Analysis ........................................................................... 57 4.6.1 SpaCy’s NER Results ................................................................................... 57 4.6.2 Climate Change NER Results ....................................................................... 58 4.6.3 Results Synthesis .......................................................................................... 59 4.7 Section Summary: Results and Discussion ............................................................... 59 Conclusion .................................................................................................................................... 60 References .................................................................................................................................... 62 Appendices ................................................................................................................................... 70 Appendix A: Libraries and Software Versions ...................................................................... 71 Appendix B: Documents, Links and GitHub ......................................................................... 72 Appendix C: [Python code] Extracting Text from a PDF File Using pypdf ............................. 74 Appendix D: [Python code] Text Extraction from Wikipedia .................................................. 75 Appendix E: [Python code] Preprocessing with spaCy ......................................................... 76 Appendix F: [Python code] Lexicometry, Stylistic and Readability Processing ..................... 77 Appendix G: [Table] Top 10 TF-IDF Scores per Document .................................................. 80 Appendix H: [Python code] Expression of Modality Processing ............................................ 81 Appendix I: [Python code] Semantic Similarity Processing ................................................... 83 Appendix J: [Python code] Topic Modelling Processing ....................................................... 85 Appendix K: [Python code] Sentiment and Emotion Processing ........................................... 87 Appendix L: [Python code] Named Entity Recognition Processing ....................................... 90 Appendix M: [Table] Complete spaCy NER Results ............................................................. 93 Appendix N: [Table] Complete Climate Change NER Results .............................................. 94 Appendix O: [Python code] Data Visualisation ..................................................................... 95 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 iv List of Figures Figure 1: IPCC Structure ....................................................................................................... 3 Figure 2: SpaCy NLP Pipeline ..............................................................................................13 Figure 3: Cosine similarity in a 2-dimensional space ............................................................21 Figure 4: Earliest Version Available of CC Mitigation (26th of June 2005) .............................24 Figure 5: Text size by Number of Words as Computed with TXM’s “Dimensions” Tool .........41 Figure 6: Evolution of Herdan's C Over Time ........................................................................43 Figure 7: Evolution of Ure's Lexical Density Over Time ........................................................44 Figure 8: Relative Frequency of Function Words Over time (reversed Y Axis) ......................45 Figure 9: Flesch Reading Ease Over Time ...........................................................................46 Figure 10: Flesch-Kincaid Grade Level Over Time ...............................................................47 Figure 11: Total Modal Verb Frequency (per 1000) Over Time .............................................50 Figure 12: Total Modal Adverb Frequency (per 1000) Over Time .........................................51 List of Tables Table 1: spaCy Named Entities Categories Available in Their Models. .................................22 Table 2: Selection of Climate Change Mitigation Article Revisions from Wikipedia ...............25 Table 3: Token Count of the Raw Text WG3 SPMs from AR1 to AR6 According to Gemini..27 Table 4: IPCC Confidence Terminology ...............................................................................32 Table 5: IPCC Likelihood Terminology .................................................................................33 Table 6: Document Pairs for Semantic Similarity Comparison ..............................................33 Table 7: Climate-Change-NER Categories ...........................................................................39 Table 8: Token Count per Document Obtained with spaCy ..................................................42 Table 9: TTR, Herdan’s C, Guiraud’s R ................................................................................42 Table 10: Ure's Lexical Density per Document .....................................................................43 Table 11: Relative Frequency of Function Words per Document ..........................................44 Table 12: Sentence Count, Length and Word Length per Document ....................................45 Table 13: Readability Scores Based on the Flesch Reading Ease (FRE, 1948) and its Later Development, the Flesch-Kincaid Grade Level (FKGL, 1975) ...............................................46 Table 14: Distribution of Relevant POS Tags .......................................................................47 Table 15: Total Modal Verb/Adverb Freq. per Text ...............................................................50 Table 16: "Likely" Adverb Normalized Frequency per Text ...................................................52 Table 17: Total Likelihood and Confidence Frequencies per Text ........................................52 Table 18: Average and Median Document Similarity Scores ................................................53 Table 19: Topic Distribution for Each Text According to BERTopic.......................................54 Table 20: VADER Sentiment Analysis Results .....................................................................55 Table 21: RoBERTa Sentiment Analysis Results ..................................................................56 Table 22: RoBERTa Emotion Detection Results ...................................................................56 Table 23: spaCy Relevant NER Categories ..........................................................................57 Table 24: Relevant CC NER Categories ...............................................................................58 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 v List of Abbreviations • API: Application Programming Interface • AR: Assessment Report • CC: Climate Change • CCM: Climate Change Mitigation • CLT: Construal Level Theory • EPPM: Extended Parallel Process Model • FKGL: Flesch-Kincaid Grade Level • FLE: Flesch Reading Ease • GHG: Greenhouse Gas • HDP: Hierarchical Dirichlet Process • IPCC: Intergovernmental Panel on Climate Change • KWIC: Key Word in Context • LD: Lexical Density • LDA: Latent Dirichlet Allocation • LLM: Large Language Model • NER: Named Entity Recognition • NLP: Natural Language Processing • OCR: Optical Character Recognition • PDF: Portable Document Format • POS: Part-of-Speech • RAG: Retrieval Augmented Generation • RFFW: Relative Frequency of Function Words • SBERT: Sentence-BERT • SPM: Summary for Policymakers • TF-IDF: Term Frequency-Inverse Document Frequency • TTR: Type-Token Ratio • WG: Working Group Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 1 Introduction The Earth, our beautiful planet, is in deep trouble. Centuries of industrial activity have pushed it to the brink of collapse. The climate is changing, causing irreversible damage to ecosystems, including, but not limited to, species extinction. We, as humans, are the primary cause of climate change (CC), and we now risk the extinction of our own species: humankind. Of course, we are not responsible for this extinction as individuals, but as a society. A single person cannot put an end to CC, but a collective effort could influence the policymakers to regulate industrial activity as part of climate change mitigation (CCM). To document and provide humankind with a state-of-the-art overview of knowledge related to CC, the IPCC (Intergovernmental Panel on Climate Change) was created under the impulse of the United Nations in 1988. In over 30 years, the IPCC has produced six assessment reports (AR), synthesizing the risks, impacts, and mitigation strategies for CC (IPCC, 2024a). Each AR consists of contributions from three working groups, each focusing on different aspects of CC. The ARs are highly technical and complex documents, making them a challenging read for non-specialists. However, they are “… widely used by policymakers, scientists and other experts …” (IPCC & WMO, 1992, p. vii) to take action on CC. Consequently, despite being published in multiple languages, these works are rarely read by the public, who instead tend to rely on other sources for information on CC. Given that Wikipedia is the 8th most visited website in the world (Similarweb.com, 2025), we can assume that it plays a significant role in informing the general public: Wikipedia hosts an entire portal, or central page, dedicated to CC in multiple languages. Considering that these two sources are read by distinct audiences yet cover the same topic, studying the differences could yield valuable insights. This thesis aims to compare the IPCC reports and the Wikipedia portal on CC using Natural Language Processing (NLP) tools, with the goal of detecting potential differences or biases in their portrayal of CC. The analysis will focus exclusively on the English-language versions of both sources, as Wikipedia is significantly more developed in English (Wikimedia Foundation, 2025), offering a more comprehensive CC portal for comparison. The IPCC reports and the Wikipedia CC portal are both extensive resources, a direct comparison between them is not effective due to significant structural differences. Their vastly different formats and content structures mean that information is organized, detailed, and communicated in distinct ways. A direct comparison could lead to content misalignment that could distort or falsify results. Instead, we found it to be more effective when aligning specific Wikipedia articles with corresponding sections of each report, particularly regarding controversial topics. The thesis is structured as follows: Section 1, “Previous Works”, provides a review of the existing literature. Section 2, “Technical State of the Art”, covers fundamental NLP concepts and tools. Section 3, “Corpus and Methods”, details the corpus and research methodology. Finally, Section 4 “Results and Discussions”, presents and interpret the findings. As an overview of the thesis, the rest of this introduction contains a detailed outline of the content of each of the sections. Section 1, “Previous Works” reviews the established research relevant to the study. It begins with an overview of the IPCC including analyses of its Assessment Reports (ARs), communication strategies, and the criticisms it has encountered concerning factual accuracy, procedural integrity, epistemic representation, and ontological limitations according to De Pryck & Hulme (2022). It also considers how the IPCC is portrayed in various forms of media. Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 2 Subsequently, this section reviews literature on Wikipedia, focusing on its development, quality control mechanisms, and its function as a public source of scientific information, especially in relation to CC. The section also explores Wikipedia’s role in NLP research and existing studies examining how it frames complex topics such as CC. Section 2, “Technical State of the Art”, outlines the theoretical and practical foundations of the NLP methods employed in the research. It introduces essential concepts such as text representation, including encoding and formatting, and describes methods of data acquisition and conversion from sources such as PDF reports from the IPCC and web pages from Wikipedia. Then, the discussion turns to techniques in statistical text analysis, tokenisation, and preprocessing. Alongside text analysis platforms, core programming tools are introduced with Python recognised as the most suitable language for NLP, including key libraries such as spaCy, NLTK, scikit-learn, BERTopic and sentence-transformers. The section concludes with a detailed account of the specific NLP techniques used in the study. Section 3, “Corpus and Methods”, outlines the study’s methodology. It begins by defining the corpus, which includes all IPCC Working Group III Summaries for Policymakers (From ARs 1-6) and different historical revisions of Wikipedia’s “Climate Change Mitigation” article. The section then explains the processes of data acquisition and cleaning, involving tools such as pypdf, regular expressions, and Large Language Models (LLMs). It then describes the preprocessing pipeline implemented using the spaCy library, followed by an explanation of how each NLP technique is applied to the corpus. These methods include lexicometric analysis, readability scoring, modality extraction, semantic similarity computation, topic modelling, sentiment and emotion classification, and Named Entity Recognition (NER). Section 4, “Results and Discussion”, presents the findings of the comparative analyses and provides a comprehensive interpretation of the results. For each NLP technique applied, it systematically compares the IPCC documents with the corresponding Wikipedia articles. The analysis highlights both similarities and differences, examining these patterns in relation to the nature of each source, its intended audience, and the way in which they influence the framing and communication of climate change mitigation. It also identifies areas of convergence and divergence between the two platforms, to assess the extent to which Wikipedia reflects the original source material. Through this approach, the thesis examines how the critical issue of climate change mitigation is represented on two influential yet contrasting platforms. By applying a range of NLP techniques, the research offers new quantitative insights into differences in framing, emphasis, and potential bias. In doing so, this thesis seeks to deepen our understanding of how scientific knowledge about climate change is communicated and reshaped in the public sphere. Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 9 encoding refers to the interpretation of text characters by a system, If the system misinterprets the bytes, the result will appear garbled or incorrect. For example, the Windows operating system has used Unicode encoding since Windows 2000, but it wasn't fully adopted until the mid-2000s: In 2008, HTML5 recommended UTF-8 as the default character encoding (Hickson & Hyatt, 2008). In NLP, encoding is not the primary concern since it does not directly relate to the linguistic or semantic content of speech or text. However, it should not be completely disregarded, as some software may lack Unicode support, leading to potential information loss. In the case of this thesis, it will not be an issue and won’t be discussed further. 2.1.2 Format Format refers to the structure or arrangement of data. While encoding applies specifically to characters and symbols, format is a much broader concept. The simplest format for text is what we call “raw text” which is text as we write and read it, without any formatting. Raw text is typically stored in .txt files and lacks any structure, which can make it harder to read. Web browsers, on the other hand, use the HTML format, a markup language composed of raw text, HTML tags and metadata that displays information on a screen and facilitates navigation. Although raw text can be regarded as a “clean slate” in terms of formatting which is already high in informational content, additional tags and metadata can enhance context by indicating titles, paragraphs, sections, colours and other elements depending on the format, but are not always relevant for NLP, and can obstruct text processing software. 2.1.3 Raw Text In text processing, metadata, literally data about data (Merriam-Webster, 2024) can refer to elements like Part-of-Speech (POS) tags in a tokenized text, sentiment labels, or even grammar rules (Lebart & Salem, 1994). However, in formats like HTML or PDF, metadata often consists of technical information for software that is largely irrelevant for NLP tasks. While the colour of a page and the format in which text is displayed can be crucial for interpretation and are often designated by tags or metadata, one could argue that this is less relevant when analysing technical documents, such as IPCC reports. Therefore, the ideal format to begin the analysis is simple raw text, as it is compatible with NLP tools and free from any additional data that could affect the results. Of course, raw text will be pre-processed and tokenized for certain tasks, but the first step should be to acquire the IPCC reports and Wikipedia articles in a clean, metadata-free raw text format. This subsection reviews the original format of the reports and articles we intend to analyse and compare, detailing the steps required to convert them to raw text. 2.2 Corpus Availability This subsection reviews the original format of the reports and articles we intend to analyse and compare, detailing the steps required to convert them to raw text. 2.2.1 The IPCC Reports The IPCC reports are distributed on the IPCC website 5 as PDF. The Portable Document Format (PDF) is a widely used document type compatible with most browsers. PDFs are known for their unique features and versatility, as they retain their original formatting across different platforms. They can include text, images, tables, hyperlinks, and most media types, 5 Accessible on www.ipcc.ch Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 10 but the primary goal of a PDF is to preserve document layout, not to allow edition. To achieve this goal, PDFs are generated by specialized software, such as Microsoft Word or LaTeX, and store information as different types of “objects”, making it challenging to edit or extract raw text directly, as the text is not stored in a continuous, flowing format but rather as individually positioned elements (Adobe®, 2006). The main challenge with automatic text extraction from PDFs is accurately separating body text from non-body elements, such as headings and footnotes. To extract text from the IPCC reports, one could manually copy and paste each title and paragraph into a .txt file. However, this approach is highly time-consuming and impractical given the length of the IPCC reports. Most of the reports (AR1 to AR5) are already available as raw text due to previous works such as the conversion from Biros et al (2021). In the case of this thesis, we already are in possession of those texts thanks to our supervisor, Dr Pablo Ruiz Fabo, who previously worked with NLP on the IPCC reports. As such, only the conversion of AR6 will be necessary, which is a less time-consuming task if done manually. As will be detailed in the related methodology subsection (3.2.1), the solution we decided to go with in order to obtain raw text for the IPCC reports is the Python library pypdf (Fenniak & Martin, 2024), which allows users to edit PDFs and extract text. While it supports basic text extraction, its distinctive feature is a “visitor” function that can filter out headers and footers based on their position (y-coordinates). In our case, we chose to use pypdf for basic text extraction and then applied a LLM to filter out headers, footers, and other unwanted content. This technique is detailed in Section 3.2.1 2.2.2 The Wikipedia Climate Change Portal The acquisition of textual data from online sources such as Wikipedia, which is typically presented in HTML format, requires conversion into raw text for use in NLP applications. Various methods can be employed for this task, each with its own set of considerations. One common technique is web scraping, which involves automatically downloading and parsing HTML pages. Libraries such as BeautifulSoup 6 , commonly used in Python, are employed to remove boilerplate content including metadata, special tags, navigation menus, and other redundant elements. This process helps isolating raw text for an analysis. Although effective, large-scale scraping can place a considerable burden on web servers. To reduce server load and support more structured data retrieval, many platforms, including Wikipedia, offer Application Programming Interfaces (API). The Wikimedia API enables the download of articles in different format. When acquiring a large number of articles, complete database dumps, distributed by Wikipedia, are convenient and encouraged. However, for obtaining a smaller and more specific set of articles, APIs are generally more appropriate. Access to the Wikipedia API can vary, it allows a limited number of unauthenticated requests, but more extensive use typically requires account registration and the use of an API key. Wikipedia also provides enterpriselevel services, offering higher request quotas and additional functionalities. Interaction with APIs and extraction of text can be carried out using a range of tools and programming languages. Command-line utilities such as cURL are commonly used, particularly in Linux environments, where they are often pre-installed on Ubuntu distributions (Canonical Ltd, 2024), but they are also available for other operating systems, including Windows. Scripting languages such as Python, PHP, and JavaScript are widely employed for 6 BeautifulSoup, developed by Leonard Richardson, is available on pypi.org/project/beautifulsoup4/ Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 11 interacting with APIs. Python is particularly popular within the NLP community due to its simplicity and the availability of comprehensive libraries. (Nugues, 2024, p. 9) When data is retrieved via an API, it is typically delivered in a structured format, most often JSON. On Wikipedia, main textual content is found within the “extract” field in the JSON response from Wikipedia’s API. Standard libraries within the chosen programming language, such as Python's “requests” 7 library for handling HTTP requests and its effectively named “json” library for JSON parsing, can be used to access and process the data. The resulting text is then converted into plain text and saved as a .txt file. Using these techniques allows for the conversion of web-based corpora into raw text, making them suitable for further analysis with NLP. 2.3 Statistical Analysis of Textual Data This section provides an overview of various aspects involved in the statistical analysis of textual data relevant for this thesis. To guide the reader through the complexity of the subject, we will begin by establishing a foundational understanding, summarising key principles and historical perspectives. This initial exploration covers crucial concepts including text segmentation, unit forms, and the significance of preprocessing (2.3.1). Building on these fundamentals, we then examine key programming environments and tools — such as Python — that are widely employed to carry out these analytical tasks (2.3.2 and 2.3.3). The section subsequently introduces Text Analysis Platforms, which offer user-friendly tools for working with large text corpora (2.3.4). Finally, we describe research domains such as Textometry (2.3.5) and related methods like Lexicometry (2.3.6), outlining their objectives and applications. The Statistical Analysis of Textual Data is a broad field of research that has been developed since the 1950s. The subjects that significantly contributed to its development are Linguistics, Mathematics, Statistics, and Computer Science. Over time, the methodologies have been refined, and the applications have been enriched with new proposals from the segmentation of texts to the development of linguistic resources, the creation of lexicons, concordance analysis, text classification, sentiment analysis. The fields of application are the most varied, ranging from Psychology to Sociology, Marketing, Economics, Medicine, and Politics (Iezzi et al., 2020) The fundamentals of textual data analysis are summarised and expanded upon “Statistique Textuelle” by Lebart & Salem (1994). According to them, the statistical method, by definition, relies on measurements made from objects we count and combine. This means we must consider these segmentation objects as equal in relevance but distinct in use. In the context of text segmentation, units such as words should be segmented appropriately based on the analysis objectives. As Lebart & Salem (1994) point out, a chemist might want to neutralize linguistic plurality by treating “acid” and “acids” as equivalent, capturing both forms under a single search through lemmatization. Conversely, a linguist might prefer to preserve and distinguish these variations, as the morphological differences carry significant meaning in linguistic studies. To ensure accurate representation, we must consider the appropriate segmentation level and unit form for our analysis. Segmentation levels range from words, sentences, paragraphs, and even larger structures like sections or thematic units, each providing unique insights. 7 The Requests library, created by Kenneth Reitz, is available on pypi.org/project/requests/ Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 12 Breaking text down into smaller units is known as tokenization: this process creates what we usually refer to as tokens, which can be words, multiple words, subwords, or even individual characters. Tokens can retain positional information if required. We could segment the text by words, treating each word as a segment. In this format, the words become unordered, which is commonly referred to as a Bag-of-Words approach. If needed, we could tokenize each word segment into smaller tokens, such as morphemes (subwords) or graphemes (individual characters). Tokenization is part of the preprocessing phase in NLP, other common preprocessing tasks are, but not limited to: • Stop words removal (common words like “the”, “and”, “to” that create noise in the analysis) • Lowercasing (standardising text to one case) • Special character removal (such as punctuation, or replacement of characters like “é”, carriage return and new line characters (also known as CRLF). In addition to tokenization, the unit form we choose influences the analysis. Graphical forms treat each word exactly as it appears, maintaining morphological detail, while lemmatized forms group words by root meaning. Semantic forms groups words that share the same meaning, even if they are lexically different (Lebart & Salem, 1994). Semantic forms are often based on a controlled vocabulary like a thesaurus. Tokenization for pre-trained language models and LLMs often differs from classical, wordbased approaches, typically employing sub-word tokenisation algorithms such as WordPiece (Wu et al., 2016) where the units are statistically derived from the training corpus (Devlin et al., 2019). 2.3.1 Implementing Tokenization In practice, tokenization can be either straightforward or challenging, depending on the objective. For example, one could use Python’s “.split()” function, which divides a string (sequence of characters) into multiple substrings at each whitespace. However, this approach does not handle punctuation or special cases like “I’m” which represents “I am” and is technically two words but will be counted as a single token if split only on whitespaces. To tokenize, we employ much more complex “tokenizer” software that solves the tokenising problem for us. Advanced and customisable tokenizers already exist, such as those provided by the NLTK or spaCy libraries in Python. While developing our own tokenizer is possible, it would be a highly complex undertaking, potentially requiring several weeks of work, and would constitute a separate project beyond the scope of this thesis. Therefore, it is more efficient to use the previously mentioned tools instead. 2.3.2 The Python Programming Language Python is a high-level, interpreted programming language created by Guido van Rossum and first released in 1991. 8 Python was designed with an emphasis on code readability and simplicity, (T. Peters, 2024) making it an accessible language for beginners. Code readability also benefits experienced programmers, as they often need to read and understand each other's code on larger projects. 8 According to Python’s “History and Licence” documentation available on docs.python.org/3/license. Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 13 As an interpreted language, Python is compiled at runtime by an interpreter rather than a compiler, allowing for immediate code execution without the need for a separate compilation step to convert handwritten code into a binary file which is the case for the C programming language, for example (Python Software Foundation, 2024). This means code is stored as plain text in “.py” files, making it easy to share and read. However, this approach incurs a significant performance cost for large applications (Ziogas et al., 2021), which is why extensive desktop applications or video games are typically not written in Python. Of course, machine learning applications are among the most resourceintensive, and large language models (LLMs) cost millions to operate, yet they are still primarily run using Python. In these cases, Python serves as a scripting language, while its libraries for data analysis and machine learning such as NumPy, pandas, scikit-learn, TensorFlow, PyTorch and others (Nagpal & Gabrani, 2019) are transmitted to faster languages backends like C, C++ or CUDA to optimize performance. For example, CuPy (Tokui, 2024) is a Python library that enables Python to employ graphical processing units (also known as GPU) for NumPy and SciPy. Python was not always the obvious choice for NLP. Perl was once preferred due to its “rich regular expression and a support for Unicode.” (Nugues, 2024, p. 9). Python eventually adopted these features as well and now hosts most of the major libraries for NLP, along with support for machine and deep learning, which is a very solid environment to study natural languages. As an additional note, the R language is also a strong contender for data analysis, with libraries such as tm (text mining) and quanteda for NLP, or Stylo for stylometry. However, Python seems to have gained greater importance as neural networks and the transformer model emerged. 9 2.3.3 The spaCy Pipeline In spaCy, a Python library for NLP, the pipeline refers to an ordered sequence of NLP tasks. While it includes pre-processing steps (labelled as “tokenizer”), it consists of both preprocessing and processing tasks. The processing pipeline can be customized as desired; the one depicted in Figure 2 includes a part-of-speech tagger, a dependency parser, and a named entity recognizer. In the case of our analysis, once the texts have been processed, metrics on NER, part-of-speech tagging, and other tasks can be evaluated and compared. 9 Hyperlinks to the libraries referenced in this section: NumPy, pandas, scikit-learn, TensorFlow, PyTorch, CuPy, SciPy, quanteda, Stylo. Figure 2: SpaCy NLP Pipeline Source: spacy.io/usage/processing-pipelines Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 14 2.3.4 Text Analysis Platforms While spaCy is a powerful tool, it is not ideal for some research-oriented tasks and exploratory visualisation. Text analysis platforms have been developed specifically to enable the exploration of linguistic patterns across corpora. For this reason, we cover such platforms in this section, and we tested the contribution of one of them to the research question in the thesis (see 4.1.1 for the results). Numerous software tools have been developed to facilitate the statistical analysis of text. Some are available as desktop applications while others are accessible online using a web browser. Pincemin (2018), proposes a list of software tools for conducting such analyses. I explored some of them and added others to create the list below: Developed by Université Nice Côte d’Azur, Hyperbase 10 was developed with a focus on French-language, but most of the features (word frequency, collocations, KWIC, concordance) are language-agnostic and English corpora can be used. Previous versions of Hyperbase were desktop applications, while the latest version is accessible online through a browser. Previous versions had more features as Hyperbase web as “In 2015, Hyperbase was released in a web version with a new interface that retained the main features of the original version” (Hyperbase, 2024). Unfortunately, the previous versions appear to be inaccessible. IRaMuTeQ 11 is an R and Python based software tool for performing multidimensional text analyses. It primarily analyses corpus themes and offers visualization of thematic classes according to the “Reinert classification method”, a statistical text analysis technique designed to identify and classify the main themes present in a corpus by grouping text segments such as sentences or paragraphs according to the co-occurrence of words within them (Reinert, 1983). IRaMuTeQ is available in multiple languages and can analyse English texts but only if they are properly preprocessed. Fortunately, IRaMuTeQ allows imports from TXM, which can assist with preprocessing. Le Trameur (Fleury & Zimina, 2014) is a tool designed for the analysis of structured and annotated texts. It segments texts into “Trame”, which refers to a sequence of positions within the text, and “Cadre”, representing the partitions or frames that structure the text. The software supports regular expressions and specializes in analysing aligned corpora, including co-occurrence and concordance analyses, with elements of correspondence analysis and visualisation. TXM (Heiden et al., 2010) is an open-source project primarily developed by the ICAR research lab in France. It aims to provide a comprehensive platform for corpus analysis, offering tools for both quantitative and qualitative text analysis, including statistical analysis, visualisation, and textual exploration. Users can perform frequency analyses, identify cooccurrences, cluster words, and map thematic patterns within a corpus. It allows for text and of corpora from various formats, including raw text, and offers customisation and scripting options for advanced users. AntConc (Anthony, 2005) can be essentially seen as a beginner-friendly version of TXM, as it covers TXM’s basic functions. However, Antconc is unrelated to TXM and was released in 2002, which is almost 10 years earlier. It handles plain text, supports regular expressions, and can perform essentially the same tasks as TXM (minus some visualisation techniques) while offering a comprehensive interface and ease of use. 10 Accessible on hyperbase.unice.fr 11 Accessible on www.iramuteq.org Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 15 Sketch Engine (Kilgarriff et al., 2014) is one of the most accessible platform for text analysis. It is an online platform accessible via any browser and provides textometry tools as well as pre-made corpora. Users can import their own texts and expand their corpus through web scraping performed by the platform itself. Sketch Engine offers additional features, such as Word Sketch, which automatically generates a summary of a word’s collocations, and a Thesaurus, however, it is a paid tool. The advantage Sketch Engine has over other platforms is its accessibility and the exceptional user experience it offers for beginners. In my case, it seems unnecessary. 2.3.5 What is Textometry? According to Pincemin & Heiden (2008), Textometry was essentially developed in France during the 1970, it “applies a wide range of linguistically significant and mathematically based calculations for methodical and renewed analysis of text collections.”. Textometry borrows techniques from multiple fields and applies them to linguistic content to uncover information, identify patterns, and reveal relationships within texts. Textometry can be a broad subject, typically including sentiment analysis, stylometry, or topic modelling. However, platforms like TXM focus solely on lexical analysis. We will start by reviewing the lexical analysis techniques. 2.3.6 Lexicometry Lexicometry is the quantitative study of vocabulary within a text. It measures linguistic features through statistical methods, with the usual measurements as follows: Word Frequency list: calculates the frequency and rank of each word in a text or corpus. Type-Token Ratio (TTR): TTR is shown in Equation 1. It measures lexical richness by dividing the number of unique words V (Types) by the total number of words N (Tokens) in a text. The higher the TTR, the greater the lexical diversity. However, TTR has a known limitation: it decreases as text length increases. Variations of TTR that are less sensitive to text length do exist, but according to Lissón & Ballier (2018), the extent to which they effectively neutralize the influence of text length remains unclear. Additional measures can be employed such as Herdan’s C, which is less sensitive to changes in length. Herdan’s C is shown in Equation 2: 𝐶=log𝑉 log𝑁 Equation 2: Herdan’s C Source: Lissón & Ballier (2018, p. 9) TTR=𝑉 𝑁 Equation 1: Type-Token Ratio Source: Lissón & Ballier (2018, p. 9) Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 16 Guiraud’s R is useful for comparing lexical richness across texts of different lengths as it accounts for text length by normalizing the count of unique words against the square root of the total number of words. Guiraud’s R is shown in Equation 3: Yule’s K focuses on repetition a high score indicates high level of repetition, which suggest lower lexical diversity and vocabulary (Lissón & Ballier, 2018). Yule’s K is available in Equation 4: 𝑁 is the total number of words. 𝑣 is the total number of distinct frequency classes 𝑖 is a word frequency class 104 is a scaling factor. The Lexical Density (LD) is the proportion of lexical words or content words (opposite of function words) divided by the total words in a text. Higher LD indicates that information is more densely packed, suggesting greater textual complexity. While not a perfect measure on its own, academic texts demonstrate higher LD. Equation 5 is Ure's (1971) LD: While Ure focuses on a word-level ratio, there is a different approach to lexical density: Equation 6 is Halliday's (1989) clause based approach: 𝐿𝐷=𝐿𝐶 Equation 6: Lexical Density (clause based) Source: Halliday (1989) 𝐿𝐷=𝑁𝐿𝐸𝑋 𝑁𝑇𝑂𝑇𝐴𝐿 Equation 5: Lexical Density Source: Ure (1971, p. 445) 𝐾=104×[−1 𝑁+∑𝑓𝑣(𝑖,𝑁)(𝑖𝑁)2 𝑣 𝑖=1 ] Equation 4: Yule’s K Source: quanteda.io/reference/textstat_lexdiv 𝑅= 𝑉 √𝑁 Equation 3: Guiraud’s R Source: Lissón & Ballier (2018, p. 9) Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 17 L represents the number of lexical items (nouns, lexical verbs, adjectives, adverbs). C refers to the number of clauses. To clarify, a clause is a group of words that contains at least a subject and a verb. Distribution of Word Classes: An examination of the distribution of word classes (nouns, verbs, adjectives, adverbs) reveals that different genres exhibit distinct distributions. For instance, scientific texts, such as IPCC reports, may contain a higher proportion of nouns. Lexical Dispersion: An examination of the spread or distribution of words across a text. Patterns of distribution may carry stylistic or thematic significance and are typically visualized using a dispersion plot. Cluster Analysis: Grouping of similar words, phrases, or documents based on previously mentioned linguistic features. N-gram Analysis: Analysis of sequences of “n” words (where “n” is an unknown) that appear together in a text. Collocation and Co-occurrence: Analysis of words that frequently appear together. Collocation is usually larger, while co-occurrence focus on smaller groups such as pairs. Key Word in Context (KWIC): A technique that displays a keyword within its immediate textual environment, showing a predefined number of words on either side. This allows for the examination of the keyword’s context. Specialized software typically allows users to click on the keyword to navigate to the actual text. Readability levels: Readability measures the ease with which a reader can comprehend a written text. In our context, assessing readability is crucial for quantifying expected differences in linguistic complexity. Typically, readability formulae provide a score based on characteristics that can be quantified, such as: • Sentence Length, as longer sentences are assumed to be harder to process syntactically. • Word Complexity, as longer words are often associated with lower frequency and greater difficulty. However, readability doesn’t assess conceptual difficulty or ambiguity. The most commonly known formula is the Flesch Reading Ease (FRE), introduced by Flesch (1948). It scores readability from 0 to 100, and higher scores indicates easier readability. Simply put, the formula is a weighted average of sentence length and average syllables per word, combined and converted into a score. The Flesch-Kincaid Grade Level (FKGL) is an evolution of Flesch’s work, developed for the U.S. Navy by Kincaid et al. (1975). Like the previous iteration, it uses average syllable and sentence lengths, but with updated weightings and a different scoring system that results in a U.S. school grade level instead of a score out of 100. The FKGL is still in use today, for example in legal texts. (Han et al., 2024) Stylistic Markers: Linguistic features that contribute to the unique style of a text. Common features that can be calculated are: • average sentence length • average word length • frequency of function words • Usage of specific grammatical structures Technical State of the Art Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 18 These features can contribute to the stylometric analysis of a text, for tasks such as identifying the author. Visualization Techniques: A visual representation of the linguistic features of a text. Certain linguistic features are more easily identified in a 2D space. Different representations exist for different measurements: • Word cloud: (frequency) • Heatmap: (co-occurrence patterns) • Dispersion plots (lexical spread) • Histogram (frequency distribution) 2.4 Classification Classification is the process of assigning predefined categories or labels, such as genre, to data. In NLP, this data can range from words to entire texts. It is regarded as a supervised machine learning technique, as it relies on labelled data to train a model that can assign appropriate labels to future, unseen data segments. For this thesis, classification does not appear to be particularly effective, as the focus is on comparing texts, a task unrelated to classification. However, sentiment analysis, which is a classification task, offers an interesting approach for comparison. Analysing the sentiment of sentences, paragraphs, and entire texts could provide valuable insights. The factors that a classification model uses to make labelling decisions are known as features. These features are extracted from the segments and may include statistical properties leveraged from lexicometry, such as word frequency. The following are the most common features utilized in classification: Word Frequency (Bag of Words): Simply counts the occurrences of each word in the text, ignoring grammar and word order. N-grams: Considered a more extensive version of the bag-of-words approach, N-grams are used to capture local context. By adjusting the value of N, we can control the amount of context captured. A single-word N-gram is referred to as a unigram, two words form a bigram, and three words make a trigram. N-grams are particularly important for sentiment analysis, as they preserve context. For instance, a sentence containing the words “not” and “happy” might be processed separately, leading to an entirely different sentiment interpretation without contextual information. Term Frequency-Inverse Document Frequency (TF-IDF): balances term frequency within a document against its uniqueness across documents to assign weights to words based on their distinctiveness. It typically eliminates stop words, provided they have not already been removed during preprocessing, as well as domain-specific terms like “climate” in the case of IPCC reports. While “climate” is undeniably significant, its frequent occurrence makes it an expected result, whereas TF-IDF is designed to allow more distinctive patterns to emerge. Part-of-Speech Tags: Identifies grammatical roles of the words for a more semantic understanding. This way, nuances in style and argumentation can be uncovered. Syntactic Patterns: Measures sentence complexity using parse trees. It identifies the sentence structure (simple, compound, complex) and count the average number of clauses. More complex sentences suggest a more sophisticated style, which is typically the case with technical, academic documents. Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 25 3.1.1 Selected Article Revisions Given that Wikipedia articles are often edited multiple times per day, it is necessary to identify a specific, appropriate revision to compare with each AR. Typically, we searched for revisions published after the release of an AR, including its findings. However, this was not possible for the 2005 revision, for which we simply used a version published on the day of the article was created. Following this methodology, we found revisions may relate to IPCC findings, presented in Table 2. Links to their online versions on Wikipedia are available in Appendix B. Wikipedia article Corresponding AR Exact date chosen Reasoning CC Mitigation (2005) AR1 (1990), AR2 (1995), AR3 (2001) 26 June 2005, 23:39 (UTC) Late at night on the day of the article’s creation to ensure the author had finished editing CC Mitigation (2007) AR4 (2007) 7 May 2008, 10:15 (UTC) Addition of a citation from AR4 to define global warming, confirming that AR4 influenced the article’s content. CC Mitigation (2014) AR5 (2014) 2 September 2014, 07:44 (UTC) Major revision which added multiple references to AR5 CC Mitigation (2022) AR6 (2022) 13 June 2022, 22:16 (UTC) While AR6 WG3 was referenced in the article on the day of its release, the 13 June revision added a substantial amount of information from AR6. Table 2: Selection of Climate Change Mitigation Article Revisions from Wikipedia 3.1.2 Corpus Size The proposed corpus, consisting of the six WG3 SPMs and the corresponding versions of the CCM Wikipedia articles (when applicable), offers a focused dataset that facilitates comparison between the two entities and over time. While this selection enables precise comparison on the specific topic of climate change mitigation, it yields a corpus that may be too limited in size for certain NLP applications, such as topic modelling. Latent Dirichlet Allocation (LDA), for instance, is a topic modelling technique that performs optimally on large datasets (Härdle & Chen, 2016), as such, it wouldn’t work in this case. Although this limitation may seem like an oversight, it is in fact a deliberate methodological choice as it offers the following advantages: Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 26 • Controlled comparison: This selection ensures a meaningful comparison between functionally analogous documents, aimed at different audiences across consistent time periods. • Quality and depth: It allows for a more detailed qualitative and quantitative analysis of linguistic features, framing and semantic nuances. • Practicality: It maintains a manageable scope suitable for a master’s thesis, allowing sufficient time for potentially time-consuming processing steps, such as the Named Entity Recognition sub-categorisation (detailed in Section 2.6.3) Therefore, while we acknowledge certain limitations associated with specific techniques, the defined corpus is considered sufficient and appropriate for the primary aims of this thesis. The selection and interpretation of NLP methods has taken the corpus size into account and emphasis has been placed on analyses that are robust to medium-sized corpora, including detailed lexicometric comparisons, readability assessments, Sentence-BERT based similarity measures, and NER. For topic modelling, approaches such as BERTopic were prioritised over traditional methods, as pre-trained models generally perform more effectively than LDA on smaller datasets. The topic modelling methodology is detailed in Section 3.3.4, with results presented in Section 4.4. 3.2 Preprocessing The following sections (3.2 and 3.3) are structured to provide a comprehensive overview of the analytical pipeline, progressing from data acquisition to the execution of specific textual analyses. The Preprocessing section includes the following subsections: • Section 3.2.1, entitled Data Acquisition and Cleaning, describes the procedures employed to compile and standardise textual data from both IPCC reports and Wikipedia, thereby ensuring a comparable corpus. • Section 3.2.2, Preprocessing and Linguistic Annotations, which details the utilisation of the spaCy library for essential NLP tasks, including tokenisation, lemmatisation, and part-of-speech tagging. All subsections are presented with an explicitly stated “Objective”, followed by a thorough account of the tools and methodologies applied. Software and library versions are outlined in Appendix A. 3.2.1 Data Acquisition and Cleaning Objective: To obtain clean, comparable raw text from both sources. As previously mentioned, a total of 10 documents (6 for IPCC, 4 for Wikipedia) had to be collected and converted into raw text format to ensure compatibility with an NLP preprocessing pipeline. For the IPCC WG3 SPMs, while versions in raw text form existed online, such as the version from Biros et al. (2021), their edition did not fully meet my requirements. Specifically, we needed the WG3 SPM from each AR, and this document was not always included for each AR in their compilation. We were already in possession of five out of the six reports converted to raw text, thanks to my supervisor, Dr Pablo Ruiz Fabo, who had previously worked with NLP on the IPCC. However, the most recent report, AR6, released in Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 27 2022–2023, has no raw text conversion available online. To convert the report, we first downloaded the original PDF document from the official IPCC website, we then carried out the conversion using Python. The safest way to work with Python is to create a new virtual environment for each project, to minimise conflicts between dependencies and avoid unpredictable behaviour. While various software solutions offer this functionality, we found Python’s Anaconda distribution to be the most practical. We worked with Python 3.11.11, as using the latest or an older version is not always the best choice in programming, due to compatibility and security considerations. Python 3.11.11 seemed to be the most balanced choice in this regard. Using Python 3.11.11, we began by installing and using pypdf to extract raw text from the report. This approach was straightforward, and saving a new file with the extracted raw text required only 15 lines of code (Python code for PDF extraction is available in Appendix C). However, the issue with this method was that pypdf extracted all the text without cleaning it: many elements that could be considered unnecessary remained, such as metadata, authors, headers, footers, and figure captions. While several solutions exist to clean the file, one possibility in this case was to use a state-of-the-art LLM, due to the file’s length. LLMs can be exceptionally effective at a wide range of repetitive tasks, provided they are supplied with examples and a clear prompt. This approach is known as few-shot learning or few-shot prompting. In this instance, we already had examples: the cleaned SPMs from AR1 to AR5. Therefore, all that was needed was a well-crafted prompt and an LLM with a very large context window. As of April 2025, Google has released Gemini 2.5 Pro Preview 03-25 on their AI Studio. Google’s AI Studio is a platform for developers to test and deploy software solutions using Google AI models. Notably, Gemini supports an exceptionally large context window of 1,048,576 tokens. Token counts may vary depending on the task and model. In our case, token count for the raw text SPMs, as reported by Google's AI Studio, are listed in Table 3: AR1 WG3 SPM 21,024 tokens AR2 WG3 SPM 12,555 tokens AR3 WG3 SPM 12,603 tokens AR4 WG3 SPM 18,595 tokens AR5 WG3 SPM 23,461 tokens AR6 WG3 SPM (uncleaned) 65,508 tokens Table 3: Token Count of the Raw Text WG3 SPMs from AR1 to AR6 According to Gemini 2.5 Pro Preview 03-25 All reports totalled 153,746 tokens, meaning Gemini’s context window was more than large enough to support a few-shot learning approach. The conversion methodology with the LLM was the following: • The SPMs from AR1 to AR5 were sent to Gemini to establish examples context. • A custom prompt was then submitted, including the raw text of AR6, instructing Gemini to return a cleaned version in the same style as the previously provided files. Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 28 Thankfully, Gemini supports an output limit of 65,536 tokens, which was sufficient to return the entire cleaned document in a single response. This minimised the need for output segmentation and reduced the risk of truncation or context loss. Therefore, Prompt 1 was submitted to Gemini: You are an NLP computer scientist. The previously sent texts comprise the summaries for policymakers from Working Group III in Assessment Reports 1 to 5. They are formatted as raw text, without metadata, tables, headers and footers, document references, unnecessary line breaks, as these elements are not useful for NLP tasks. However, keep figures and tables captions. I am now sending you the Summary for Policymakers from the Sixth Assessment Report as a raw text file. Please clean the file in the same manner as the previous ones, as it has already been converted to text but still contains unnecessary information for NLP. Return the cleaned text. Prompt 1: Cleaning AR6 with Gemini 2.5 Pro After carefully reading the results, this approach returned a very satisfactory outcome. The text had been cleaned of unnecessary elements, and a few minor fixes had to be applied, in order to resolve formatting issues (spaces before commas and full stops, consecutive spaces, unnecessary line breaks). Although we could have proceeded to the next step, several additional cleaning procedures were applied across all SPMs. These included the manual removal of special characters (such as bullets “•”), the merging of hyphenated words that had been split across lines (e.g. “intellectual” in AR1–AR4), and the insertion of spaces between words in a few cases where they were missing (notably in AR4). We also addressed words fragmented by individual characters (e.g. “S T R A T E G I E S”), consolidating them into proper form (“STRATEGIES”) using Regular Expression 1: \b[a-zA-Z](?: [a-zA-Z])+\b Regular Expression 1: Detecting Fragmented Words in Raw Text While this was a time-consuming manual task, the small size of our corpus demanded the highest possible quality to ensure robust results. For larger corpora, recent advancements in the information retrieval field have led to the development of several tools that are highly effective at processing PDFs and preparing them for NLP applications. By combining multiple processing techniques, such as basic text extraction, OCR and layout identification, Docling (Auer et al., 2024) is also able to extract text from all types of PDFs along with images, tables, and charts. While pypdf and Gemini were sufficient for our task, Docling could be a valuable tool for future work, offering faster document conversion and the ability to extract information from data that was previously inaccessible to NLP. For Wikipedia articles, we used the requests library and Python code provided in Appendix D. We called Wikipedia’s API and retrieved the articles in JSON format and extracted the content to save it as raw text. However, the output included Wikipedia’s markup language, which is not compatible with NLP applications and had to be cleaned. Fortunately, the Python library mwparserfromhell 12 allowed us to parse and remove Wikipedia’s markup, cleaning a large part of the files before saving them. 12 Mwparserfromhell, developed by Ben Kurtovic, is available on pypi.org/project/mwparserfromhell/ Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 29 After a careful evaluation of the files, we still identified a few elements that needed to be removed, such as excessive line breaks and special formatting of figure captions. Fortunately, most captions were easy to detect, as they always contained the word “thumb” and one of several “|” symbols, which can be automatically identified using Regular Expression 2: (.+?)\| Regular Expression 2: Detecting Wikipedia Captions in Raw Text. However, we noticed later a few of these captions were not completely removed by that step, though this did not noticeably affect the analysis. There were also URLs that were manually removed. While we could have used regular expressions to locate them quickly, only a few appeared in the 2014 and 2022 versions of the report, and they were easy to remove manually. Finally, the earliest revisions contained some irrelevant words typical of Wikipedia, such as “See also:”, which were trivial to identify and remove. In the next section, we will discuss the preprocessing steps of those documents, a crucial step in NLP applications. 3.2.2 Preprocessing and Linguistic Annotations Objective: To prepare the text for deeper analysis by performing fundamental NLP tasks. Raw text documents will be processed using the spaCy Python library, employing its large English model: en_core_web_lg This Python library is multifunctional and will automatically perform the following tasks: • Sentence segmentation • Tokenization • Lemmatization • Part-of-Speech tagging (POS tagging) • Dependency parsing • Named Entity Recognition This process is non-destructive, spaCy is designed to handle all annotations and stores the results in a Doc object in Python. Specific components, such as NER, can then be extracted from this Doc object and either used as they are or further processed. Using spaCy as an initial step to generate multiple processed versions of the document was highly efficient and required very little code. Although we could have used different software for each task, such as tokenization, lemmatization, NER and so on, spaCy proved significantly faster, and its output was immediately usable by other Python components. Preprocessing with spaCy could still be time-consuming if it had to be repeated across multiple texts each time a script or notebook was run. Ideally, each text would be processed once, and the resulting Doc object saved for later use. Fortunately, spaCy provides a dedicated class, DocBin, which allows Doc objects to be saved to disk in a binary format, helping to avoid reprocessing large corpora. Technically, multiple Doc objects can be stored in a single DocBin object and saved to disk. This means the entire preprocessed corpus can be stored in a single file. However, due to an unknown issue encountered when handling a large DocBin object, we chose to store a single Doc per Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 30 DocBin object instead, resulting in one file per document on disk. This did not affect the analysis in any way. Code demonstrating how the documents were preprocessed and how the resulting DocBin objects were saved to disk is available in Appendix E. However, spaCy preprocessing was not always sufficient for certain tasks that required tailored treatment to produce meaningful results. For example, while lexicometric analysis typically involved stop-word removal for tasks such as calculating TF-IDF, sentence similarity using SBERT required the original form of the text to be preserved. For this reason, any additional preprocessing steps will be addressed at the beginning of each respective section under 3.3 below (Methodology), where we will specify whether spaCy’s output was used as a starting point and outline the further steps taken. 3.3 Methodology Methodology subsections (3.3.1 to 3.3.6) are each dedicated to a specific analytical approach. The analysis encompasses the following domains: • Lexicometric analysis and readability (3.3.1) • Modality analysis (3.3.2) • Semantic similarity assessment (3.3.3) • Topic modelling (3.3.4) • Sentiment and emotion analysis (3.3.5) • Named entity recognition (3.3.6) 3.3.1 Lexicometry, Stylistic, and Readability Analysis Objective: To quantify and compare fundamental textual characteristics such as vocabulary richness or readability. To get an initial view of the lexical features of our corpus, we began by using TXM, an opensource platform designed for corpus analysis. As we mentioned in Section 2.3.4, TXM includes a variety of tools for statistical analysis, visualisation, and detailed textual exploration. We used the latest TXM version, 8.4, released in February 2025. Although TXM supports the import of corpora in a variety of formats and allows for the addition of metadata, we imported only unprocessed raw text into the software. This was due to the limited metadata available and the fact that text exploration using TXM was not the primary focus of our analysis. • The TXM exploration results are available in Section 4.1.1. After this initial exploration with TXM, we used the texts preprocessed with spaCy to begin the analysis with a word count for each text, corresponding to a simple token count. To achieve this, we imported the saved Doc objects from disk using spaCy’s DocBin class for each document and counted the total number of tokens generated, including both the raw count and a count excluding stop words and punctuation marks. • Word counts results are available in Section 4.1.2. A different approach was used for assessing lexical diversity, as TTR, Herdan’s C, and Guiraud’s R were computed using lowercased and lemmatized tokens. Lemmatizing and lowercasing improve the reliability of vocabulary richness measures compared to using raw text. Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 31 • Lexical diversity results are available in Section 4.1.3. For Ure's (1971) lexical density, we used spaCy’s POS tags to identify nouns, proper nouns, verbs, adjectives, adverbs, and numbers, and divided them by the total number of tokens (excluding punctuation). Numbers are considered lexical items because they function as content words by conveying quantitative information that can be very relevant for technical documents. • Lexical density results are available in Section 4.1.4. Next, we calculated the Relative Frequency of Function Words (RFFW). RFFW can be viewed as a reversed form of lexical density. Technical texts typically have a lower RFFW, which corresponds to a higher lexical density. RFFW was calculated by grouping function words using spaCy’s POS tags to identify them and dividing their total by the overall word count (excluding punctuation). We considered the following POS tags as function words: “ADP” (adposition, which are prepositions and postpositions), ”AUX” (auxiliaries), “CCONJ” (coordinating conjunction), “SCONJ” (subordinating conjunction), “DET” (determiner), “PRON” (pronoun), “PART” (particle) and “INTJ” (interjection). While it is not particularly useful to compute both lexical density and RFFW, it was used to verify previously obtained information. • RFFW results are available in Section 4.1.5. While not particularly useful on its own, sentence count per text is based on spaCy’s sentence segmentation and reflects the total length of the texts. They are included here for reference only. Average sentence and word length per text were also calculated and provide insight into a text’s difficulty, as they form the basis for computing readability levels. • Sentence metrics are available in Section 4.1.6. Readability levels were calculated using textstat, a Python library dedicated to textual metrics such as “readability, complexity, and grade level” (Shivam & Chaitanya, 2014). We focused on calculating FRE (Flesch, 1948) and FKGL (Kincaid et al., 1975) metrics which are based on sentence and word length, as we previously described in Section 2.3.6. • Readability levels results are available in Section 4.1.7. POS Tags Distribution: during preprocessing, spaCy assigned a part-of-speech tag to each token, which enabled us to compute a probability score for each tag. • POS Tags Distribution results are available in Section 4.1.8. For the TF-IDF analysis, the documents were lowercased and stripped of punctuation and stop words before lemmatization. We used scikit-learn and limited the vocabulary to 2,000 terms (max_features=2000 in scikit-learn’s TfidfVectorizer) to reduce noise and exclude very rare terms unlikely to be significant across the corpus. TfidfVectorizer was fitted to the entire corpus, and for each document, a list of ten lemmatized words was then selected based on their TF-IDF scores. • TF-IDF results are available in Section 4.1.9. 3.3.2 Modality Analysis Objective: To analyse the expression of certainty and uncertainty based on the studies by Roeder (2011) and Herrando-Pérez et al. (2019) Modality is expressed through modal verbs and adverbs, as well as likelihood and confidence expressions. All these elements play a crucial role in effective CC communication. According to Roeder (2011) and Herrando-Pérez et al. (2019) The IPCC's use of modality and expressions of uncertainty reflects the scientific rigour and complexity involved in Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 32 generating its reports, as such choices inevitably shape how the public and policymakers perceive the urgency of climate change. For modality analysis, we first calculated the raw frequency of modal verbs and adverbs for each document in our corpus and compared their normalized frequency over time. The analysis used the previously pre-processed spaCy Doc objects (Section 3.2.2) and filtered modal expressions using a combination of POS tags and a curated list of verbs and adverbs. Likelihood and confidence expressions were also retrieved using a list borrowed directly from the IPCC. This approach is considered rule-based, as it does not take context into account, relying solely on simple word filtering and detection. For modal verbs, the curated list was the following: "can", "could", "may", "might", "must", "shall", "should", "will", "would". These are the core modal auxiliaries and their preterit version in English. For modal adverbs, the list is a little longer. In the Cambridge Grammar of the English language, (Huddleston & Pullum 2002, p. 102) the following list was adopted for our analyses: "apparently", "arguably", "assuredly", "certainly", "clearly", "conceivably", "definitely", "doubtless", "evidently", "hopefully", "indubitably", "ineluctably", "inescapably", "incontestably", "likely", "manifestly", "maybe", "necessarily", "obviously", "patently", "perhaps", "plainly", "possibly", "presumably", "probably", "seemingly", "surely", "truly", "unarguably", "unavoidably", "undeniably", "undoubtedly", "unquestionably". We filtered all verbs and adverbs by selecting tokens with the POS tags “AUX” for verbs and “ADV” for adverbs, then refined the selection using the previously presented lists of modal verbs and adverbs. Finally, we calculated the normalized frequency of each item per text. While it would have been possible to analyse the use of each modal verb and adverb individually, such a detailed analysis falls outside the primary scope of this master’s thesis. Instead, we opted to compute the total frequency of both categories as the basis for our comparison. • Results for modal verb and adverbs analysis are available in Section 4.2.1 Likelihood and confidence expressions are a specific category of standardized language introduced by the IPCC to convey degrees of certainty through quantified modalities. They are listed in Table 4 and Table 5 (Le Treut & Somerville, 2007) Confidence Terminology Degree of confidence in being correct Very high confidence At least 9 out of 10 chance High confidence About 8 out of 10 chance Medium confidence About 5 out of 10 chance Low confidence About 2 out of 10 chance Very low confidence Less than 1 out of 10 chance Table 4: IPCC Confidence Terminology 13 13 Original tables accessible on archive.ipcc.ch/publications_and_data/ar4/wg1/en/ch1s1-6 Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 33 Likelihood Terminology Likelihood of the occurrence / outcome Virtually certain > 99% probability Extremely likely > 95% probability Very likely > 90% probability Likely > 66% probability More likely than not > 50% probability About as likely as not 33 to 66% probability Unlikely < 33% probability Very unlikely < 10% probability Extremely unlikely < 5% probability Exceptionally unlikely < 1% probability Table 5: IPCC Likelihood Terminology Expressions of likelihood and confidence were retrieved using the entries in Tables 4 and 5, along with a simple rule-based text matching approach. • Results for likelihood and confidence modal expressions are available in Section 4.2.2. 3.3.3 Semantic Similarity Comparison Objective: To evaluate the semantic correspondence between paired IPCC and Wikipedia documents. As mentioned in Section 2.6.1, we assess the semantic similarity between the IPCC WG3 SPM and Wikipedia articles on Climate Change Mitigation (CCM) by encoding the texts as vectors using Sentence-BERT embeddings (Reimers & Gurevych, 2019) and applying cosine similarity to evaluate their degree of alignment in a multi-dimensional vector space. While we previously measured and evaluated each document to compare them collectively and track the evolution of both the IPCC and Wikipedia, directly comparing the IPCC documents to one another would have been of limited value, as semantic differences between them are expected. Instead, we focused on comparing pairs of documents: the IPCC SPMs and their corresponding Wikipedia articles from the same period, as shown in Table 6: AR3-WG3-SPM (2001) Wikipedia:Climate Change Mitigation (2005) AR4-WG3-SPM (2007) Wikipedia:Climate Change Mitigation (2008) AR5-WG3-SPM (2014) Wikipedia:Climate Change Mitigation (2014) AR6-WG3-SPM (2022) Wikipedia:Climate Change Mitigation (2022) Table 6: Document Pairs for Semantic Similarity Comparison Corpus and Methods Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 34 As the first two IPCC WG3 SPM from 1990 and 1995 do not have corresponding Wikipedia articles, they are not included in this analysis. There are various methods for computing semantic similarity. Some are more qualitative and involve first identifying relevant or similar paragraphs or sections to compare, before calculating a similarity score. Since the structures of the documents differ significantly, identifying comparable paragraphs would have introduced additional complexity and potential room for error, particularly if unsuitable sections were selected. Instead, we adopted an automatic approach by calculating a similarity score for each document using sentence-level similarity: Each document was segmented into sentences using spaCy, and individual sentences were stripped using .strip() (which removes spaces, tabs, newline characters like “\n”). Once stripped, embeddings were generated for every sentence in both documents. Semantic similarity was then calculated for each sentence, and the highest similarity score (i.e. the closest semantic match) was recorded. The final score was obtained by averaging these maximum sentence similarity values. For this analysis, we used documents that have been preprocessed with spaCy (Section 3.2.2), and the sentence-transformers 4.1.0 Python library using the “all-MiniLM-L6-v2” model (Aarsen, 2024). This is an efficient 384-dimensional model, but it is limited to 256word pieces per sentence. As a result, although it could have been interesting for the comparison, we did not compute entire documents as single embeddings. • Semantic similarity results are available in Section 4.3. 3.3.4 Topic Modelling Objective: To identify, group and compare thematic structures within the documents Topic modelling typically identifies clusters of words that frequently co-occur, helping to reveal distinct underlying topics that are usually imperceptible to the human eye. As mentioned in Section 2.5, since the corpus is not large enough for classic topic modelling, the Python library BERTopic (Grootendorst, 2022) will be the principal tool for this analysis. BERTopic can use embeddings and clustering techniques to extract thematic structures or “topics”. While BERTopic should produce better results than LDA on this corpus due to its reliance on sentence embeddings (which tend to perform better on smaller corpora) it does not necessarily guarantee that our results are of exploitable value given the very limited size of our corpus. For topic modelling, we used lowercased, lemmatized text with punctuation and stop words removed, as pre-processed with spaCy in Section 3.2.2. Lemmatization seemed necessary in the case of this corpus, even though it further reduces an already limited vocabulary. Without lemmatizing the text, the semantic impact of words would have been diminished, as topics would have been influenced by word forms. For example, BERTopic would have treated “emission” and “emissions” as distinct, rather than recognising them as part of the same concept. The “all-MiniLM-L6-v2” sentence-transformer model (Aarsen, 2024) was used to generate embeddings for BERTopic, which has all other parameters set to default (UMAP for dimensionality reduction, HDBSCAN for clustering) 14 . While we could have manually selected the number of topics, BERTopic can determine this automatically, so we decided to leave it 14 BERTopic documentation, which gives some details about these methods, is accessible on maartengr.github.io/BERTopic/ Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 41 4.1.1 Initial TXM Exploration The first significant aspect to observe is the size of the texts, text size in number of words is shown in Figure 5: We observed that while SPMs show more gradual changes in terms of document sizes, with a decline after AR1 and increase after AR3, Wikipedia CCM articles displayed a more abrupt upward trajectory, possibly indicating a growing interest about CC over time. But this is hardly the only possible explanation, as it could also be due to the growing interest in Wikipedia, or the increased accessibility of the Internet, which is far more common now than it was in 2005-2008. As such, we do not consider document size a reliable factor in this corpus, and its impact is mitigated through normalisation of any measures or analysis. The most compelling aspect of TXM is how easily it facilitates the exploration of a corpus using features such as the lexicon, concordance, and co-occurrences. However, we were unable to identify any meaningful differences between the texts, apart from the observation that the Wikipedia texts seemed to use simpler vocabulary. To pursue the analysis further in TXM, unprocessed raw text was no longer sufficient. To obtain more insightful results with TXM, we would have needed to import different versions of the texts: one with lowercased text to standardise the lexicon, and one with stop words removed. Although preparing multiple processed versions of the data for use in TXM was possible, it would have added an additional layer of preparation that was not necessary for the specific quantitative comparisons addressed in this study. Since our main goal was to systematically compute and compare predefined metrics mentioned at the beginning of this section, a straightforward approach using Python offered a more efficient and targeted alternative. Figure 5: Text size by Number of Words as Computed with TXM’s “Dimensions” Tool Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 42 4.1.2 Word Count Word count per document obtained with spaCy is shown in Table 8. Document Total Tokens Excluding Punct/stop-words AR1_WG3_SPM 16983 9043 AR2_WG3_SPM 11224 5767 AR3_WG3_SPM 9739 5261 AR4_WG3_SPM 14496 8086 AR5_WG3_SPM 17667 9399 AR6_WG3_SPM 26100 14328 Wiki_CCM_2005-06-26 1459 769 Wiki_CCM_2008-05-07 8156 4466 Wiki_CCM_2014-09-02 14888 7997 Wiki_CCM_2022-06-13 20477 10835 Table 8: Token Count per Document Obtained with spaCy While we previously observed an unprocessed word count using TXM, we are now able to exclude specific categories of words thanks to spaCy’s preprocessing. We excluded punctuation and stop-words in a second column. While not useful on its own, this metric is a foundation for other metrics such as average sentence length and POS tag frequencies, that we will observe later. 4.1.3 Lexical Diversity Table 9 presents the TTR, Herdan’s C and Guiraud’s R which were defined in Section 2.3.6 above. Recall from 3.3.1 that we computed the values based on lowercased lemmatized tokens. Values were rounded to 3 decimal places to facilitate reading. In this table, TTR should not be considered a reliable basis for comparison between the texts, as it does not account for text length. Document TTR Herdan’s C Guiraud’s R AR1_WG3_SPM 0.220 0.834 20.895 AR2_WG3_SPM 0.225 0.828 17.119 AR3_WG3_SPM 0.248 0.837 18.019 AR4_WG3_SPM 0.222 0.833 19.939 AR5_WG3_SPM 0.196 0.822 19.041 AR6_WG3_SPM 0.147 0.799 17.561 Wiki_CCM_2005-06-26 0.563 0.914 15.614 Wiki_CCM_2008-05-07 0.364 0.880 24.346 Wiki_CCM_2014-09-02 0.295 0.864 26.368 Wiki_CCM_2022-06-13 0.254 0.853 26.486 Table 9: TTR, Herdan’s C, Guiraud’s R (rounded values, 3 d.p.) By contrast, Herdan’s C and Guiraud’s R appeared to be more robust, except for the 2005 Wikipedia article for Guiraud’s R, which produced an unexpected result. Nevertheless, focusing on Herdan’s C revealed a slight decline in lexical diversity over time in both Wikipedia and IPCC documents, although Wikipedia consistently exhibits greater lexical diversity overall. Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 43 Lower lexical diversity in the IPCC reports may reflect repetitive or technical phrasing, which is typical of scientific writing, whereas the slightly broader vocabulary observed in Wikipedia articles could be attributed to the varied authorship inherent to the platform. While Figure 6 shows a slight trend of declining lexical diversity over time in the WG3 SPMs (in blue), a larger similar trend can be observed in Wikipedia (purple), which could be linked to improvements in article quality and fidelity to the IPCC over time. 4.1.4 Lexical Density Lexical Density results are available in Table 10. Figure 7 provides a visual representation of the results. Recall that lexical density (2.3.6) refers to the proportion of content words over total words, and that the status of each word was based on spaCy POS tags (see 3.3.1 above), considering numbers as content words given their informative role when discussing scientific issues. Document Lexical Density AR1_WG3_SPM 0.641 AR2_WG3_SPM 0.621 AR3_WG3_SPM 0.658 AR4_WG3_SPM 0.697 AR5_WG3_SPM 0.678 AR6_WG3_SPM 0.683 Wiki_CCM_2005-06-26 0.642 Wiki_CCM_2008-05-07 0.664 Wiki_CCM_2014-09-02 0.667 Wiki_CCM_2022-06-13 0.648 Table 10: Ure's Lexical Density per Document (rounded, 3 d.p.) Figure 6: Evolution of Herdan's C Over Time Source: Figures are generated using the Python script available in Appendix O Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 44 The closer lexical density is to 1, the denser (i.e. information heavy) are the documents. According to Figure 7, the IPCC maintains a consistently higher lexical density than Wikipedia across all time points, except for AR1 and AR2, when there was no equivalent. This is likely due to the IPCC’s more technical language. Although Wikipedia’s articles have increased slightly in lexical density over the years, the 2022 version remains below that of the IPCC AR6, indicating that Wikipedia has not yet matched the IPCC’s level of technical language. This suggests that Wikipedia still maintains a less technical and more accessible tone compared to the IPCC. 4.1.5 Relative Frequency of Function Words Document Rel. Freq. Function Words (%) AR1_WG3_SPM 35.55 AR2_WG3_SPM 37.67 AR3_WG3_SPM 33.78 AR4_WG3_SPM 29.23 AR5_WG3_SPM 32.00 AR6_WG3_SPM 31.18 Wiki_CCM_2005-06-26 35.69 Wiki_CCM_2008-05-07 33.47 Wiki_CCM_2014-09-02 33.12 Wiki_CCM_2022-06-13 35.01 Table 11: Relative Frequency of Function Words per Document (rounded, 2 d.p.) As mentioned in Section 3.3.1, RFFW can be seen as a reversed lexical density. As such, we used the RFFW values from Table 11 to produce Figure 8, but we reversed the Y axis which represents the values, and observed if Figure 8 and Figure 7 are visually similar. Figure 7: Evolution of Ure's Lexical Density Over Time Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 45 This hypothesis was confirmed, as Figure 8 appeared to be an exact replica of Figure 7, thereby supporting the results on lexical density just presented (4.1.4). 4.1.6 Sentence Count, Length, and Word Length Sentence count based on spaCy’s segmentation, average length (in words), and average word length (in characters) are available in Table 12. Document Sentence count Avg. Sentence Length (words) Avg. Word Length (chars) AR1_WG3_SPM 555 26.9 5.7 AR2_WG3_SPM 420 23.9 5.7 AR3_WG3_SPM 343 24.6 5.6 AR4_WG3_SPM 558 21.7 5.4 AR5_WG3_SPM 580 25.2 5.5 AR6_WG3_SPM 896 24.2 5.6 Wiki_CCM_2005-06-26 48 26.8 5.2 Wiki_CCM_2008-05-07 263 27.3 5.4 Wiki_CCM_2014-09-02 583 21.8 5.4 Wiki_CCM_2022-06-13 786 22.8 5.3 Table 12: Sentence Count, Length (in words) and Word Length (in characters) per Document (rounded, 1 d.p.) Unfortunately, these metrics alone were not sufficient to provide insight into a text’s readability. That is why, in the next section, we applied two different methods to assess readability based on sentence and word length. Figure 8: Relative Frequency of Function Words Over time (reversed Y Axis) Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 46 4.1.7 Readability As mentioned in 3.3.1, readability levels were calculated using textstat. Results are available in Table 13. The measures were defined in Section 2.3.6. Document FRE FKGL AR1_WG3_SPM 22.34 16 AR2_WG3_SPM 13.68 17.2 AR3_WG3_SPM 22.45 15.9 AR4_WG3_SPM 25.49 14.7 AR5_WG3_SPM 24.68 15.1 AR6_WG3_SPM 25.49 14.7 Wiki_CCM_2005-06-26 27.76 15.9 Wiki_CCM_2008-05-07 29.79 15.2 Wiki_CCM_2014-09-02 25.8 14.6 Wiki_CCM_2022-06-13 33.34 13.8 Table 13: Readability Scores Based on the Flesch Reading Ease (FRE, 1948) and its Later Development, the Flesch-Kincaid Grade Level (FKGL, 1975) FRE produces a score out of 100. The higher the score, the easier the text is to read. A score between 0 and 30 is generally considered to indicate college-level difficulty. FKGL is expressed as a U.S. school grade. In the United States, grade 13 corresponds to the first year of university, while grade 17 refers to the first year of a postgraduate programme (Master’s or PhD). Figure 9: Flesch Reading Ease Over Time Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 47 According to textstat results in Table 13, the SPMs and Wikipedia’s CCM are rated as college-level texts, which seems appropriate. As indicated by Figure 9 and Figure 10, readability has improved over time. The FRE score increased for both Wikipedia and the IPCC; however, the IPCC's score has remained stable since AR4. A similar trend is observed for the FKGL, although it is inversely related, as the lower the score, the easier the text is to read. Finally, Wikipedia has shown a clear trend of increasing readability over time. This may be intentional, as Wikipedia makes a conscious effort to enhance readability by offering editorial advice to its contributors. (Wikipedia, 2024c) 4.1.8 POS Tags Distribution We did not cover all the possible tags applied by spaCy to analyse the distribution of POS tags, as we believe some are irrelevant or occur with very low frequency. We highlighted the most important ones for our analysis in Table 14. Document NOUN VERB AUX ADJ ADV PRON ADP CCONJ SCONJ AR1_WG3_SPM 32.0 10.3 4.4 11.3 2.8 2.2 13.3 5.4 1.3 AR2_WG3_SPM 32.5 10.4 5.2 12.7 3.2 2.1 13.8 4.8 1.4 AR3_WG3_SPM 32.7 9.0 3.9 11.4 2.5 1.8 13.5 5.3 1.1 AR4_WG3_SPM 32.2 8.4 3.8 10.9 2.4 1.6 12.0 4.5 0.9 AR5_WG3_SPM 31.7 8.3 3.7 11.7 2.7 1.4 14.0 4.4 0.9 AR6_WG3_SPM 32.7 10.6 3.3 12.6 2.1 1.5 13.6 5.9 0.7 Wiki_CCM_2005-06-26 26.4 11.0 3.7 7.2 2.3 2.8 12.7 2.6 1.6 Wiki_CCM_2008-05-07 27.5 10.3 4.3 8.9 2.9 2.2 12.6 3.3 1.2 Wiki_CCM_2014-09-02 26.5 9.9 4.5 9.1 2.8 1.9 12.8 3.2 1.4 Wiki_CCM_2022-06-13 30.5 10.9 5.1 10.0 3.1 2.0 13.3 3.8 1.4 Table 14: Distribution of Relevant POS Tags (percentages) Figure 10: Flesch-Kincaid Grade Level Over Time Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 48 Nouns (NOUN): A higher proportion of nouns is found in SPMs. This is consistent with the general understanding that a higher noun count suggests a text is more informational or descriptive, aligning with the SPMs’ primary purpose. Wikipedia’s 2022 article showed a similar trend, confirming earlier observations that Wikipedia is increasingly aligning with the IPCC style. Verbs (VERB): Wikipedia tends to use more verbs overall. This suggests a greater emphasis on actions and events within Wikipedia articles, aligning with Korte et al.'s (2023) observation that Wikipedia frames CC primarily through events. Auxiliary Verbs (AUX): No significant difference or variation in the use of auxiliary verbs was noticeable between the two sources. While a high proportion of auxiliary verbs can indicate the use of complex verb phrases, the presence of questions, or the use of non-present tenses, these features did not emerge as distinguishing factors in this analysis. Adjectives (ADJ): The SPMs clearly contains a higher proportion of adjectives. This finding supports earlier observations that they aim for precision, as adjectives are characteristic of descriptive writing, providing additional detail about attributes. Interestingly, Wikipedia has gradually increased its use of adjectives over the years, indicating a move towards a similar style found in technical documents aiming for precision. Adverbs (ADV): The variation in adverb usage is less than 1% for both SPMs and Wikipedia, making any meaningful interpretation difficult. While adverbs can contribute to precision by modifying verbs and adjectives to provide more detail, as a higher frequency might be expected in precise texts, they were not considered a strong distinguishing feature in this comparison due to the minimal variation. Pronouns (PRON): Wikipedia consistently uses pronouns at a rate of 2%. In contrast, the IPCC appears to have actively reduced pronoun usage in its SPMs over time, nearly halving it from their first SPM in 1990 to the AR6 SPM in 2022. This reduction aligns with the expectation for technical texts to use fewer pronouns to ensure precision and maintain a more formal style, as a higher use of pronouns can lead to ambiguity. Adpositions (ADP): No significant difference or variation in the use of adpositions (which include prepositions and postpositions) was noticeable. While a high frequency of adpositions is often associated with complex noun phrases and may indicate syntactic complexity, this was not a distinguishing factor between the analysed texts. Coordinating Conjunctions (CCONJ): It might seem a reasonable hypothesis that a higher proportion of CCONJ in a document reflects a higher proportion of longer sentences in that document. However, in our corpus, an increase of decrease in CCONJ from one document to the other does not always match an increase or decrease in sentence length (compared to results in Section 4.1.6). Accordingly, the results suggest a more nuanced relationship between CCONJ and sentence length in these documents. Subordinating Conjunctions (SCONJ): The IPCC appears to have deliberately reduced its use of SCONJ over time in its SPMs, halving their frequency compared to its initial AR. Since SCONJ indicate the presence of subordinate clauses and can be used to assess syntactic complexity, this reduction seems deliberate and suggests a move towards less syntactically complex sentence structures in later IPCC SPMs. On the other hand, Wikipedia showed no particular trend and remained at a level similar to that of the first AR. 4.1.9 TF-IDF scores As mentioned in Section 3.3.1, for TF-IDF, documents were lemmatized and the vocabulary was limited to 2,000 terms, while the TF-IDF model was fit on the entire corpus. We discuss Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 49 here the results based on the 10 words with the highest TF-IDF per document; see Appendix G for the complete results. While many terms shared between documents appeared in the top 10 TF-IDF results, we focused on the unique terms to gain the most insight into each document. In AR1_WG3_SPM, we noticed the words “greenhouse”, “gas” and “emission”, which refer to greenhouse gas emissions, an important factor in climate change. There are also the terms “develop”, “country”, “energy” and “resource”. This could be a reference to countries using their resources to produce energy, which in turn generates greenhouse gases. It seems the first WG3 focuses on identifying the issues and their link to energy. In AR2_WG3_SPM, the words “cost”, “economic”, “policy”, “damage” and “estimate” indicate a shift towards a more economic and social perspective on climate change. Rather than identifying the issues, it seems that AR2’s SPM focuses on quantifying the damage. In AR3_WG3_SPM, we noticed the appearance of the term “mitigation”. This suggests that the focus is now on direct action, with terms such as “scenario”, “carbon”, “cost” and “reduction”. AR4_WG3_SPM appeared to continue AR3 SPM’s focus on mitigation. Carbon is now referred to as “CO₂”, and “greenhouse gas” is abbreviated as “GHG”. The term “eq” refers to equivalent, which is often paired with “CO₂” as “CO₂-eq”, 20 which is a standard unit for measuring carbon footprint. In AR5_WG3_SPM, the terms “evidence”, “medium” and “high” emerged. This reflects a focus on the certainty of findings and may suggest an attempt to address previous criticism regarding uncertainty about climate change or climate change denial. The words “confidence” and “high” are particularly prominent in AR6_WG3_SPM. It follows AR5’s approach, which emphasises certainty, while also introducing “pathways” as mitigation strategies. While the earliest version of Wikipedia's CCM article from 2005 uses terminology similar to that of the earlier ARs, such as “global”, “warming”, “carbon” and “emission”, it clearly places emphasis on events, with terms like “Kyoto”, “Protocol” and “2005”. Wikipedia’s 2008 CCM article has a shift towards “energy” and “power” sources and the need to “reduce” them. Wikipedia’s 2014 CCM article continued to focus on “energy”, but the emergence of the term “nuclear” is notable, as this technology was likely considered an emerging mitigation strategy at the time. While “2011” could refer to various events, it is difficult not to associate it with the Fukushima disaster of that year, especially when “nuclear” is also a key term. The article likely discusses the dangers of nuclear technology, a common topic in public debate. Finally, Wikipedia’s 2022 CCM article features a classic list of words that reflects contemporary discourse on CCM. The appearance of the word “mitigation” in the list indicates a convergence with the language of the IPCC, suggesting that Wikipedia does, at least in part, reflect the scientific discourse. According to these observations, while the IPCC adjusts its focus over the years, likely in response to evolving expectations, it consistently maintains a highly technical tone, using precise metrics and terminology. In contrast, Wikipedia, although it incorporates key IPCC terms, also reflects public discourse and contemporary events. However, the most recent 20 In the corpus, these terms were preprocessed as 'co2', 'eq', and sometimes 'co2eq'. The additional formatting in this section is intended solely to support understanding of the terms. Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 50 iteration of the CCM article from 2022 aligns more closely than ever with the IPCC's language and framing, which is what we sometimes also observed in previous sections. 4.2 Modality Analysis 4.2.1 Verb and Adverbs To reveal the trends in our corpus, we calculated the normalized frequency, calculated as the number of occurrences of modal verbs and adverbs per 1,000 words, excluding punctuation and stop words. Frequencies of modal verbs and adverbs are available in Table 15. They were processed using Python code available in Appendix H. Document Total Modal Verb Freq. Total Modal Adverb Freq. AR1_WG3_SPM 27.093 0.774 AR2_WG3_SPM 31.153 1.038 AR3_WG3_SPM 24.520 0.570 AR4_WG3_SPM 16.819 0.247 AR5_WG3_SPM 12.132 1.415 AR6_WG3_SPM 15.983 0.349 Wiki_CCM_2005-06-26 11.704 None Wiki_CCM_2008-05-07 21.900 0.448 Wiki_CCM_2014-09-02 18.519 0.497 Wiki_CCM_2022-06-13 25.544 0.643 Table 15: Total Modal Verb/Adverb Freq. per Text (normalized per 1000, rounded, 3 d.p.) In Table 15, we observed that modal verbs are much more frequent than modal adverbs, but this is an expected result, as modal verbs are generally used more often than adverbs. Figure 11: Total Modal Verb Frequency (per 1000) Over Time Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 57 4.6 Named Entity Recognition Analysis As described in 3.3.6, both spaCy NER and a CC specific NER were used. A single Python script was run to count spaCy entities, which were already processed and stored in spaCy Doc objects (see 3.2.2), and to execute the transformer-based CC NER. The script is available in Appendix L. In this section, we evaluate NER results in the context of climate change mitigation, while evaluating Korte et al.'s (2023) hypothesis that Wikipedia frames climate change more in terms of events and personalities than the IPCC does. Accordingly, some categories deemed irrelevant or supported by limited data have been excluded from Table 23 and Table 24. However, complete tables containing all the results are available in Appendix M for spaCy’s NER and Appendix N for CC NER. This section is divided into three subsections. The first subsection (4.6.1) is an assessment of spaCy’s NER results. The second subsection (4.6.2) assess results from a dedicated CC NER transformer-based model, and the third subsection (4.6.3) synthesises the results. 4.6.1 SpaCy’s NER Results For spaCy, we retrieved dates or periods (DATE), named events (EVENT), geopolitical entities (GPE), nationalities and religious/political groups (NORP), organizations (ORG) and persons or fictional characters (PERSON), as shown in Table 23. SpaCy NER categories and definitions are available in Section 2.6.3 (Table 1, p.22). Document DATE EVENT GPE NORP ORG PERSON AR1_WG3_SPM 7.409 0.442 1.106 0.221 15.371 0.663 AR2_WG3_SPM 4.500 0.173 0.173 0.346 5.365 0.000 AR3_WG3_SPM 11.785 0.190 1.901 0.950 17.107 1.711 AR4_WG3_SPM 14.222 0.247 1.113 0.495 22.755 2.473 AR5_WG3_SPM 24.669 0.404 1.921 0.607 19.412 2.831 AR6_WG3_SPM 17.797 0.140 0.628 1.047 22.334 0.768 Wiki_CCM_2005-06-26 33.810 2.601 42.913 2.601 45.514 5.202 Wiki_CCM_2008-05-07 20.376 1.120 16.346 1.567 31.572 11.196 Wiki_CCM_2014-09-02 30.698 0.746 13.547 2.237 41.511 15.287 Wiki_CCM_2022-06-13 20.583 0.643 11.578 3.124 22.972 5.881 Table 23: spaCy Relevant NER Categories (frequencies normalized per 1000, rounded, 3 d.p.) Results in Table 23 indicated that dates occur more frequently in Wikipedia articles, although the IPCC also relies on them considerably. This difference, however, is not significant enough to draw firm conclusions, even if a higher frequency of dates combined with other categories such as events could indicate a tendency to frame content around specific moments in time. Regarding events, there is also a higher count in the earlier Wikipedia articles, but the 2014 and 2022 articles reduced this by half, bringing them closer to the IPCC’s usage, though still notably different. Both statistics support Korte et al.'s (2023) claim. In Wikipedia, the results showed a very high count of geopolitical entities, up to 42 times higher for the 2005 article compared to the first 1990 SPM. However, that article is very short and should not be heavily relied upon. The difference remains substantial in other cases, Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 58 such as the 2022 Wikipedia article which contains more than 18 times the number of geopolitical entity occurrences compared to its SPM counterpart. The remaining categories, which include nationalities or religious and political groups, organisations, and persons, also appeared with much higher frequency in Wikipedia. This provided strong evidence supporting Korte et al.'s (2023) claims, confirming that Wikipedia does indeed frame climate change differently from the IPCC in the documents analysed. That said, we observed a downward trend over time, with the most recent Wikipedia article showing frequencies closer to its latest SPM counterpart, although significant differences remain in some categories. However, the frequency of organisations is nearly equal in the 2022 article and its SPM counterpart 4.6.2 Climate Change NER Results While comparing named entities across the different documents was informative, examining them through the lens of climate change proved to be more insightful. CC NER categories and definitions are available in Section 3.3.6 (Table 7, p.38-39), while complete CC NER results are available in Appendix N. Note: Original class names were prefixed with 'climate-'; this prefix has been omitted in the tables for conciseness. Document Impacts Mitigations Greenhouse Gases Organizations Problem Origins Hazards AR1_WG3_SPM 1.106 40.142 12.275 19.020 42.243 10.727 AR2_WG3_SPM 10.211 31.499 3.461 7.442 16.961 8.827 AR3_WG3_SPM 1.331 50.561 13.496 11.595 37.635 4.372 AR4_WG3_SPM 0.989 57.878 22.384 10.017 49.963 4.081 AR5_WG3_SPM 0.809 46.608 20.928 9.099 33.970 3.235 AR6_WG3_SPM 0.907 57.300 23.032 6.281 38.945 2.931 Wiki_CCM_2005-06-26 2.601 71.521 6.502 55.917 9.103 29.909 Wiki_CCM_2008-05-07 2.463 74.116 20.376 55.083 38.065 11.420 Wiki_CCM_2014-09-02 1.740 71.837 21.999 47.974 39.150 6.587 Wiki_CCM_2022-06-13 1.838 77.001 7.535 20.674 43.646 11.853 Table 24: Relevant CC NER Categories (per 1000, rounded, 3 d.p.) The results in Table 24 show that, for climate-impacts (effects of hazards to humans), AR2 is a notable outlier: It contains up to ten times more climate-impacts entities than the other reports. Consequently, AR2 is much lower on other categories than other ARs. This aligns with the theme of its own report, which focuses on exploring how climate change could affect society. Wikipedia tends to mention these impacts slightly more often than all AR but AR2, which suggests that it places greater emphasis on the consequences of inaction and the urgency of action, thereby framing climate change as a crisis. For climate-mitigations (activities to reduce the impact of CC), which is the focus of our corpus, both sources dedicate a portion of their content to the topic. However, Wikipedia showed a higher frequency. This may be because Wikipedia presents information in a more direct manner, whereas the IPCC includes studies and focuses on feasibility. Except for AR2, climate-greenhouse-gases (mention of gases that impacts the atmosphere) are consistently mentioned in the SPMs, but their presence is more variable on Wikipedia. Results and Discussion Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 59 This was difficult to assess, but it is possible that the most recent Wikipedia article on climate change mitigation has become less technical and no longer refers explicitly to the names of specific gases, to attract a more general audience. Climate-organization (organizations with CC interests) are mentioned far more frequently in Wikipedia articles than in the SPMs. The SPMs are more moderate and show a decreasing trend in this category over time, a pattern that Wikipedia also follows while still significantly higher than its counterpart, the 2022 Wikipedia article shows a significant reduction in frequency, indicating a closer alignment with the original IPCC material. Both the IPCC and Wikipedia show strong interest in the climate-problem-origins category (Issues that results in CC), only the 2005 document shows a significantly lower frequency, which may be due to the short length of the article. In this aspect, both sources are equal. Climate-hazards are a major focus of the 2005 article, indicating an emphasis on the dangers themselves rather than on their underlying causes, as discussed previously. Wikipedia shows a greater tendency to frame climate change through the lens of hazards, which once again aligns with the “crisis” narrative often found in public media, as noted by Korte et al. (2023). In contrast, the SPMs have adopted a more measured tone over time, particularly in the most recent report. Overall, the climate change NER analysis shows that Wikipedia CCM articles successfully cover the key thematic areas of climate change mitigation, although the emphasis, level of technical detail, and framing can differ. In contrast, the IPCC Working Group III SPM maintains a scientifically assessed presentation, as is expected of a Summary for Policymakers. 4.6.3 Results Synthesis Combined NER analyses revealed distinct styles between the sources. Wikipedia’s CCM articles showed a high frequency of PERSON, EVENT, GPE, and NORP entities, supporting Korte et al.'s (2023) theory of an event-driven framing. This tendency is also reflected in the emphasis on climate hazards identified by the CC NER model. In contrast, the IPCC WG3 SPMs maintain an institutional tone with a consistently high level of technical detail, particularly in reference to greenhouse gases. While the two sources differed in framing, both managed to address the core themes of CCM. Notably, Wikipedia has shown increasing alignment with the IPCC SPMs over time. This is consistent with our earlier analyses, which also indicated that the most recent Wikipedia article is becoming more closely aligned with the latest SPM. 4.7 Section Summary: Results and Discussion To summarize Section 4, the presented analyses revealed a nuanced relationship between the IPCC WG3 SPMs and Wikipedia’s CCM articles. While stylistic, lexicometric, and readability metrics initially highlighted distinct approaches tailored to different audiences, a clear trend of convergence emerged over time. Semantic similarity increased, and Wikipedia’s thematic content and framing, particularly in its most recent article revision, have become more closely aligned with those of the IPCC. Modality and NER analysis further highlighted contrasting communicative strategies: the IPCC tends to employ formal, codified language, while Wikipedia reflects a broader, more event-driven public discourse. However, these differences suggest variations in emphasis and simplification for a general audience, rather than a bias or distortion of the core scientific message presented by the IPCC. Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 60 Conclusion In this study, we conducted a comparative analysis of the Intergovernmental Panel on Climate Change (IPCC) Working Group III (WG3) Summaries for Policymakers (SPM), which focus on climate change mitigation, and thematically and temporally corresponding versions of Wikipedia’s “Climate Change Mitigation” (CCM) article. The primary objective was to employ Natural Language Processing (NLP) techniques and tools to identify potential differences or biases in how Wikipedia presents CCM compared to the IPCC, as this is a key scientific topic often targeted by disinformation. Using an array of NLP techniques including lexicometry, stylistic analysis, readability assessments, modality analysis, semantic similarity, topic modelling, sentiment and emotion detection, and both general and climate-specific Named Entity Recognition (NER), the study conducted a detailed comparison between six IPCC WG3 SPM (from AR1 to AR6) and four temporally aligned Wikipedia CCM articles from 2005, 2008, 2014, and 2022. This revealed a complex and evolving relationship between the two sources. The study began with lexicometric, stylistic, and readability analyses. These techniques highlighted fundamental differences, likely influenced by the intended audiences of each source. The IPCC SPMs consistently demonstrated a higher level of technicality, greater lexical density, and college-level readability, all of which are characteristic of scientific reports aimed at policymakers. In contrast, Wikipedia was initially simpler but over time exhibited a clear progression towards greater technical complexity, increased lexical richness, and improved readability. These developments suggest a gradual stylistic convergence with the IPCC. A TF-IDF analysis illustrated the IPCC’s shifting focus from problem identification to economic impacts, mitigation strategies, and controlled expressions of certainty. Wikipedia, while adopting much of the IPCC’s terminology, also reflected elements of public discourse and references to contemporary events. Modality analysis revealed a clear shift in the IPCC’s language, from the use of general modal verbs in earlier reports to the adoption of highly standardised expressions of likelihood and confidence in later assessments, particularly in AR5 and AR6. This transition reflects a deliberate effort to communicate uncertainty with greater precision. Wikipedia did not adopt these codified forms and instead showed an increase in the use of modal verbs and adverbs over time, consistent with its aim to inform a more general audience. Semantic similarity scores, calculated using Sentence-BERT and cosine similarity, revealed a slow yet gradual and consistent convergence between the IPCC SPMs and their corresponding Wikipedia articles. The average similarity score rose from 0.424 for the AR3 and 2005 pairing to 0.553 for the AR6 and 2022 pairing. This suggests that Wikipedia is increasingly reflecting the content of the IPCC’s findings on CCM, although notable differences remain due to the distinct purposes of each source. Although the corpus was limited in size, topic modelling successfully grouped some of the IPCC SPMs and Wikipedia articles together based on shared mitigation themes. In contrast, the later IPCC reports formed distinct clusters, characterised by their modal expressions of likelihood and confidence. The first Working Group III SPM was identified as an outlier, due to its distinct focus on identifying climate change (CC) issues rather than presenting mitigation strategies. Sentiment and emotion analyses indicated that both sources generally maintained a predominantly neutral tone, as expected in scientific writing. While VADER initially suggested a positive sentiment, transformer-based models offered a more nuanced interpretation, Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 61 revealing overall neutrality with an upward trend in positivity within the IPCC SPMs. Wikipedia articles showed a slightly higher occurrence of “fear”, which aligns with Korte et al. (2023) findings on media framing CC as a “crisis”. Named Entity Recognition (NER) provided evidence of contrasting framing strategies between the two sources. A first NER analysis using spaCy showed that Wikipedia’s articles more frequently included entities related to events, persons, and geopolitical locations. This supports Korte et al.'s (2023) hypothesis that Wikipedia adopts a more event and politically driven narrative style. A second NER analysis was then conducted using a specialised CC NER model. This analysis revealed that, while both sources addressed core CCM themes, Wikipedia tended to place greater emphasis on hazards, impacts, and organisations. The IPCC maintained a more consistent and technical focus, particularly regarding greenhouse gases. Finally, both NER analyses indicated a clear trend towards convergence, with the most recent Wikipedia CCM article from 2022 showing entity frequencies in several categories that are increasingly aligned with those in the IPCC 2022 SPM. To answer the research question posed in the introduction, specifically whether there are differences or biases in Wikipedia’s portrayal of the IPCC reports, the findings from the analysis of Wikipedia’s CCM articles and the WG3 SPMs indicate that Wikipedia does not exhibit bias in the form of deliberate distortion or omission. Instead, it shows differences in framing, emphasis, and the simplification of technical content, which are consistent with its function as a publicly accessible resource intended for a general audience. While Wikipedia is influenced by public discourse, events, and an inclination towards a “crisis” narrative, it demonstrates a clear effort to align with and accurately convey the IPCC's core scientific findings on CCM. While it aimed to be comprehensive, this study has some significant limitations. The size of the corpus restricted the extent of some Natural Language Processing analyses, particularly topic modelling. The focus on WG3 SPMs and multiple revisions of a single Wikipedia article means that the findings may not be generalisable to all IPCC content or to the broader scope of Wikipedia’s climate change portal. Finally, the dynamic nature of Wikipedia means that the 2022 snapshot analysed in this study is already outdated. Considering these limitations, future research could expand the corpus to include all IPCC working groups as well as a broader selection of related Wikipedia articles. While this study focused on English due to its prominence on Wikipedia, a cross-linguistic analysis would offer valuable insights into how CCM is represented across different languages. Although this research examined changes over time, a more extensive longitudinal study capturing all article revisions would provide a deeper understanding of content evolution. Additionally, the application of other NLP methods, such as Aspect-Based Sentiment Analysis or the use of cross-encoders for semantic similarity, could enrich the findings. A greater emphasis on qualitative approaches could also complement the quantitative analyses, offering further insight. In conclusion, this study demonstrates that although the IPCC and Wikipedia serve different audiences, Wikipedia’s coverage of climate change mitigation is not marked by significant bias against IPCC findings. It rather reflects a sustained effort to make complex scientific information accessible to the public, with growing alignment to IPCC assessments and an increasingly important role in supporting the public understanding of one of the most critical challenges for humankind. Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 62 References Aarsen, T. (2024, January 5). Sentence-transformers/all-MiniLM-L6-v2 · Hugging Face. https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 Adobe®. (2006). PDF Reference sixth edition. https://web.archive.org/web/20081001170454/https://www.adobe.com/devnet/ acrobat/pdfs/pdf_reference_1-7.pdf Anthony, L. (2005). AntConc: Design and development of a freeware corpus analysis toolkit for the technical writing classroom. IPCC 2005. Proceedings. International Professional Communication Conference, 2005., 729–737. https://doi.org/10.1109/IPCC.2005.1494244 Auer, C., Lysak, M., Nassar, A., Dolfi, M., Livathinos, N., Vagenas, P., Ramis, C. B., Omenetti, M., Lindlbauer, F., Dinkla, K., Mishra, L., Kim, Y., Gupta, S., Lima, R. T. de, Weber, V., Morin, L., Meijer, I., Kuropiatnyk, V., & Staar, P. W. J. (2024). Docling Technical Report (No. arXiv:2408.09869). arXiv. https://doi.org/10.48550/arXiv.2408.09869 Barbieri, F., Camacho-Collados, J., Espinosa, A., Luis, & Neves, L. (2024). Cardiffnlp/twitter-roberta-base-sentiment · Hugging Face. https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment Barkemeyer, R., Dessai, S., Monge-Sanz, B., Renzi, B. G., & Napolitano, G. (2016). Linguistic analysis of IPCC summaries for policymakers and associated coverage. Nature Climate Change, 6(3), 311–316. https://doi.org/10.1038/nclimate2824 Beutler, W. (2019). Paid With Interest: COI Editing and its Discontents. Wikipedia @ 20. https://wikipedia20.mitpress.mit.edu/pub/kmwtidhw/release/4 Bhattacharjee, B., Trivedi, A., Muraoka, M., Ramasubramanian, M., Udagawa, T., Gurung, I., Pantha, N., Zhang, R., Dandala, B., Ramachandran, R., Maskey, M., Bugbee, K., Little, M., Fancher, E., Gerasimov, I., Mehrabian, A., Sanders, L., Costes, S., Blanco-Cuaresma, S., … Lee, T. (2024). INDUS: Effective and Efficient Language Models for Scientific Applications (No. arXiv:2405.10725). arXiv. https://doi.org/10.48550/arXiv.2405.10725 Biros, C., & Peynaud, C. (2019). Disseminating climate change knowledge. Representation of the International Panel on Climate Change in three types of specialized discourse. https://doi.org/10.1285/I22390359V29P179 Biros, C., Rossi, C., & Talbot, A. (2021). Corpus GIEC [Corpus]. ORTOLANG. https://hdl.handle.net/11403/corpus-giec/v1 Bounegru, L., De Pryck, K., Venturini, T., & Mauri, M. (2020). “We only have 12 years”: YouTube and the IPCC report on global warming of 1.5oC. First Monday. https://doi.org/10.5210/fm.v25i2.10112 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 63 Canonical Ltd. (2024). Ubuntu – Package Search Results. https://packages.ubuntu.com/search?lang=en&suite=all&searchon=names&ke ywords=curl Ceylan, C. (2022). Application of Natural Language Processing to Unstructured Data: A Case Study of Climate Change [Thesis, Massachusetts Institute of Technology]. https://dspace.mit.edu/handle/1721.1/144647 Cohen, N. (2009, August 24). Wikipedia to Limit Changes to Articles on People. The New York Times. https://www.nytimes.com/2009/08/25/technology/internet/25wikipedia.html De Pryck, K., & Hulme, M. (Eds.). (2022). A Critical Assessment of the Intergovernmental Panel on Climate Change. Cambridge University Press. https://doi.org/10.1017/9781009082099 Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (No. arXiv:1810.04805). arXiv. https://doi.org/10.48550/arXiv.1810.04805 Duran, N. (2024, May 21). Nicolauduran45/specter-climate-change-NER · Hugging Face. https://huggingface.co/nicolauduran45/specter-climate-change-NER Explosion. (2025). spaCy Models Documentation. English. https://spacy.io/models/en Fenniak, M., & Martin, T. (2024). pypdf: A pure-python PDF library capable of splitting, merging, cropping, and transforming PDF files (Version 5.1.0) [Python; OS Independent]. Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221–233. https://doi.org/10.1037/h0057532 Fleury, S., & Zimina, M. (2014). Trameur: A Framework for Annotated Text Corpora Exploration. In L. Tounsi & R. Rak (Eds.), Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations (pp. 57–61). Dublin City University and Association for Computational Linguistics. https://aclanthology.org/C14-2013/ Gallo, I., Binaghi, E., Carullo, M., & Lamberti, N. (2008). Named Entity Recognition by Neural Sliding Window. 2008 The Eighth IAPR International Workshop on Document Analysis Systems, 567–573. https://doi.org/10.1109/DAS.2008.13 Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TFIDF procedure (No. arXiv:2203.05794). arXiv. https://doi.org/10.48550/arXiv.2203.05794 Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2013). The Rise and Decline of an Open Collaboration System: How Wikipedia’s Reaction to Popularity Is Causing Its Decline. American Behavioral Scientist, 57(5), 664–688. https://doi.org/10.1177/0002764212469365 Halliday, M. A. K. (1989). Spoken and written language. Oxford University Press. Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 64 Han, Y., Ceross, A., & Bergmann, J. H. M. (2024). The Use of Readability Metrics in Legal Text: A Systematic Literature Review (No. arXiv:2411.09497). arXiv. https://doi.org/10.48550/arXiv.2411.09497 Härdle, W., & Chen, C. (2016). Probabilistic Topic Models in Natural Language Processing. Hart, R. P. (1984). Systematic analysis of political discourse: The development of DICTION. Political Communication Yearbook, 1, 97–134. Hartmann, J. (2022). J-hartmann/emotion-english-distilroberta-base · Hugging Face. https://huggingface.co/j-hartmann/emotion-english-distilroberta-base Hearst, M. A. (1997). Text Tiling: Segmenting Text into Multi-paragraph Subtopic Passages. Computational Linguistics, 23(1), 33–64. Heiden, S., Magué, J.-P., & Pincemin, B. (2010). TXM: Une plateforme logicielle open-source pour la textométrie - conception et développement. 13. Herrando-Pérez, S., Bradshaw, C. J. A., Lewandowsky, S., & Vieites, D. R. (2019). Statistical Language Backs Conservatism in Climate-Change Assessments. BioScience, 69(3), 209–219. https://doi.org/10.1093/biosci/biz004 Hickson, I., & Hyatt, D. (2008). HTML 5. https://www.w3.org/TR/2008/WD-html520080122/#references Huddleston, R., & Pullum, G. K. (2002). The Cambridge Grammar of the English Language. Cambridge University Press. Hutto, C., & Gilbert, E. (2014). VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. Proceedings of the International AAAI Conference on Web and Social Media, 8(1), Article 1. https://doi.org/10.1609/icwsm.v8i1.14550 Hyperbase. (2024). Hyperbase.unice.fr. https://hyperbase.unice.fr/ Ibm-research/Climate-Change-NER · Datasets at Hugging Face. (2024, October 11). https://huggingface.co/datasets/ibm-research/Climate-Change-NER Iezzi, D. F., Mayaffre, D., & Misuraca, M. (Eds.). (2020). Text Analytics: Advances and Challenges. Springer International Publishing. https://doi.org/10.1007/9783-030-52680-1 IPCC. (2018). Special Report: Global Warming of 1.5 oC (SR15). https://www.ipcc.ch/sr15/ IPCC. (2024a). Procedures—IPCC. Ipcc.Ch. https://www.ipcc.ch/documentation/procedures/ IPCC. (2024b). TFI — IPCC. https://www.ipcc.ch/working-group/tfi/ Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 65 IPCC & WMO (Eds.). (1992). Climate change: The 1990 and 1992 IPCC assessments, IPCC first assessment report overview and policymaker summaries and 1992 IPPC supplement. IPCC. Keegan, B., Gergle, D., & Contractor, N. (2012). Staying in the loop: Structure and dynamics of Wikipedia’s breaking news collaborations. Proceedings of the Eighth Annual International Symposium on Wikis and Open Collaboration, 1– 10. https://doi.org/10.1145/2462932.2462934 Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., Rychlý, P., & Suchomel, V. (2014). The Sketch Engine: Ten years on. 1(1). https://www.muni.cz/en/research/publications/1193200 Kincaid, J. P., Fishburne Jr., R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel (Research Branch No. ADA006655; p. 51). NAVAL TECHNICAL TRAINING COMMAND MILLINGTON TN RESEARCH BRANCH. https://apps.dtic.mil/sti/citations/ADA006655 Kobaliani, L. (2023). Stanley Kubrick: The life and work of the great filmmaker [Online magazine]. Artdevivre.Com. https://artdevivre.com/articles/stanley-kubrick-thelife-and-work-of-the-great-filmmaker/ Korte, J. W., Bartsch, S., Beckmann, R., El Baff, R., Hamm, A., & Hecking, T. (2023). From causes to consequences, from chat to crisis. The different climate changes of science and Wikipedia. Environmental Science & Policy, 148, 103553. https://doi.org/10.1016/j.envsci.2023.103553 Le Treut, H., & Somerville, R. (2007). Box 1.1 Treatment of Uncertainties in the Working Group I Assessments—AR4 WGI Chapter 1. Archive.Ipcc.Ch. https://archive.ipcc.ch/publications_and_data/ar4/wg1/en/ch1s1-6.html Lebart, L., & Salem, A. (1994). Statistique textuelle. Lissón, P., & Ballier, N. (2018). Investigating Lexical Progression through Lexical Diversity Metrics in a Corpus of French L3. Discours. Revue de Linguistique, Psycholinguistique et Informatique. A Journal of Linguistics, Psycholinguistics and Computational Linguistics, 23, Article 23. https://doi.org/10.4000/discours.9950 Liu, B. (2012). Sentiment Analysis and Opinion Mining. Springer International Publishing. https://doi.org/10.1007/978-3-031-02145-9 Loria, S. (2024). Sloria/TextBlob [Python]. https://github.com/sloria/TextBlob (Original work published 2013) Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S. L., Péan, C., Berger, S., Caud, N., Chen, Y., Goldfarb, L., Gomis, M. I., Huang, M., Leitzell, K., Lonnoy, E., Matthews, J. B. R., Maycock, T. K., Waterfield, T., Yelekçi, Ö., Yu, R., & Zhou, B. (Eds.). (2021). Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 66 Intergovernmental Panel on Climate Change. Cambridge University Press. https://doi.org/10.1017/9781009157896 Merriam-Webster. (2024, November 6). Definition of METADATA. https://www.merriam-webster.com/dictionary/metadata Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013, January 16). Efficient Estimation of Word Representations in Vector Space. International Conference on Learning Representations. https://www.semanticscholar.org/paper/Efficient-Estimation-of-WordRepresentations-in-MikolovChen/f6b51c8753a871dc94ff32152c00c01e94f90f09 Nagpal, A., & Gabrani, G. (2019). Python for Data Analytics, Scientific and Technical Applications. 2019 Amity International Conference on Artificial Intelligence (AICAI), 140–145. https://doi.org/10.1109/AICAI.2019.8701341 Nandwani, P., & Verma, R. (2021). A review on sentiment analysis and emotion detection from text. Social Network Analysis and Mining, 11(1), 81. https://doi.org/10.1007/s13278-021-00776-6 Nugues, P. M. (2024). Python for Natural Language Processing: Programming with NumPy, scikit-learn, Keras, and PyTorch. Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-57549-5 Pennington, J., Socher, R., & Manning, C. (2014). GloVe: Global Vectors for Word Representation. In A. Moschitti, B. Pang, & W. Daelemans (Eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1532–1543). Association for Computational Linguistics. https://doi.org/10.3115/v1/D14-1162 Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep Contextualized Word Representations. In M. Walker, H. Ji, & A. Stent (Eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 2227–2237). Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1202 Peters, T. (2024). The Zen of Python. Python Enhancement Proposals (PEPs). https://peps.python.org/pep-0020/ Pincemin, B. (2018). Sept logiciels de textométrie. halshs-01843695. Pincemin, B., & Heiden, S. (2008). What is textometry? Introduction. Textometry project website. https://txm.gitpages.huma-num.fr/textometrie/en/Introduction/ Poortvliet, P. M., Niles, M. T., Veraart, J. A., Werners, S. E., Korporaal, F. C., & Mulder, B. C. (2020). Communicating Climate Change Risk: A Content Analysis of IPCC’s Summary for Policymakers. Sustainability, 12(12), Article 12. https://doi.org/10.3390/su12124861 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 73 • AR6 WG3 SPM https://www.ipcc.ch/report/ar6/wg3/downloads/report/IPCC_AR6_WGIII_SummaryForPolicymakers.pdf • Treatment of Uncertainties (Likelihood and Confidence Expressions) https://archive.ipcc.ch/publications_and_data/ar4/wg1/en/ch1s1-6.html HuggingFace Models: • all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 • distilroberta-finetuned-financial-news-sentiment-analysis https://huggingface.co/mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis • emotion-english-distilroberta-base https://huggingface.co/j-hartmann/emotion-english-distilroberta-base • specter-climate-change-NER https://huggingface.co/nicolauduran45/specter-climate-change-NER • specter2_base https://huggingface.co/allenai/specter2 Datasets: • Climate-Change-NER https://huggingface.co/datasets/ibm-research/Climate-Change-NER Software: • TXM 8.4 https://txm.gitpages.huma-num.fr/textometrie/index.html • Gemini 2.5 (Google AI Studio) https://aistudio.google.com/ Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 74 Appendix C: [Python code] Extracting Text from a PDF File Using pypdf from pypdf import PdfReader reader = PdfReader("IPCC_AR6_WGIII_SummaryForPolicymakers.pdf") number_of_pages = len(reader.pages) all_text = "" for page_num in range(number_of_pages): page = reader.pages[page_num] text = page.extract_text() if text: all_text += text + " " with open("AR6_WG3_SPM.txt", "w", encoding="utf-8") as file: file.write(all_text) Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 75 Appendix D: [Python code] Text Extraction from Wikipedia import requests import mwparserfromhell title = "Climate change mitigation" endpoint = "https://en.wikipedia.org/w/api.php" date_iso = "2014-09-02T07:44:59Z" params = { "action": "query", "format": "json", "prop": "revisions", "titles": title, "rvlimit": 1, "rvprop": "ids|timestamp|content", "rvdir": "older", "rvstart": date_iso, } data = requests.get(endpoint, params=params).json() page = next(iter(data["query"]["pages"].values())) # Manages old and new Wikipedia revision format (pre or post 2020) revision = page["revisions"][0] wiki_markup = revision.get("*") or revision.get("slots", {}).get("main", {}).get("*", "") # Convert wiki markup to plain text wikicode = mwparserfromhell.parse(wiki_markup) plain_text = wikicode.strip_code() with open("wikipedia_article_revision.txt", "w", encoding="utf-8") as f: f.write(plain_text) print(f"Downloaded and cleaned revision from {revision['timestamp']}") Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 76 Appendix E: [Python code] Preprocessing with spaCy import spacy from spacy.tokens import DocBin from pathlib import Path import os INPUT_DIR = Path("data") OUTPUT_DIR = Path("processed_docs") MODEL = "en_core_web_lg" nlp = spacy.load(MODEL) print(f"Processing .txt files from: {INPUT_DIR}") text_files = list(INPUT_DIR.glob("*.txt")) for text_file_path in sorted(text_files): print(f"Processing: {text_file_path.name}...") with open(text_file_path, "r", encoding="utf-8") as f: text_content = f.read() doc = nlp(text_content) doc_bin = DocBin(docs=[doc]) output_file_path = OUTPUT_DIR / (text_file_path.stem + ".spacy") doc_bin.to_disk(output_file_path) print(f"Successfully saved: {output_file_path}") print("Preprocessing complete.") Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 77 Appendix F: [Python code] Lexicometry, Stylistic and Readability Processing import spacy from spacy.tokens import DocBin from sklearn.feature_extraction.text import TfidfVectorizer import textstat from collections import Counter import math import os import pandas as pd # Configuration INPUT_DIR = "processed_docs/" MODEL = "en_core_web_lg" OUTPUT_CSV = "corpus_metrics.csv" # Loading spaCy to use en_core_web_lg's stop words list nlp = spacy.load(MODEL) STOP_WORDS = nlp.Defaults.stop_words print(f"Successfully loaded spaCy model and {len(STOP_WORDS)} stop words.") all_doc_metrics_data = [] corpus_for_tfidf = [] doc_names = [] doc_files = [f for f in os.listdir(INPUT_DIR) if f.endswith(".spacy")] for file_name in sorted(doc_files): # Sort for consistent order print(f"Now processing: {file_name} !") file_path = os.path.join(INPUT_DIR, file_name) doc_bin_loaded = DocBin().from_disk(file_path) loaded_docs_from_file = list(doc_bin_loaded.get_docs(nlp.vocab)) # While we could have saved multiple documents per binary file # We decided to save only 1 document per file to keep document names doc = loaded_docs_from_file[0] doc_names.append(file_name) current_doc_metrics = {"document_name": file_name} raw_text = doc.text all_tokens = [token for token in doc if not token.is_space] words_no_punct = [token for token in all_tokens if not token.is_punct] words_no_punct_no_stop = [token for token in words_no_punct if token.text.lower() not in STOP_WORDS] lemmas_no_punct_no_stop = [token.lemma_.lower() for token in doc if not token.is_punct and not token.is_space and not token.is_stop] # Word Counts current_doc_metrics["total_tokens_incl_punct"] = len(all_tokens) current_doc_metrics["total_words_excl_punct"] = len(words_no_punct) current_doc_metrics["total_words_excl_punct_stop"] = len(words_no_punct_no_stop) # Lexical Diversity total_lemmas_ld = len(lemmas_no_punct_no_stop) unique_lemmas_ld = len(set(lemmas_no_punct_no_stop)) current_doc_metrics["TTR"] = (unique_lemmas_ld / total_lemmas_ld) current_doc_metrics["herdan_c"] = (math.log(unique_lemmas_ld) / math.log(total_lemmas_ld)) Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 78 current_doc_metrics["guiraud_r"] = (unique_lemmas_ld / math.sqrt(total_lemmas_ld)) # Lexical Density content_pos = {'NOUN', 'VERB', 'ADJ', 'ADV', 'NUM', 'PROPN'} words_for_ld = words_no_punct content_word_tokens_for_ld = [token for token in words_for_ld if token.pos_ in content_pos] current_doc_metrics["lexical_density"] = (len(content_word_tokens_for_ld) / len(words_for_ld)) # Readability current_doc_metrics["FRE"] = textstat.flesch_reading_ease(raw_text) current_doc_metrics["FKGL"] = textstat.flesch_kincaid_grade(raw_text) # Avg Sentence Length num_sentences = len(list(doc.sents)) current_doc_metrics["num_sentences"] = num_sentences current_doc_metrics["avg_sentence_length"] = (current_doc_metrics["total_words_excl_punct"] / num_sentences) # Avg Word Length total_chars_in_words = sum(len(token.text) for token in words_no_punct) current_doc_metrics["avg_word_length"] = (total_chars_in_words / len(words_no_punct)) # Relative Frequency of Function Words function_pos_categories = ['ADP','AUX','CCONJ','SCONJ','DET','PRON','PART', 'INTJ'] function_word_count = sum(token.pos_ in function_pos_categories for token in words_no_punct) current_doc_metrics["rel_freq_function_words"] = (function_word_count / current_doc_metrics["total_words_excl_punct"]) * 100 # POS Tags Distribution pos_tags_list = [token.pos_ for token in words_no_punct] pos_counts = Counter(pos_tags_list) total_words_for_pos = len(pos_tags_list) major_pos_cats = ['NOUN', 'VERB', 'AUX', 'ADJ', 'ADV', 'PRON', 'ADP', 'CCONJ', 'SCONJ'] for pos_cat in major_pos_cats: current_doc_metrics[f"pos_{pos_cat}"] = (pos_counts.get(pos_cat, 0) / total_words_for_pos) * 100 current_doc_metrics["pos_OTHER"] = sum(count for tag, count in pos_counts.items() if tag not in major_pos_cats and tag != 'SPACE') / total_words_for_pos * 100 all_doc_metrics_data.append(current_doc_metrics) # Appending text for TF-IDF tfidf_text = " ".join([token.lemma_.lower() for token in doc if not token.is_punct and not token.is_space and not token.is_stop]) corpus_for_tfidf.append(tfidf_text) # Configuration TF-IDF vectorizer = TfidfVectorizer(max_features=2000) tfidf_matrix = vectorizer.fit_transform(corpus_for_tfidf) feature_names = vectorizer.get_feature_names_out() num_top_tfidf_terms = 10 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 79 for i, doc_name_from_order in enumerate(doc_names): metrics_dict_for_doc = all_doc_metrics_data[i] doc_tfidf_scores = tfidf_matrix[i].toarray().flatten() top_indices = doc_tfidf_scores.argsort()[-num_top_tfidf_terms:][::-1] top_terms_scores = [(feature_names[j], doc_tfidf_scores[j]) for j in top_indices if doc_tfidf_scores[j] > 0.0001] metrics_dict_for_doc[f"top_{num_top_tfidf_terms}_tfidf_terms"] = "; ".join([f"{term}:{score:.4f}" for term, score in top_terms_scores]) # Conversion to DF and saving as CSV for analysis metrics_df = pd.DataFrame(all_doc_metrics_data) metrics_df.to_csv(OUTPUT_CSV, index=False) print(f"Metrics successfully saved to: {OUTPUT_CSV}") Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 80 Appendix G: [Table] Top 10 TF-IDF Scores per Document IPCC SPMs TF-IDF Terms Wikipedia Articles TF-IDF Terms AR1_WG3_SPM emission:0.3439 change:0.2305 gas:0.2026 energy:0.1934 climate:0.1934 country:0.1729 greenhouse:0.1655 use:0.1580 develop:0.1432 resource:0.1364 N/A N/A AR2_WG3_SPM cost:0.3520 change:0.2900 climate:0.2848 country:0.2090 emission:0.2012 damage:0.1663 economic:0.1542 equity:0.1427 policy:0.1333 estimate:0.1259 N/A N/A AR3_WG3_SPM cost:0.3231 emission:0.2692 mitigation:0.1977 gas:0.1862 change:0.1805 scenario:0.1569 reduction:0.1506 climate:0.1489 country:0.1461 carbon:0.1432 Wiki_CCM_2005-06-26 global:0.2379 2005:0.2301 warming:0.2259 energy:0.2221 change:0.2062 kyoto:0.1905 climate:0.1904 emission:0.1904 carbon:0.1745 protocol:0.1738 AR4_WG3_SPM emission:0.3420 mitigation:0.2347 eq:0.2250 spm:0.1957 co2:0.1912 energy:0.1789 cost:0.1443 global:0.1356 ghg:0.1348 potential:0.1313 Wiki_CCM_2008-05-07 energy:0.3701 carbon:0.2628 emission:0.2295 climate:0.2073 global:0.2073 warming:0.1986 change:0.1851 gas:0.1591 reduce:0.1517 power:0.1517 AR5_WG3_SPM emission:0.3030 mitigation:0.2447 scenario:0.2330 energy:0.1943 evidence:0.1911 high:0.1676 co2eq:0.1631 spm:0.1573 medium:0.1508 figure:0.1444 Wiki_CCM_2014-09-02 emission:0.3501 energy:0.3369 climate:0.2290 carbon:0.2092 change:0.1982 global:0.1850 gas:0.1652 power:0.1608 nuclear:0.1592 2011:0.1240 AR6_WG3_SPM confidence:0.4142 emission:0.4038 high:0.2604 mitigation:0.2071 pathway:0.1906 ghg:0.1851 global:0.1670 spm:0.1445 warm:0.1135 2019:0.1118 Wiki_CCM_2022-06-13 emission:0.3276 climate:0.3189 energy:0.2506 change:0.2488 carbon:0.2488 global:0.1629 reduce:0.1489 gas:0.1489 mitigation:0.1324 2021:0.1090 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 81 Appendix H: [Python code] Expression of Modality Processing import spacy from spacy.tokens import DocBin from collections import Counter import os import re import pandas as pd # Configuration INPUT_DIR = "processed_docs/" OUTPUT_CSV = "modality_metrics.csv" NORMALIZATION_FACTOR = 1000 MODAL_VERBS = [ "can", "could", "may", "might", "must", "shall", "should", "will", "would" ] MODAL_ADVERBS = ["apparently", "arguably", "assuredly", "certainly", "clearly", "conceivably", "definitely", "doubtless", "evidently", "hopefully", "indubitably", "ineluctably", "inescapably", "incontestably", "likely", "manifestly", "maybe", "necessarily", "obviously", "patently", "perhaps", "plainly", "possibly", "presumably", "probably", "seemingly", "surely", "truly", "unarguably", "unavoidably", "undeniably", "undoubtedly", "unquestionably" ] CONFIDENCE_DICT = { "very high confidence": "90-100%", "high confidence": "80%", "medium confidence": "50%", "low confidence": "20%", "very low confidence": "0-10%" } LIKELIHOOD_DICT = { "virtually certain": "99-100%", "extremely likely": "95-100%", "very likely": "90-100%", "likely": "66-100%", "more likely than not": "50-100%", "about as likely as not": "33-66%", "unlikely": "0-33%", "very unlikely": "0-10%", "extremely unlikely": "0-5%", "exceptionally unlikely": "0-1%" } nlp = spacy.load("en_core_web_lg") all_results_data = [] doc_files = [f for f in os.listdir(INPUT_DIR) if f.endswith(".spacy")] for filename in sorted(doc_files): file_path = os.path.join(INPUT_DIR, filename) doc_name = os.path.basename(filename) print(f"Currently processing: {doc_name} !") doc_bin_loaded = DocBin().from_disk(file_path) loaded_docs_from_file = list(doc_bin_loaded.get_docs(nlp.vocab)) doc = loaded_docs_from_file[0] current_likelihood_counts = Counter() current_confidence_counts = Counter() current_modal_verb_counts = Counter() current_modal_adverb_counts = Counter() current_negation_modal_count = 0 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 82 text_content_from_doc = doc.text lower_text_content = text_content_from_doc.lower() for phrase in LIKELIHOOD_DICT.keys(): pattern = r'\b' + re.escape(phrase.lower()) + r'\b' matches = re.findall(pattern, lower_text_content) if matches: current_likelihood_counts[phrase] += len(matches) for phrase in CONFIDENCE_DICT.keys(): pattern = r'\b' + re.escape(phrase.lower()) + r'\b' matches = re.findall(pattern, lower_text_content) if matches: current_confidence_counts[phrase] += len(matches) for token in doc: if not token.is_punct and not token.is_stop: current_total_words += 1 if token.pos_ == "AUX" and token.lemma_ in MODAL_VERBS: current_modal_verb_counts[token.lemma_] += 1 # Negation check if token.i + 1 < len(doc) and doc[token.i + 1].lemma_ == "not": current_negation_modal_count += 1 elif token.head.lemma_ == "not" and token.head.i == token.i -1 : current_negation_modal_count += 1 if token.pos_ == "ADV" and token.lemma_ in MODAL_ADVERBS: current_modal_adverb_counts[token.lemma_] += 1 for term, count in current_likelihood_counts.items(): norm_freq = (count / current_total_words) * NORMALIZATION_FACTOR all_results_data.append([doc_name, "Likelihood", term, count, norm_freq, current_total_words]) for term, count in current_confidence_counts.items(): norm_freq = (count / current_total_words) * NORMALIZATION_FACTOR all_results_data.append([doc_name, "Confidence", term, count, norm_freq, current_total_words]) for term, count in current_modal_verb_counts.items(): norm_freq = (count / current_total_words) * NORMALIZATION_FACTOR all_results_data.append([doc_name, "Modal Verbs", term, count, norm_freq, current_total_words]) for term, count in current_modal_adverb_counts.items(): norm_freq = (count / current_total_words) * NORMALIZATION_FACTOR all_results_data.append([doc_name, "Modal Adverbs", term, count, norm_freq, current_total_words]) norm_freq_neg = (current_negation_modal_count / current_total_words) * NORMALIZATION_FACTOR all_results_data.append([doc_name, "Negation Near Modal", "count", current_negation_modal_count, norm_freq_neg, current_total_words]) # --- Create Pandas DataFrame and Print/Save --- metrics_df = pd.DataFrame(all_results_data, columns=[ "Document", "Modal_Category", "Term", "Raw_Count", f"Normalized_Freq_{NORMALIZATION_FACTOR}", "Total_Words" ]) metrics_df.to_csv(OUTPUT_CSV, index=False) print(f"Metrics successfully saved to: {OUTPUT_CSV}") Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 89 df_combined = pd.DataFrame(all_document_metrics) column_order = ["document", "total_paragraphs", "avg_VADER_compound", "VADER_positive", "VADER_negative", "VADER_neutral", "roberta_positive", "roberta_negative", "roberta_neutral"] emotion_cols_ordered = [f"emotion_{label}" for label in ["anger", "disgust", "fear", "joy", "neutral", "sadness", "surprise"]] column_order.extend(emotion_cols_ordered) existing_columns_in_order = [col for col in column_order if col in df_combined.columns] df_combined = df_combined[existing_columns_in_order] df_combined.to_csv(OUTPUT_CSV, index=False) print(f"Done. Metrics saved to {OUTPUT_CSV}") Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 90 Appendix L: [Python code] Named Entity Recognition Processing import os import spacy from spacy.tokens import DocBin from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline, logging as hf_logging import pandas as pd from collections import Counter import torch # Configuration INPUT_DIR_SPACY = "processed_docs/" SPLIT_TEXT_DIR = "split_texts/" OUTPUT_CSV = "results_ner.csv" MODEL_CC = "nicolauduran45/specter-climate-change-NER" nlp = spacy.load("en_core_web_lg") tokenizer_hf = AutoTokenizer.from_pretrained(MODEL_CC) model_hf = AutoModelForTokenClassification.from_pretrained(MODEL_CC) # Config for CC NER cc_ner_pipeline = pipeline( "ner", model=model_hf, tokenizer=tokenizer_hf, aggregation_strategy="simple", device=0 ) all_results = [] doc_files = [f for f in os.listdir(INPUT_DIR_SPACY) if f.endswith(".spacy")] # Main loop for filename in sorted(doc_files): filepath = os.path.join(INPUT_DIR_SPACY, filename) doc_name = os.path.basename(filename) # ----------------------- Part 1 ----------------------- # ------------------------------------------------------ print(f"Processing: {doc_name}") doc_bin = DocBin().from_disk(filepath) loaded_docs = list(doc_bin.get_docs(nlp.vocab)) doc_obj = loaded_docs[0] # Total words from spaCy will be used for split_texts paragraphs as well # No need to recount words as texts are the same total_words = 0 for token in doc_obj: if not token.is_punct and not token.is_stop: total_words += 1 result_row = {'Document': doc_name, 'Total Words': total_words} Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 91 spacy_ents = [ent.label_ for ent in doc_obj.ents] raw_ent_counts_spacy = Counter(spacy_ents) normalized_ent_counts_spacy = Counter() if total_words > 0: for ent_type, count in raw_ent_counts_spacy.items(): normalized_ent_counts_spacy[ent_type] = (count / total_words) * 1000 else: for ent_type, count in raw_ent_counts_spacy.items(): normalized_ent_counts_spacy[ent_type] = 0.0 for ent_type, count in raw_ent_counts_spacy.items(): result_row[f'{ent_type}_spacy_raw'] = count for ent_type, norm_count in normalized_ent_counts_spacy.items(): result_row[f'{ent_type}_spacy_norm'] = norm_count print(f"Found {len(doc_obj.ents)} entities with spaCy !") # ----------------------- Part 2 ----------------------- # ------------------------------------------------------ split_text_filename = doc_name.replace(".spacy", ".txt") split_text_path = os.path.join(SPLIT_TEXT_DIR, split_text_filename) raw_ent_counts_cc = Counter() normalized_ent_counts_cc = Counter() total_cc_ent_doc = 0 if os.path.exists(split_text_path): with open(split_text_path, 'r', encoding='utf-8') as f: split_text = f.read() if split_text.strip(): paragraphs = [p.strip() for p in split_text.split('\n\n') if p.strip()] if not paragraphs: print(f"Error: No paragraphs found") else: all_ent_classes = [] for i, paragraph_text in enumerate(paragraphs): # CC NER processing paragraph_results = cc_ner_pipeline(paragraph_text) for ent in paragraph_results: all_ent_classes.append(ent['entity_group']) total_cc_ent_doc += len(paragraph_results) raw_ent_counts_cc = Counter(all_ent_classes) print(f"CC NER complete for {len(paragraphs)} paragraphs. Found {total_cc_ent_doc} entities.") if total_words > 0: for ent_type, count in raw_ent_counts_cc.items(): normalized_ent_counts_cc[ent_type] = (count / total_words) * 1000 else: for ent_type, count in raw_ent_counts_cc.items(): normalized_ent_counts_cc[ent_type] = 0.0 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 92 else: print(f"Cannot read: '{split_text_path}'. Check content or encoding and try again") else: print(f"file not found: '{split_text_path}'. Check paths and try again") for ent_type, count in raw_ent_counts_cc.items(): result_row[f'{ent_type}_cc_raw'] = count for ent_type, norm_count in normalized_ent_counts_cc.items(): result_row[f'{ent_type}_cc_norm'] = norm_count all_results.append(result_row) # Convert to DF results_df = pd.DataFrame(all_results) results_df = results_df.fillna(0) fixed_cols = ['Document', 'Total Words'] all_ent_prefixes = set() for col_name in results_df.columns: if col_name not in fixed_cols: parts = col_name.split('_') if len(parts) > 1: prefix_candidate = "_".join(parts[:-1]) all_ent_prefixes.add(prefix_candidate) sorted_ent_prefixes = sorted(list(all_ent_prefixes)) ent_cols = [] for prefix in sorted_ent_prefixes: raw_col_name = f'{prefix}_raw' norm_col_name = f'{prefix}_norm' if raw_col_name in results_df.columns: ent_cols.append(raw_col_name) if norm_col_name in results_df.columns: ent_cols.append(norm_col_name) final_columns = fixed_cols + ent_cols results_df = results_df[[col for col in final_columns if col in results_df.columns]] results_df.to_csv(OUTPUT_CSV, index=False, encoding='utf-8') print(f"Done. All metrics saved to: '{OUTPUT_CSV}'") Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 93 Appendix M: [Table] Complete spaCy NER Results Frequencies normalized per 1000 words (punct. and stop-words excluded), rounded to 3 d.p. Document CARDINAL DATE EVENT FAC GPE LANGUAGE AR1_WG3_SPM 25.323 7.409 0.442 0.111 1.106 0.000 AR2_WG3_SPM 10.211 4.500 0.173 0.000 0.173 0.000 AR3_WG3_SPM 30.983 11.785 0.190 0.570 1.901 0.000 AR4_WG3_SPM 52.065 14.222 0.247 0.124 1.113 0.000 AR5_WG3_SPM 59.650 24.669 0.404 0.202 1.921 0.000 AR6_WG3_SPM 24.637 17.797 0.140 0.349 0.628 0.000 Wiki_CCM_2005-06-26 13.004 33.810 2.601 0.000 42.913 0.000 Wiki_CCM_2008-05-07 12.539 20.376 1.120 0.224 16.346 0.000 Wiki_CCM_2014-09-02 19.389 30.698 0.746 0.249 13.547 0.497 Wiki_CCM_2022-06-13 17.367 20.583 0.643 0.184 11.578 0.000 Document LAW LOC MONEY NORP ORDINAL ORG AR1_WG3_SPM 1.769 1.659 0.885 0.221 1.216 15.371 AR2_WG3_SPM 1.558 0.173 1.211 0.346 1.211 5.365 AR3_WG3_SPM 3.802 0.570 3.421 0.950 0.190 17.107 AR4_WG3_SPM 1.731 1.360 2.350 0.495 2.350 22.755 AR5_WG3_SPM 1.618 0.101 1.921 0.607 0.708 19.412 AR6_WG3_SPM 1.256 0.349 0.209 1.047 0.698 22.334 Wiki_CCM_2005-06-26 7.802 3.901 1.300 2.601 1.300 45.514 Wiki_CCM_2008-05-07 2.463 2.239 2.687 1.567 2.463 31.572 Wiki_CCM_2014-09-02 3.107 1.119 2.610 2.237 1.740 41.511 Wiki_CCM_2022-06-13 1.930 2.205 2.113 3.124 0.368 22.972 Document PERCENT PERSON PRODUCT QUANTITY TIME WORK_OF_ART AR1_WG3_SPM 5.529 0.663 0.995 2.543 0.111 0.442 AR2_WG3_SPM 2.942 0.000 0.000 0.000 0.000 0.173 AR3_WG3_SPM 2.661 1.711 5.132 1.140 0.190 0.190 AR4_WG3_SPM 7.915 2.473 5.813 2.226 1.360 0.371 AR5_WG3_SPM 10.312 2.831 2.730 2.629 0.101 0.000 AR6_WG3_SPM 16.052 0.768 5.793 0.489 0.070 0.209 Wiki_CCM_2005-06-26 6.502 5.202 0.000 0.000 0.000 0.000 Wiki_CCM_2008-05-07 6.941 11.196 3.135 4.254 0.896 1.120 Wiki_CCM_2014-09-02 5.841 15.287 2.859 2.610 0.249 0.621 Wiki_CCM_2022-06-13 12.037 5.881 1.562 3.032 0.368 0.459 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 94 Appendix N: [Table] Complete Climate Change NER Results Frequencies normalized per 1000 words (punct. and stop-words excluded), rounded to 3 d.p. Document climate-assets climate-datasets climate-greenhouse-gases AR1_WG3_SPM 20.126 6.303 12.275 AR2_WG3_SPM 12.288 1.385 3.461 AR3_WG3_SPM 7.603 25.470 13.496 AR4_WG3_SPM 14.222 19.664 22.384 AR5_WG3_SPM 20.119 31.847 20.928 AR6_WG3_SPM 20.310 21.078 23.032 Wiki_CCM_2005-06-26 18.205 2.601 6.502 Wiki_CCM_2008-05-07 17.689 0.224 20.376 Wiki_CCM_2014-09-02 12.304 2.983 21.999 Wiki_CCM_2022-06-13 22.604 1.930 7.535 Document climate-impacts climate-mitigations climate-models AR1_WG3_SPM 1.106 40.142 1.548 AR2_WG3_SPM 10.211 31.499 0.173 AR3_WG3_SPM 1.331 50.561 6.653 AR4_WG3_SPM 0.989 57.878 13.480 AR5_WG3_SPM 0.809 46.608 11.425 AR6_WG3_SPM 0.907 57.300 10.050 Wiki_CCM_2005-06-26 2.601 71.521 1.300 Wiki_CCM_2008-05-07 2.463 74.116 6.270 Wiki_CCM_2014-09-02 1.740 71.837 10.689 Wiki_CCM_2022-06-13 1.838 77.001 5.329 Document climate-nature climate-observations climate-organisms AR1_WG3_SPM 13.712 0.553 1.106 AR2_WG3_SPM 2.596 0.000 0.346 AR3_WG3_SPM 6.273 0.000 0.380 AR4_WG3_SPM 8.410 0.000 0.618 AR5_WG3_SPM 3.134 0.101 0.708 AR6_WG3_SPM 5.095 0.349 1.396 Wiki_CCM_2005-06-26 6.502 1.300 0.000 Wiki_CCM_2008-05-07 12.987 0.224 2.015 Wiki_CCM_2014-09-02 9.197 0.746 1.989 Wiki_CCM_2022-06-13 17.183 0.643 4.962 Document climate-organizations climate-problem-origins climate-properties AR1_WG3_SPM 19.020 42.243 7.851 AR2_WG3_SPM 7.442 16.961 5.365 AR3_WG3_SPM 11.595 37.635 7.983 AR4_WG3_SPM 10.017 49.963 23.621 AR5_WG3_SPM 9.099 33.970 22.040 AR6_WG3_SPM 6.281 38.945 13.819 Wiki_CCM_2005-06-26 55.917 9.103 3.901 Wiki_CCM_2008-05-07 55.083 38.065 8.285 Wiki_CCM_2014-09-02 47.974 39.150 14.666 Wiki_CCM_2022-06-13 20.674 43.646 10.567 Document climate-hazards AR1_WG3_SPM 10.727 AR2_WG3_SPM 8.827 AR3_WG3_SPM 4.372 AR4_WG3_SPM 4.081 AR5_WG3_SPM 3.235 AR6_WG3_SPM 2.931 Wiki_CCM_2005-06-26 29.909 Wiki_CCM_2008-05-07 11.420 Wiki_CCM_2014-09-02 6.587 Wiki_CCM_2022-06-13 11.853 Lucas PRÉVOT | Master’s Thesis | Université de Strasbourg | 2023-2025 95 Appendix O: [Python code] Data Visualisation Instead of using matplotlib in each script, we used a single script in which we manually changed the values for each figure. This appendix contains the original function, with generic values and all parameters set to default. import matplotlib.pyplot as plt # Document dates years_ipcc = [1990, 1995, 2001, 2007, 2014, 2022] years_ccm = [2005, 2008, 2014, 2022] # Document names ipcc_labels = ['AR1', 'AR2', 'AR3', 'AR4', 'AR5', 'AR6',] ccm_labels = ['CCM 2005', 'CCM 2008', 'CCM 2014', 'CCM 2022'] # Values ipcc_val = [1, 2, 3, 4, 5, 6] ccm_val = [1, 2, 3, 4] # Plot size plt.figure(figsize=(8, 5)) # Style plt.plot(years_ipcc, ipcc_val, marker='o', color='blue', label='IPCC WG3 SPM') plt.plot(years_ccm, ccm_val, marker='o', color='purple', label='Wikipedia CCM') # Text label positions are set individually by hand otherwise it overlaps # IPCC plt.annotate(ipcc_labels[0], (years_ipcc[0], ipcc_val[0]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='blue') plt.annotate(ipcc_labels[1], (years_ipcc[1], ipcc_val[1]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='blue') plt.annotate(ipcc_labels[2], (years_ipcc[2], ipcc_val[2]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='blue') plt.annotate(ipcc_labels[3], (years_ipcc[3], ipcc_val[3]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='blue') plt.annotate(ipcc_labels[4], (years_ipcc[4], ipcc_val[4]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='blue') plt.annotate(ipcc_labels[5], (years_ipcc[5], ipcc_val[5]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='blue') # Wiki plt.annotate(ccm_labels[0], (years_ccm[0], ccm_val[0]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='purple') plt.annotate(ccm_labels[1], (years_ccm[1], ccm_val[1]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='purple') plt.annotate(ccm_labels[2], (years_ccm[2], ccm_val[2]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='purple') plt.annotate(ccm_labels[3], (years_ccm[3], ccm_val[3]), xytext=(0, 0), textcoords='offset points', fontsize=9, color='purple') # Title plt.xlabel('Year') plt.ylabel("Current Metric") plt.title("Title of Metric + Over Time") plt.legend() plt.grid(True, linestyle='--', linewidth=0.5, alpha=0.3) plt.tight_layout() plt.show()