Fusion of Retrieval, Grammar Rules and Decision Trees for Text Generation

FUSION OF RETRIEVAL, GRAMMAR RULES AND DECISION TREES FOR TEXT GENERATION E. Niehaus†, W. Kühn, S. Müller University of Kaiserslautern-Landau (RPTU), Landau, Germany Abstract The generation of scientific documents is accompanied by decisions of the author, e.g. the type of paper, publishing journal, and selection of an appropriate methodology. This generates a decision tree. Algorithmic support can provide options that the author selects. Combining this approach with Generative Artificial Intelligence (GenAI) , that is applied to trees and rules, results in a syntactic and a semantic structure. This conceptual paper discusses the fusion of Retrieval Augmented Generation (RAG), grammars, and decision trees with citable tree nodes. Tree nodes are defined by grammar rules with or without decision options and a user driven or randomized selection. The transparency of text generation is supported by a version control for the documentation of the paper evolution. INTRODUCTION The increasing application of GenAI [1] in the scientific domains requires transparency for the document evolution. Citations are used as a standard method to refer to publications and new scientific results based on the existing scientific knowledge. In the evolution of scientific publications, it is necessary to have transparency about the algorithmic support of Large Language Models (LLMs) and the human collaborative effort of the involved scientists. Publications in proceedings share requirements for the scientific structure of papers. The conceptual design of algorithmically generated syntactical options and decisions of an author generates a decision tree and simultaneously an Abstract Syntax Tree (AST) of the generation process. Decisions might be reverted, so tracking the changes is required for a transparent generation process. In this concept, the generation process can include algorithmic steps and steps created under control of the author (e.g., the modification of the generated text and the adaptation to the requirements and constraints of the paper). An alternating version in a version control system (VCS) [2] can be used to validate a collaborative modification history. Submissions of papers can be accompanied by a repository with version control. To be compliant with a transparent history of the evolution, citable grammar rules can be referred to in the document, similar to citations, as a grammar list. A tree node that was populated by GenAI requires the prompt, the used LLM, and the product of the LLM as part of the Version Control Repository. Even a more advanced use, like in software development, can be used with citable references in the version history. Branching from a specific version towards a new objective or an advancement of the current state of the art can be used with version control, which accelerate document development and text production. In graph theory, the concept of a graph can be applied to an AST of the document structure (e.g., “PAPER” expands to 5 sections: “INTRODUCTION”, “METHODOLOGY”, “DATA ANALYSIS” “RESULTS” and “CONCLUSION”). The bottom-up strategy in computer science is used in a compiler that parses the code of a library and checks if the code is syntactical correct. For this paper we follow a top-down strategy, starting with start node as the root (“DOCUMENT”) and creating decision options for the author, e.g. “Write a scientific paper” or “Write a poem” or “Write a novel”. The decision option at the root node of the graph creates a version in the VCS, providing all decision options and saving the selected option. Reverting changes allows going back in the decision tree and changing the option. Version control supports this work-in-progress through a transparent evolutionary protocol. This is especially in the educational system an approach for assessment because it allows tracking the changes and the contribution (especially in a collaborative work). Decision options applied grammar rules, and applications of LLMs to populate a node with text paragraphs can be complemented with RAG. The retrieval with keywords provided by the author or by a keyword extraction out of the generated paragraph or tree node descriptions creates, for example references retrieved from a local database of references. Matching references are provided to the authors as a decision option. Any new retrieval with a selection process can create a new version in the VCS. The classical compiler involves a bottom-up process for parsing and a top-down process to generate from a program written in source language A into a functional equivalent code into a programming language B. A LLM can be used to extract parts of the syntactical structure of documents. This can be applied to sections in novels where the protagonist's sections are responsible for a particular activity that can be summarized or even tokenized by a particular grammar rule. Comparisons between novels of the same structure can be performed as preliminary steps for a scientific analysis by a literary scholar. This shows the link between a bottom-up and a top-down application of decision and ASTs for generation and analysis. Considering the technical process, a compilation of source code requires a tokenizer first to convert the given source code into an array of tokens. The list of tokens is transformed into an AST [3]. A parser reduces the string of the code, which is written in language A, to a start symbol S. A specific set of grammar rules is implemented for this ___________________________________________ † email: [email protected] https://doi.org/10.5281/zenodo.17237756 purpose. In the conceptual design, the parsing of existing documents can be performed bottom-up while generation is top-down from the basic idea written in the first prompt towards a complete document an author wants to create. This selection process can also be influenced by the system prompt in GenAI, making it an integral part of the version control history. The fully controlled part of the document generation can be accomplished by searching in a database for an appropriate grammar that serves a specific document generation task. In general, a grammar consists of a set of non-terminal symbols N, a set of terminal symbols T, a system of rules R, and a start symbol S as an element of N. An AST represents the parsing process of the source code after the analysis of syntax and semantic. In the next step, the generated (and semantically attributed) AST is used to create an output source code in language B. The output language B can be binary/executable or a low-level programming language. Thus far, the parsing of source code in A and generation in code B can be distinguished conceptually. In contrast to a bottom-up process, the top-down process for text generation creates decision options for a non-terminal symbol 𝑆𝑆0, that can be replaced by 𝑆𝑆1,𝑆𝑆2,𝑆𝑆3 or 𝑆𝑆4. In the graph structure of the generated document, the selection of the author is documented in the VCS. After the selection, the other non-terminal symbols are not considered for the next steps of the document generation. The selection process is a difficult decision. A probability distribution over the decision options determined by empirical data for an organisation can support automated selection by a Monte Carlo approach [4] in conjunction with a fuzzy logic approach for describing the acceptance of natural language elements and their matching with grammar rules [5]. BASIC EXAMPLE OF CHOICES As a example, it uses a non-terminal symbol S, that can be replaced by one of two decision options, as rule (1) 𝑆𝑆 → 𝑆𝑆𝑆𝑆𝑆𝑆 ∣𝑁𝑁𝑁𝑁𝑁𝑁 (1) The decision process allows the selection of a scientific paper SCI or a novel as a cultural contribution. The VCS stores the selection of rule (1) and the decision of the author to write a scientific paper. So, in the generation process, S is replaced by SCI and not by NOV. SCI represents a scientific article, and MA is a manual, e.g. for application of a workflow. This second rule (2) is branching into a sequence of five non-terminal symbols without a choice for the author. This replacement can be automated without user interaction. 𝑆𝑆𝑆𝑆𝑆𝑆 → 𝑆𝑆𝐴𝐴𝑆𝑆𝐼𝐼𝑆𝑆𝑀𝑀𝑆𝑆𝑅𝑅𝑆𝑆𝐶𝐶 (2) Such a deterministic expansion will not create a version in the VCS until another author decision is required, or GenAI is used to populate the non-terminal symbol with a LLM and a prompt. Rule (2) defines the grammar structure of a scientific paper. A scientific paper in this definition of grammar consists of five sections represented by the nonterminal symbols: •𝑆𝑆𝐴𝐴 is the section Abstract, •𝑆𝑆𝐼𝐼 the section Introduction, •𝑆𝑆𝑀𝑀 the section Methodology, •𝑆𝑆𝑅𝑅 the section Results and •𝑆𝑆𝐶𝐶 represents the section Conclusion. For example, above the abstract of the paper denoted by the non-terminal symbol 𝑆𝑆𝐴𝐴 can be generated by a summary of the other section by using a LLM. So, summarizing the basic example above, it is necessary to distinguish four types of non-terminal symbols. •(ONT) optional non-terminal symbols allow user selection of provided choices in the rule – see rule (1), •(DNT) deterministic non-terminal symbols have no options and can be applied directly expanding a given non-terminal symbol 𝑆𝑆𝑆𝑆𝑆𝑆 by a sequence of symbols - see rule (2), •(PNT) probabilistic non-terminal symbols have a list of options with a probability distribution over the decision options, and one of the options is selected by a Monte Carlo approach, •(RNT) retrieval non-terminal symbols have a query call to a search engine and return a list of choices. The author is asked to select one for further text generation and inclusion of one or more selected options. The selection process is similar to rule (1), but it dependents on a retrieval call, •(LLNT) Large Language non-terminal symbols have a prompt and/or system prompt with a selected LLM. If the generation process reaches such a tree node, the prompt submits the current generation context to the LLM and the call of the LLM populates the tree node with generated text. The non-terminal symbol of the type LLNT might also call other types of non-terminal symbols, such as PNT, DNT, or RNT. The generation process stops when author interaction is required. This is the case, when editing or reviewing of a section is required (such as the methodology section of a scientific paper) needs a quality assurance. Validation and reviewing of sections or generation steps will be documented in the VCS by adding a validation flag for a single author within a collaborative team. CITATION OF GRAMMAR RULES The different types of non-terminal symbols require different levels of citation. According to transparency, it is necessary to distinguish between a Monte Carlo selection of choices where an intellectual knowledge of the author is https://doi.org/10.5281/zenodo.17237756 not involved and an ONT or RNT where the author has to select from a given number of choices. An author might want to annotate a tree node with a comment, stating why a specific choice is the most appropriate. It might also happen that none of the choices provided by the grammar rule ONT or RNT is appropriate, and the author provides in a traditional manner a section for the generated document. This step requires the most knowledge of the author, because the support was not useful. At the same time, edited options might be added to the local RAG system as an option because they might be reused in further documents in the authors’ domain. This can be helpful for a transparent version history of reusable non-terminal symbols and tree node options as choices for upcoming text generations of the collaborative team. For PNT the probability distribution for an OR-expression, as in rule (1), can be defined as follows: The option SCI is assigned a probability of 0.7, while the replacement of S by NOV has the probability of 0.3. A generated random number less than 0.7 will result in the replacement of S by SCI, whereas a random number greater than or equal to 0.7 will lead to the replacement of S by NOV. TEXT GENERATION – PROBABILITY Text generation can be dependent on random experiments in an AST, where the rule allows choices. A probability distribution on a finite set of options describes this mathematically. The text generation creates decision numbers that are used in the generative process when optional cases in a rule are possible. For the rule 𝑆𝑆 → 𝑆𝑆𝑆𝑆𝑆𝑆 ∣𝑀𝑀𝑀𝑀 a random number r between 0 and 1 with a uniform distribution on the interval [0,1] determines the replacement for a start symbol S. SCI is selected if r is smaller than 0.7, and MA is selected otherwise. The interval [0,1] is decomposed into n subintervals for n different options in the grammar rule. To ensure transparency in the derivation process from a tree node in an AST to child nodes, these random numbers (𝑟𝑟 1,𝑟𝑟2,…,𝑟𝑟 𝑛𝑛 for n) for n different random experiments should be explicitly assigned to the corresponding rules (𝑅𝑅1,𝑅𝑅2,…,𝑅𝑅𝑛𝑛) of the grammar. DOCUMENT OBJECT IDENTIFIER The different types of grammar rules extend the classical concept of a grammar. For a transparent version history, all non-terminal rules are regarded as a digital object that can be selected for application in text generation. Generalizing the approach conceptually, a digital object identifier (DOI) can be used to create a unique and persistent identifier for a rule and/or the corresponding subtree of an AST. Similar to page references in books, we extend probabilistic or decision-making processes of the author. Selecting an option is part of the citation process for grammar rules. Due to the fact, that DOI can handle various digital objects [6], the generation of multimedia documents, that include digital objects like audio, animation, data, etc., a rule for text generation can be referred to consistently. DOI is standardized by the International Organization for Standardization (ISO) and is an existing implementation of the Handle System, so a reference for grammar rules follows an established work. It ensures a transparent history of a grammar rule by providing a unique identifier that allows retrieval of the rule and its corresponding subtree. Since the DOI functions consistently within the Uniform Resource Identifier framework, it enables reliable referencing and access to these elements. This paper allows generative models to be transparent if they are applied on the root file of academic, professional, and government documents and also on the decomposition of the documents reflecting generative processes transparently. CONTEXT DEPENDENT GRAMMAR Context-free or -dependent grammars represent the syntactical structure of the document or a language [7]. In compiler theory, an application of grammars are relevant for the replacement and application of a rule. In this context, we consider the current state of the decision tree or AST as the context in which the next steps for the text generation are performed. The existing validation of parts of the tree nodes in the AST or the decision tree affects the priority and relevance of the automated prompt selection for LLNT or RNT. APPLICATION OF THE CONCEPT IN LITERARY STUDIES In recent years, Literary Studies have transitioned from a model of individual research to interdisciplinary collaboration. The Digital Humanities demonstrated that traditional humanities methodologies can be expanded through the use of algorithmic processes. The integration of RAG and decision trees offers new possibilities for text generation and structuring, documentation, and analysis of scholarly arguments. A central challenge in literary scholarship is ensuring the transparency and versioning of argumentative processes. Interpretations and methodological choices emerge through an iterative process of evaluating theories and readings. The fusion of retrieval grammar rules and decision trees could serve as a model for a documented research practice. Versioned decision trees can enhance transparency in scholarly work by making interpretive pathways explicit. Automated source retrieval through RAG could facilitate this process by suggesting relevant scholarly texts and integrating them into the decision-making structure. Additionally, the structuring of literary arguments can be improved by formalizing academic texts through ASTs and decision rules, thereby making recurring argumentative patterns identifiable. Furthermore, literary studies provide a compelling testbed for generative models. Algorithmic versioning could, for example, increase transparency by tracking the evolution of research questions over time and highlighting paradigm shifts. Recent research has emphasized the role of human-machine collaboration in shaping new research methodologies [8]. These findings align with the applicahttps://doi.org/10.5281/zenodo.17237756 tion of retrieval-based decision trees, as they offer a framework where human interpretative agency and algorithmic support coalesce to foster more structured, transparent, and reproducible scholarly work. The increasing collaboration between literary scholars, computer scientists, and data scientists underscores the relevance of these methodologies. The combination of humanities and computer science perspectives opens up new methodological possibilities and calls for a critical reflection on the epistemological foundations of both disciplines. While classical philology long relied on individual hermeneutic analysis, contemporary literary research is shifting towards collaborative, technology-assisted approaches. This development aligns with the broader “laboratory turn” in the humanities, where research environments are increasingly modelled after scientific laboratories, fostering interdisciplinary exchange and methodological innovation. As Pawlicka-Deger states: “The humanities lab does not simply imitate the science lab but adapts this new infrastructure for its own purposes and needs.” [9] This shift also affects methodological frameworks: “The laboratory turn has emerged […] as a part of a wider process of the laboratoriation of social life, which has been occurring since the 1980s and with a significant intensification in the last ten years.” [10] The combination of digital and traditional approaches requires technical proficiency and a reconsideration of fundamental scholarly paradigms. [11]. In this context, the application of retrieval-based decision trees to academic writing presents a transformative perspective: a structured, transparent, and interdisciplinary literary studies. LIMITATIONS OF THE CONCEPT According to the conceptual design, the context of the AST, and the decision, it requires additional conceptual work to handle a context as a tree structure and extracting the relevant context for a specific rule from a tree structure, e.g. given as JSON for a web-based generation on a client side. The current paper does not provide a conceptual design solution to derive the specific context from structure data given as decision tree or AST. CONCLUSION This conceptual approach describes the fusion of text generation by grammar rules of different types and using them in a VCS as an identifiable digital object. The main step is that the VCS can provide transparency for text generation. Recursive application of rules towards a final generated text creates an AST with tree nodes that represent an author decision, validation, or quality assurance in contrast to automated steps of document generation with deterministic or probabilistic grammar rules. The DOI serves as a mechanism to search, find, and identify these grammar rules and ASTs uniquely and the decision tree incorporated the work of humans with the document. This is a major step toward transparent separation of automated and human work on a product. Quality assurance of authors and reviewers of the document might not be an automated part of the document evolution, but it changes the trust of the community in generated documents if the results have passed a human quality assessment. Reusability as part of FAIR data principle is supported for text generators with Uniform Resource Identifiers to fetch and apply the grammar rule for a specific generative task. For transparency, the DOIs for the grammar rules are accompanied by the author selection if the rules replace a non-terminal symbol of type RNT and ONT. Finally, by application of the proposed concept, a new grammar-driven document generation together with retrieval and GenAI. e.g., for scientific articles. Version control offers the possibility to share the generative steps with the version history. Beyond the final product, the version control offers transparency for “who did when what” in the evolution of the document. This is relevant for an educational system in which the contribution of students in collaborative learning should be separated from automated generation process of GenAI. REFERENCES [1] B. A. Nosek et al., “Promoting an open research culture”, Science, vol. 348, no. 6242, pp. 1422-1425, 2015. doi:10.1126/science.aab2374 [2] N. Nizamuddin et al., “Decentralized document version control using ethereum blockchain and IPFS” Comput Electr Eng, vol. 76, pp. 183-197, 2019. doi:10.1016/j.compeleceng.2019.03.014 [3] C. Clark, “ASTs for optimizing compilers”, ACM SIGPLAN Notices, vol. 36, no. 9, pp. 25-30, Sep. 2001. doi: 10.1145/609769.609773 [4] A. Gamba, “Real options valuation: A Monte Carlo approach” Faculty of Management, University of Calgary WP, 2002/3, 2003. doi:10.2139/ssrn.302613 [5] G. Satta and O. Stock, “Bidirectional context-free grammar parsing for natural language processing” Artificial Intelligence, vol 69, no. 1-2, pp. 123-164, 1994. doi: 10.1016/00043702(94)90080-9 [6] R. Chandrakar, “Digital object identifier system: an overview.”, The Electronic Library, vol. 24, no. 4, pp. 445-452, Jul. 2006. doi:10.1108/02640470610689151 [7] R. Simmons and Y. Yu, “The acquisition and use of contextdependent grammars for English”, Computational Linguistics, vol. 18, no. 4, pp. 391-418, 1992. doi:10.5555/176313.176314 [8] V. de Boer and L. Stork, “Hybrid Intelligence for Digital Humanities” HHAI 2024: Hybrid Human AI Systems for the Social Good, vol. 386, pp. 94-104, 2023. doi: https://doi.org/10.48550/arXiv.2406.15374 [9] U. Pawlicka-Deger, “The laboratory turn: exploring discourses, landscapes, and models of humanities labs.” Digital Humanities Quarterly, vol. 14 no. 3, p. 1, 2020. [10] Ibid., p. 63 [11]A. Lucke, “Methodologische Potenziale und Herausforderungen einer transdisziplinären Zusammenarbeit“ in: A. Lucke and H. Johannes (ed.): Literaturwissenschaft und Informatik. Transdisziplinäre Perspektiven, digitale Methoden und selbstlernende Algorithmen. Bielefeld: transcript, pp. 7-34, 2024. doi: 9783839470039-001 https://doi.org/10.5281/zenodo.17237756