Documentation template for the use of Generative AI in research

Documentation template for GenAI Documentation template for the use of Generative AI in research Metadata and decision-making criteria By Thijs Devriendt, PhD – Science Policy Officer Open Science & Generative AI Vrije Universiteit Brussel, Belgium EUTOPIA-MORE Train-the-Trainer material doi: 10.5281/zenodo.17243261 Documentation template for GenAI 1 Table of Contents Table of Contents ...................................................................................................................................................................... 1 1. Introduction ............................................................................................................................................................... 2 2. When to use the template? ..................................................................................................................................... 2 3. Sections of the document ......................................................................................................................................... 3 3.1 Explanation ................................................................................................................................................................ 3 3.2 Decision-making criteria ............................................................................................................................................ 3 3.3 Metadata .................................................................................................................................................................... 3 4. Examples of generative AI applications .................................................................................................................... 5 4.1 Uploading long format items in generative AI ......................................................................................................... 5 4.2 Assigning categories to research papers .................................................................................................................. 6 4.3 Conducting a systematic review ............................................................................................................................... 6 5. Conclusion .................................................................................................................................................................. 6 Documentation template for GenAI 2 1. Introduction Generative AI is being increasingly integrated into research processes. These include but are not limited to literature exploration, the generation of code, and the execution of specific analyses (e.g., sentiment analysis). Despite this integration, the metadata related to use of generative AI is not always being fully or correctly reported. Some research studies omit entirely essential details of what models were used to generate responses. A common misreported element is the intermixing of ChatGPT (the service offered by OpenAI that is built on LLM technology) and the underlying base models, such as GPT-4o. This leads to confusing contortions such as ChatGPT-4o. Other elements are also often missing, which impacts the credibility of the research and the degree to which results can be reproduced. At Vrije Universiteit Brussels (VUB), we developed a standard for reporting the use of generative AI in research processes (= the documentation template). The intention was to make VUB researchers aware of the type of information they need to report when using generative AI. The template was designed to be easy to use, and to be applicable to use of chatbot services in a research context. Furthermore, particular attention was paid to commonly misreported elements in scientific literature. 2. When to use the template? One ongoing debate and commonly asked question is when use of generative AI is substantial enough to be reported in a research publication. While we cannot provide definitive answers to this question that would be applicable to all possible scenarios across all research fields, we provide researchers with three criteria to guide their decision to report use of generative AI. All these criteria must apply for the use of the template to be highly recommended: - Use of generative AI must be for a research-specific purpose (e.g., generation of hypotheses) and not generic (e.g., making Powerpoint-slides, preparing a speech). - Use of generative AI must impact an intellectual step within research. This includes data analysis, data interpretation, design of an empirical study, building argumentation... The term "intellectual" should be broadly interpreted and understood within the context of the research domain. For instance, in research fields that do not traditionally use empirical research methods and are largely text-based (e.g., philosophy), reasoning will be the primary intellectual step. A simple example of a step that is not considered intellectual is the writing down of study results in the Result section of the paper. - Use of generative AI must be more substantial than merely providing minor assistance to researcher-led tasks. The use must put the brunt of the intellectual work of a specific research process on the model itself rather than on the researcher. Use within a structured framework (e.g., as intricate part of a study design, systematic literature review, or formal analysis pipeline) is considered more substantial than ad hoc, unstructured use (e.g., sporadic inquiries for convenience). We provide examples: Within scope (if intellectual step & substantial use) Outside of scope Systematic hypothesis generation Grammar and style checking (modifications are manually reviewed and not automatically inserted) Data analysis automation (e.g., sentiment & thematic analysis) Literature exploration (informal, not part of structured study) Experimental design setup Code debugging Generating arguments Assisting in literature review design Literature synthesis Citation formatting Data cleaning & transformation Creating non-research graphs Developing interview questions Generating summaries & abstracts (unstructured use, not part of method) Writing entire paragraphs, including citations Answering questions based on paper Documentation template for GenAI 3 3. Sections of the document 3.1 Explanation The document is split into two sections. The first section includes information (called “decision-making criteria”) that may help researchers discuss whether generative AI should be employed. These may be reported within the methodology section. The second section includes information (called “metadata”) that should be reported as part of the publication itself. 3.2 Decision-making criteria Information protection #1 Are you taking measures to be sure that any data are going to be processed responsibly and in accordance with relevant legislation? You may want to think about (a) contractual arrangements; (b) license conditions; and (c) data protection regulations applicable to your region. Context window #2 Are you sure that the processing of information you input going to take place within the context window*of models? Please be aware that, even within the context window, models do not always "factor in" all information to the same degree to make predictions ("lead bias"). *Context window refers to the maximum amount of tokens that the model can process to formulate an answer. Verification #3 What measures will you take to verify outputs? If you are not taking measures, does empirical evidence exist on accuracy or reliability of LLM use in similar settings? Explainability #4 If explainability* is relevant to your use of models, are you going to take steps to improve it? *Explainability refers to the ability to explain why a model provided a certain response. Accountability #5 Are you going to keep logs of all (or some) of your interactions with the models? Transparency #6 What parts of interactions, if any, are you going to report within your publication? Plagiarism #7 Are you going to take steps to ensure proper attribution of ideas and sources when engaging in ideation? If so, how? Research integrity #8 Could the use of LLMs potentially conflict with research integrity (e.g., when using LLMs for exploratory data analysis)? Drawbacks #9 Could the drawbacks of the models undermine the intended purpose of your research (e.g., introducing bias, randomness prevents reproducibility...)? If so, how? Harms #10 Could there potentially be disproportionate societal costs associated with using the models for these purposes (e.g., climate impact, group harm due to biases...)? Method #11 Could there be potential differences between outputs created by humans versus outputs created by models? If so, are these differences problematic? (e.g., themes emerging from thematic analysis) 3.3 Metadata Categories Question No. Question Definitions & Examples Expected format Documentation template for GenAI 4 Basic information #1 What service are you using (e.g., ChatGPT, Claude, Gemini, API, software to run models locally…)? Example: ChatGPT chatbot interface, API of OpenAI models, models locally run on my computer… Name of service #2 What model and model version are you accessing under this service? Check the website of the provider. If the version name is not findable, report the date of use. Example: GPT-4o-2024-05-13, GPT-41106-preview… Model version Technical characteristics #3 Are you using an unmodified model (i.e., base model) or are you working with a modified version (e.g., fine-tuned)? Unmodified version = The model as it was originally released (e.g., GPT-4, Llama 3 8b) Modified = The model as it was originally released plus modifications that were made to the parameters (e.g., through fine-tuning). Fine-tuned = Further adjusting model parameters based on specific datasets to tailor models better to one specific task Category #4 Are data processed locally or elsewhere? Local processing = The data does not leave your computer. The computing power comes from your RAM memory or a local source . Not local processing = The data leaves the computer. For instance, it might be processed on regional data servers when you enter prompts into chatbots. Afterward this, the data will also be stored for limited duration on other servers, often located outside the European Union. Category #5 What is the cut-off date of the model? Cut-off date = The most recent date that the training data has. For instance, if the training data contains no data more recent than 1st of April, this will be the cut-off date. Date #6 What is the context window of the model? Context window = The maximum amount of tokens that the model can process to predict the next token. Note: This is only important for data analysis and long text processing (e.g., legal texts, books…). Number Workflow #7 What type of data was inputted (e.g., audio, tabular data, prompt)? Input data = The data you enter as a user of the model. If text, this is called a "prompt". Input type #8 What type of data was outputted (e.g. text, audio, tabular data)? Output data = The data that is generated. This may be natural language, images, code, video… Output type Documentation template for GenAI 5 #9 Were specific parameters manually set when working with the model or not? Only answer for the final configuration. Parameters = Variables that you may give a certain value when working with the model (e.g., temperature, Top P...) Category #10 If ever, which parameters did you set manually? Only describe for the final configuration. What is the rationale if you used particular settings of parameters? Describe this in your methodology section. Parameters, Text #11 Did you reuse existing prompts or use certain prompting techniques (e.g., few-shot prompting, Chain-of-Thought)? If so, refer to these in your methodology section. Only describe the final configuration. Prompts = An input, often in natural language, that elicits a response by the model Prompting techniques = Techniques which are known through either formal research or empirical testing to elicit more desirable outputs (e.g., higher quality). Category, Reference #12 How was the model applied and integrated within your research workflow? Describe this in your methodology section. Only describe the final configuration. Method: Describe the specific procedure, technique, or approach used to apply the model. If a preexisting method was used, reference it within the methodology section. Depict the procedure in a flowchart. Examples: Generation followed by manual verification, retrievalaugmented generation... Category, Reference 4. Examples of generative AI applications 4.1 Uploading long format items in generative AI Imagine that you upload an 800-page book on the history of international law. You would like to know what specific historical case law you could reference related to specific cases that you are looking into. A book has about 300 words per page, which totals about 240.000 words for the entire book. This means the model has to process about 320.000 tokens (counting that in general the amount of tokens multiplied by 0.75 gives the amount of words). For each case you present, you get a list of historical events. Here is why reporting documentation of generative AI matters: • Models have a maximum amount of tokens that they can process before they start “forgetting” things (this maximum is called the context window) • It is known that there is already degradation of performance taking place even within the context window. If you do not report this information, other researchers cannot verify whether your use of the model made sense methodologically. If you opted for a model with very limited context window (e.g., ~200.000 tokens), the model will not be able to give answers that take the entire book into account. Documentation template for GenAI 6 4.2 Assigning categories to research papers Imagine you are doing a scientometric study. As part of the methodology, you are using generative AI to assign categories to information. (You don’t want to rely on pre-existing classification schemes that are known to contain many flaws). You are assigning classifiers to entire papers based on their contents. You decide to use one of OpenAI’s models via their API. You leave the hyperparameters in their default values as they are because you don’t understand them. Here is why reporting documentation of generative AI matters: • The randomness of responses is influenced by several hyperparameters, including the temperature variable. The standard setting might be a value between 0.8 and 1. At setting 1, the model will occasionally pick “unlikely” tokens for the prediction. At setting 0, the model will always pick the most likely token every time, leading to reproducible outputs. • The categories that you assigned will be subject to randomness! Doing the same run twice will result in different categories assigned. • You just used generative AI as one step in an entire workflow. You need to document and illustrate your entire workflow in a flowchart. If you do not report this information, other researchers cannot verify whether your use of the model made sense methodologically. Your assignment of categories will be irreproducible by other researchers if you just use default values of hyperparameters. Additionally, the methodological validity of your use of generative AI will depend on the workflow itself (i.e., does your use make sense). 4.3 Conducting a systematic review Imagine that you are conducting a systematic review. You decide to use two ways of identifying articles. First, you will design search strings that you run in academic databases. You read the abstracts and contents of the papers. Based on inclusion criteria, you then decide whether to include the articles. Second, you decide to also gather articles via Deep Research Modules offered by AI-tools. The Deep Research Modules autogenerate search strings and run them against different academic databases and search engines. The Modules read through the articles and decide what to include in the final report. You verify whether these articles meet your inclusion criteria. If so, you extract the citations and use these articles in your review. Here is why reporting documentation of generative AI matters: • Using a Deep Research Module is a completely different thing from just using a standard Large Language Model. It is a workflow that includes multiple steps, such as search string generation, running the search strings against academic databases, reading through the collected articles and drafting an entire report on the topic. This needs to be reported correctly. • Deep Research Modules cannot do comprehensive searches as there are limitations to how many search strings they run, how many databases they can access and how many papers they read. The number of run search strings, accessed databases and read articles must also ideally be reported. A further problem is that the output of a Module is not reproducible because the creation of search strings generated is subject to randomness. If you do not report this information, other researchers cannot determine whether your methodology is sound. They may also not understand the shortcomings of the Deep Research Module that you used. If you have not reflected on the limitations, you may wrongly portray the Deep Research Module as highly reliable and reproducible. 5. Conclusion This documentation template helps researchers report accurate metadata on the use of generative AI in research. This boosts transparency over the analytical choices made, and reproducibility of the research study. We encourage researchers to incorporate the template into their workflow (insofar no domain-specific standard already exists). Journals may also draw inspiration from the reporting standards described in this document.