NFDI4Earth Chatbot: From Living Handbook to Interactive Dialogue

Abstract

1. Introduction This poster was presented during the 2025 joint plenary of NFDI4Biodiversity and NFDI4Earth in Bremen. It is intended to provide the community with an initial technical overview of the NFDI4Earth chatbot and its architecture. Furthermore, the poster serves as a communicative basis for discussions and feedback, helping to foster exchange and collaboration around interactive knowledge access within NFDI4Earth. 2. Motivation The NFDI4Earth Chatbot is an innovative solution for interactive knowledge transfer in the geosciences, developed as part of the National Research Data Infrastructure for Earth System Data (NFDI4Earth). Its primary goal is to simplify access to curated, domain-specific knowledge and lower barriers to specialized information by providing a natural-language interface. This enables direct interaction with content from the NFDI4Earth Living Handbook and supports researchers, community members, and infrastructure teams in their daily work. Key motivations for developing the chatbot include: Lowering Access Barriers to Specialized Knowledge: Providing intuitive, natural-language access to complex scientific materials, making expert knowledge more broadly available. Architectural Flexibility: Allowing easy integration of additional knowledge sources, such as publications, ontologies, or database schemas, without requiring structural changes to the system. This is currently realized through a modular architecture. Domain-Specific Responses: Ensuring semantic relevance and accuracy by leveraging curated content and advanced vector search techniques. Transparency: Communicating the provenance of the answers to users, to enable their verification. Overall, the chatbot facilitates knowledge transfer within NFDI4Earth, helping both experienced researchers and new community members navigate and understand the Living Handbook and its related resources. 3. System Architecture (RAG) & Implementation The technical foundation of the NFDI4Earth Chatbot is a Retrieval-Augmented Generation (RAG) architecture. This combines the capabilities of large language models, provided through the ScaDS.AI LLM API, with semantic search in curated knowledge sources. By integrating ChromaDB as a vector database, documents from GitLab repositories are automatically indexed and made semantically searchable. This ensures that the system generates precise and contextually relevant responses while reducing the processing load for the language model. User queries submitted via the OneStop4All frontend are processed by retrieving the most relevant Living Handbook articles from ChromaDB. These articles, together with the original user query, are used to generate context-aware responses through the ScaDS.AI LLM API. This pipeline ensures that answers are grounded in curated sources and remain contextually meaningful. The implementation features a modular Python library with clear extension points, a lightweight Flask-based RESTful API for seamless integration with the OneStop4All frontend, and an administrative interface for inspecting relevance scores and usage statistics. Automated indexing of Living Handbook documents from GitLab repositories ensures that the knowledge base remains up to date and extensible. Future content—such as publications, ontologies, or database schemas—can be incorporated seamlessly, allowing the chatbot to evolve alongside the needs of the community. 4. Use Cases The NFDI4Earth Chatbot provides a simple yet illustrative use case for making the contents of the Living Handbook interactively accessible. It demonstrates how curated scientific knowledge can be explored and utilized in a more dynamic and user-friendly way, serving as a blueprint for further interactive applications within the NFDI4Earth community. The following aspects highlight the practical benefits and integration of the chatbot within NFDI4Earth: Onboarding and Community Integration: Prospective and current members of NFDI4Earth can use the chatbot as a first point of contact for NFDI4Earth-related questions. It helps them navigate the Living Handbook, understand project structures, and find resources for their research. User Interaction Insights: The chatbot collects usage statistics and query patterns, which are analyzed to identify frequently asked questions and knowledge gaps. This information supports various teams in NDFI4Earth in prioritizing content updates and expanding the Living Handbook according to community needs. Integration with the OneStop4All: Through the seamless integration with the OneStop4All frontend, the chatbot enables interactive dialogue and immediate access to expert knowledge, fostering collaboration and knowledge sharing within the NFDI4Earth community. 5. Acknowledgements This work was funded by the German Research Foundation (DFG), project number 460036893.