Retrieval Augmented Generation (RAG) in Wikibase4Research

Bailly, Kolja; Rossenova, Lozana; Blümel, Ina

doi:10.5281/zenodo.17939506

Retrieval Augmented Generation (RAG) in Wikibase4Research

Bailly, Kolja; Rossenova, Lozana; Blümel, Ina

Publisher: Zenodo
DOI: 10.5281/zenodo.17939506
Repository: zenodo.org
Language: en

Abstract

Poster presented at the 5th Culture Community Plenary organised by NFDI4Culture. The poster presents the experimental prototype of a RAG LLM extension for the NFDI4Culture Service Wikibase4Research developed at the Open Science Lab (OSL) at TIB Hannover. OSL develops open source solutions for the management of research data with Wikibase, an extension of the MediaWiki software suite. The poster shows the integration of AI-based approaches within MediaWiki, utilizing Retrieval-Augmented Generation (RAG), a methodology that allows Large Language Models (LLMs) to interact with custom data sources. Computational resources can be used from remote services such as Huggingface API or GWDG SAIA or locally, preserving user privacy by keeping all data local. The results provide context-aware responses to user queries in natural language or support the user in the creation of SPARQL queries. OSL has updated the service to index data saved in several structured formats including MediaWiki pages and Wikibase statements. A demo instance of the service has been applied to a Wiki instance containing data about historic manor houses in the Baltic Sea Region, a joint project between OSL and the University of Greifswald. NFDI4Culture is funded by the Deutsche Forschungsgemeinschaft (DFG) under grant no. 441958017.

Full text

Retrieval Augmented Generation (RAG) in Wikibase4Research Test Case Study What is RAG?Wikibase4Research Kolja Bailly k[email protected] Lozana Rossenova lozana.rossenov[email protected] Ina Blümel [email protected] • Configurable installation tool for Mediawiki, Wikibase, Semantic Mediawiki (SMW) and Semantic Wikibase; • Docker based All-in-one setup; • Mediawiki extension management; • Data import / export and system updates; • Open source, provided and maintained by the Open Science Lab @ TIB Hannover. • Method for context-based question answering; • Context data is indexed for later retrieval; • Large Language Model (LLM) generates answer from relevant context data; • RAG lowers risk of LLM hallucination; • Provides data that wasn’t included in LLM training; • Supports domain specific knowledge. • Performance of model inference (hardware requirements / response time); • Configuration of RAG strategies; • Hybrid search (structured + unstructured data); • Provide standalone usage without Wikibase4Research; • BlazegraphStore implementation for LlamaIndex (avoid data cloning); • AutoRag (automatic config, based on example questions). User Question Answer Retriever Context Herrenhauszentrum des Ostseeraums (Manor Houses in the Baltic Sea Region), Wikibase4Research instance: 7B LLamav2 Model (german). Chunks built via html <td> tags. Vectorindex in Faiss DB Challenges & Outlook Contact Acknowledgements Test Question: RAG Answer: Correctness: Who owns the manorhouse Christinehof? Ownership of the Christinehof estate remains with the Piper family. In which year was the main building of Christinehof manor house build? 1737–1740 Which manor house was built by the widow of a man who played a significant role in the Swedish state apparatus? Hedvig Mörner built the main building of Stola Manor. * * The third answer was not entirely incorrect, as through the marriage, Hedwig Mörner's husband became the brother-in-law of Carl Gustav Mörner, who served as a governor in western Sweden. However, the RAG system failed to retrieve the more significant information about Christinehof Manor, where the husband himself played an important role in the Swedish state. Current State Prototype NFDI4Culture is funded by the Deutsche Forschungsgemeinschaft (DFG) under grant no. 441958017. Indexer Simple property graph store (in memory) Chunks GET requesthmtl / css tags MW API Vector space Wiki pages Wikibase / SMW Index updates triggered via API call Gitlab repository access