ADDING RETRIEVAL AUGMENTED GENERATION TO THE MOSAIC
FRAMEWORK
F. Holz1, D. Scha 1, A. Nussbaume 1, S. Gü l1
1G az Uni e si y o Technology, G az, Aus ia
Abs ac
This pape p esen s a concep o adding Re ie al-
Augmen ed Gene a ion (RAG) ea u es o he MOSAIC
amewo k. MOSAIC enables web sea ch in segmen s o
he Open Web Index (OWI), in o de o es ablish a special-
pu pose sea ch engine. An ex ension, MOSAIC-RAG, has
been de eloped ha adop s a RAG app oach. I is designed
as a modula amewo k ha has in eg a ed a se o p ocess-
ing modules buil on gene a i e AI models, such as a module
o e- anking he sea ch esul , a module o summa ising
he ull ex s o he sea ch esul , o a module o summa is-
ing all sea ch esul s. These modules can be o de ed in an
a bi a y sequence, in o de o con igu e an o e all p ocess
o imp o e he sea ch esul . Such con igu a ions can be
adap ed o speci ic pu poses and sa ed o la e euse.
INTRODUCTION
Recen ly, La ge Language Models (LLM) ha e become
e y popula , because humans can in e ac wi h hem in na u-
al language when eques ing in o ma ion. They a e capable
o gene a ing ex s in a ious con ex s, such as answe ing
ques ions, p o iding ex ensi e in o ma ion, o summa is-
ing ex s. In con as o adi ional sea ch engines, hey do
no deli e o iginal web documen s, bu gene a e esponses
based on a as amoun o in o ma ion ha has been used o
ain hem. Though his ype o in o ma ion sea ching migh
be a ac i e o many people, he e a e also p oblems such
as he phenomenon o hallucina ions, ou da ed in o ma ion,
and missing in o ma ion sou ces.
Re ie al-Augmen ed Gene a ion (RAG) seeks o com-
bine LLMs wi h adi ional sea ch engines. Di e en ech-
niques ha e been p oposed explaining how sea ch engines
a e en iched wi h LLM unc ionali ies[1]. A simple ech-
nique consis s o he use o ex chunks e ie ed om a
sea ch engine o eeding and p omp ing an LLM. Mo e ad-
anced ea u es include he imp o emen o he sea ch que y,
as well as he e- anking o summa isa ion o he esul s wi h
he help o an LLM. Such an in eg a ion has se e al ad an-
ages and pa ially o e comes he a o emen ioned p oblems
o LLMs. A web index wi h cu en da a can injec up- o-
da e in o ma ion in o LLMs, and also p o ide o iginal web
documen s on demand. Thus, hallucina ion is mi iga ed by
p o iding ac ual knowledge in combina ion wi h gene a ed
ex s.
This pape p esen s a RAG app oach ha is based on he
Open Web Index (OWI). A special-pu pose sea ch engine
c ea ed wi h da a om he OWI is in eg a ed wi h a ame-
wo k ha p ocesses he e ie ed da a using di e en kinds
o AI models. The nex sec ion desc ibes he o e all concep
o his amewo k. This sec ion is ollowed by a mo e de-
ailed desc ip ion o he modules used o imp o e he sea ch
p ocess. Finally, an applica ion is p esen ed ha showcases
how a RAG sys em can be se up wi h ou app oach.
CONCEPT AND MODULAR FRAMEWORK
The o e all aim o MOSAIC-RAG
1
is o en ich sea ch
engines using he Open Web Index (OWI) wi h ea u es p o-
ided by La ge Language Models (LLMs). The en ichmen
is mainly pe o med by u he p ocessing he sea ch esul ,
such as p o iding summa isa ions, e- ankings, o con e sa-
ional sea ch. The esul is deli e ed o he end-use ia a
buil -in web in e ace o an API ha can be used by ex e nal
applica ions. The o e all concep is depic ed in Figu e 1 and
explained in mo e de ail in his sec ion.
OWI Index
Slice
MOSAIC
MOSAIC-RAG
Web In e ace Applica ion
Download
Impo
Sea ch and Re ie e
Module
Module
Module
API
API
Cha bo
Ch omaDB API
Sea ch and Re ie e
Figu e 1: Concep ual design o MOSAIC-RAG.
The i s s ep o c ea ing a MOSAIC-RAG applica ion
consis s in he c ea ion o an index slice ha se es as he
unde lying da abase o he sea ch engine. Index slices a e
small- o medium-sized indices con aining web documen s
ela ed o a ce ain opic o a pa icula pu pose. Mo e p e-
cisely, hey con ain an in e ed index ep esen ed in CIFF
o ma
2
and me ada a o each web documen ep esen ed in
Pa que o ma
3
. The me ada a include he i le, ull ex ,
1h ps://opencode.i 4i.eu/openwebsea cheu-public/mosaic- ag
2h ps://gi hub.com/osi c/ci
3h ps://pa que .apache.o g/
h ps://doi.o g/10.5281/zenodo.17209496
URL, language, geo-coo dina es, opic, and o he in o ma-
ion o he web documen . Such slices can be downloaded
om he OWI using que ies ha speci y he domain and
con en o he index slice [2]. Fo example, index slices can
con ain web documen s ela ed o a ce ain opic, such as
science news, a speci ic language, such as Finnish, o a e
pa o a ce ain op-le el domain.
The second s ep consis s o he p epa a ion o he sea ch
engine ha deli e s sea ch esul s using he index slice.
The e a e wo op ions ha a e compa ible wi h MOSAIC-
RAG. Fi s , MOSAIC is a amewo k and gene ic sea ch
applica ion ha makes index slices sea chable [3]. Second,
Ch oma
4
is a ec o da abase ha allows o sea ch documen s
using ec o embeddings. Bo h sea ch engines p o ide an
API ha allows he sea ch o web documen s and deli e s
lis s o web documen s including hei me ada a and ull
ex .
Inges ing da a slices wo ks di e en o each o hese
sea ch engines. MOSAIC is designed o easily in eg a e
index slices by jus copying hem in o a esou ce di ec o y.
Each index slice is ep esen ed as an index in MOSAIC and
can be sea ched indi idually. Impo ing index slices in o
Ch oma needs some p e-p ocessing, as i equi es ec o em-
beddings o each web documen , ha can be c ea ed wi h
sui able models, such as he Jina Embeddings 2 Model[4].
In Ch oma, each web documen is ep esen ed as a iple
consis ing o an ID, he ec o embedding, and he me ada a
om he Pa que ile. La e he sea ch que y is also ep e-
sen ed as ec o embedding using he same model, which
allows Ch oma o e ie e ma ching documen s. In he u-
u e, he ec o embedding will also be pa o he OWI,
which simpli ies he impo ing p ocedu e.
The co e o MOSAIC-RAG is a modula pipeline ha
en iches he sea ch esul e ie ed om he sea ch engine.
I includes a sui e o p ocessing modules ha can pe o m
a ious ans o ma ions o he sea ch esul . The cu en ly
a ailable modules a e desc ibed in he nex sec ion. Fo
example, he ull ex o each esul i em (web documen )
can be summa ised, he lis o esul i ems can be e- anked,
o an o e all summa y can be c ea ed ou o he sea ch esul .
The se o cu en ly a ailable modules is ex ensible and new
modules can be added by implemen ing a base class ha
manges a da a ame consis ing o he sea ch esul . The ows
o he da a ame consis o he indi idual web documen s
and he columns comp ise hei me ada a. Each module
can manipula e he da a ame in any way. Typically, a ow
wi h newly calcula ed in o ma ion is added, o example wi h
summa isa ion o he ull ex o by compu ing a new anking
(see Fig 2). Each module ha uses an LLM o p ocess he
da a can ei he chose o un he LLM locally, i.e., di ec ly
om he Py hon code, o use a emo e in e ence poin . The
emo e in e ence poin can be con igu ed globally o he
whole MOSAIC-RAG ins ance. Fo his pu pose, ei he a
Li eLLM5o OpenAI compa ible endpoin is equi ed.
4h ps://www. ych oma.com/
5h ps://www.li ellm.ai
Module 1:
Da a Sou ce
231 Ti le 1 1 Tex 1
424 Ti le 2 2 Tex 2
352 Ti le 3 3 Tex 3
453 Ti le 4 4 Tex 3
Module 2:
Summa isa ion
Module 3:
Re- anke
ID Ti le Rank Full ex
231 Ti le 1 1 Tex 1 Sum 1
424 Ti le 2 2 Tex 2 Sum 2
352 Ti le 3 3 Tex 3 Sum 3
453 Ti le 4 4 Tex 3 Sum 4
ID Ti le Rank Full ex Sum.
231 Ti le 1 1 Tex 1 Sum 1 3
424 Ti le 2 2 Tex 2 Sum 2 2
352 Ti le 3 3 Tex 3 Sum 3 4
453 Ti le 4 4 Tex 3 Sum 4 1
ID Ti le Rank Full ex Sum. Re- ank
Da a om Sea ch Engine
Figu e 2: Modula Pipeline wi h da a ames
The modules can be sequenced in any o de depending
on he pu pose how he esul s should be p ocessed. Thus
a use can c ea e a ce ain sequence o p ocessing mod-
ules, in o de o de ine he beha iou o MOSAIC-RAG (see
also nex sec ion). Such a con igu a ion is epheme al, only
las ing o he du a ion o he b owse session. Howe e ,
MOSAIC-RAG p o ides wo ways o loading and sa ing he
ull con igu a ion, i.e., cus om colo heme, cus om i les,
and he pipeline con igu a ion. Fi s , con igu a ion can be
downloaded in JSON o ma . Use s can upload his JSON
ile o he on end o es o e a sa ed con igu a ion. Second,
his con igu a ion can be s o ed on he se e unde a unique
ID o be e ie ed using a cus om URL. Each module has
also a ew pa ame e s o s ee hei beha iou , such as he
selec ion which LLM should be used o he summa isa ion.
In addi ion o he modula pipeline, MOSAIC-RAG also
suppo s a con e sional sea ch unc ionali y. In a cha box,
he use can ask an LLM ques ions abou he cu en se o
sea ch esul s. The con e sa ional sea ch agen is ins uc ed
o only gi e answe s based on he ac ual sea ch esul s, no
based on i s own wo ld knowledge.
In o de o in e ac wi h MOSAIC-RAG, a web in e ace
is p o ided ha enables bo h he sea ch and he con igu a ion
o he modula pipeline. The web in e ace uses MOSAIC-
RAGS ully documen ed API. This allows o he applica ions
o use he ull unc ionali ies o he se ice.
RAG MODULES
This sec ion desc ibes he 18 cu en ly implemen ed
pipeline modules. These modules a e o ganised in i e
g oups, depending on hei unc ionali y: da a sou ce, sum-
h ps://doi.o g/10.5281/zenodo.17209496
ma isa ion, e- anking, p e-p ocessing, and me ada a analy-
sis.
The da a sou ce modules deal wi h e ie ing sea ch e-
sul s om ex e nal sea ch engines when a use s a s a que y.
Cu en ly wo sea ch engines a e suppo ed, MOSAIC and
Ch oma. De ails can be con igu ed, such as he index used by
MOSAIC o he embedding model used by Ch oma. Fu he -
mo e, he numbe o sea ch esul s can be limi ed. The da a
sou ce module con e s he da a ga he ed in hose sea ch en-
gines in o a da a ame. This da a ame ge s passed h ough
he con igu ed pipeline modules sequen ially. A e he inal
module, he da a ame ge s sen o he use acco ding o
he API speci ica ion. Mul iple da a sou ce modules can
also be added o he same pipeline, allowing o he agg ega-
ion o da a om di e en sou ces (e.g. mul iple MOSAIC
ins ances).
The p e-p ocessing modules mainly deal wi h ex clean-
ing and o ganising o he esul se . The e a e modules o
emo e HTML ags and s op wo ds, o o pe o m s emming
ope a ion on he ex . These unc ions migh no be needed
in e e y case, as he sea ch esul s may al eady be cleaned
by he o iginal sea ch engine. As compu ing powe is o en
limi ed, he Reduc ion Module is impo an because i e-
duces he size o he in e nal da a ame based on a condi ion
(usually he anking). When p ocessing la ge esul se s in
a pipeline con aining a leas one LLM module, such as a
LLM Summa ize o an Embedde , he execu ion ime o
he o al pipeline can be g ea ly educed by dec easing he
numbe o p ocessed documen s. The e o e, a e pe o m-
ing some e- anking, i migh be su icien o keep he bes
ew documen s and disca d he es .
The e- anking modules change he anking o he esul
se . Cu en ly, ou e- anking modules a e implemen ed
by de aul in MOSIAC-RAG. The embedding e- anke pe -
o ms a new anking based on he simila i y o embedding
ec o s. Those embedding ec o s will be c ea ed using
he Sen enceT ans o me Py hon lib a y i hey do no al-
eady exis in he da a ame. The TF-IDF ( e m equency
- in e se documen equency) [5] e- anke is among he
simples and as es app oaches, allowing documen s o be
e- anked based on hei TF-IDF ec o ep esen a ions and
a chosen simila i y me ic. Cu en ly, MOSAIC-RAG sup-
po s he simila i y me ics Euclidean dis ance, Manha an
dis ance, cosine simila i y, and BM25. The la e di e s
sligh ly om he o he s, as i does no ely on he ull TF-IDF
ec o ep esen a ion. A BM25 anking algo i hm is also
used by MOSAIC o i s sea ch. The wo o he e- anking
modules a e based on he p inciple o la ge-language-model-
e- anking [6]. He e la ge language models (LLMs) a e
used o iden i y which documen i s he gi en que y bes
in a se o gi en candida e documen s. The G oup-S yle
LLM e- anke module anks documen s by compa ing a
se o candida e documen s agains a gi en que y and allow-
ing he LLM o de e mine which documen bes ma ches
he que y. Fo each compa ison, a sco e is assigned o he
documen ha i s bes . This p ocess is epea ed ac oss all
possible documen combina ions, gi en bo h he size o he
candida e se and he o al numbe o documen s [7, 8]. The
language model and he size o he candida e se can be con-
igu ed. The inal p e-implemen ed e- anking module is he
Tou namen -S yle LLM Re- anke . Like he G oup-S yle
a ian , i elies on an LLM o e- anking, bu i educes
he numbe o equi ed documen compa isons, he mos
ime-consuming s ep, by le e aging an exis ing anking and
e ining i locally. The p ocess ollows he s uc u e o a
ou namen ee, whe e he winning documen ad ances
while he losing one is elimina ed. This app oach equi es
signi ican ly ewe LLM compa isons, imp o ing e iciency.
Howe e , i unc ions mo e as a anking enhancemen han a
ull e- anking. As i depends hea ily on he ini ial anking
used as he seed, i s e ec i eness is g ea es o iden i ying
he op- anked documen s ele an o a que y, while anking
quali y ends o diminish u he down he lis . I is impo -
an o no e ha all p e-implemen ed e- anking modules
ope a e solely on he documen s e ie ed in he ini ial s age
and do no pe o m any addi ional e ie al hemsel es [9].
The e a e wo ypes o summa isa ion modules. The i s
one summa ises he ull ex o each web documen in he
esul se , while he second one gene a es one summa y o
all he documen s in he esul se . In bo h cases an LLM
wi h a ge ed p omp s is employed o hese asks.
Finally he e a e h ee me ada a analysis modules. The
i s one is a simple wo d coun e ha coun s he numbe o
wo ds in a web documen . The Sen imen Analyse calcu-
la es a sen imen sco e o each web documen . Based on six
ou pu sco es o each sen imen he highes sco e is aken
and s o ed in he da a ame. The ele ance ma king mod-
ule ma ks pa s o he ull ex ha a e mos ele an . Bo h
he sen imen analysing module and he ele ance ma king
module use an LLM o hei ask.
APPLICATION CASE
Fo demons a ing how a RAG sys em can be se up and
con igu ed wi h MOSAIC-RAG, an applica ion has been
c ea ed ha enables sea ch in he domain o a s. This a s
sea ch engine is depic ed in Fig. 3.
Fi s , an index slice has been c ea ed ha only includes
web documen s ela ed o a s. This was achie ed by selec -
ing web documen s in he Open Web Index ha a e agged
wi h he Cu lie label A s. A e in eg a ing his index slice
in MOSAIC, he se ice is s a ed.
Second, MOSAIC-RAG is se up o ac as an a s sea ch
engine. Hence, a MOSAIC-RAG da a sou ce module is con-
igu ed o use he da a om he a s index o he p e iously
s a ed MOSAIC se ice. Then an Embedding Re- anking
Module is added o imp o e he sea ch esul . Finally, a
summa isa ion module is added ha p o ides an o e all
summa y o he sea ch esul on op.
Finally, he appea ance o he web in e ace is con igu ed.
The i le is changed o A s Sea ch and he colou scheme
is se o da k-o ange. The whole con igu a ion is sa ed and
an ID is au oma ically c ea ed, which allows o sha e his
con igu a ion ia a single URL.
h ps://doi.o g/10.5281/zenodo.17209496
Figu e 3: The Web In e ace o MOSAIC-RAG con igu ed as a s sea ch engine. The summa isa ion and sea ch esul s a e
on he le side and he p ocessing pipeline on he igh side.
CONCLUSION AND OUTLOOK
The main con ibu ion o his pape consis s o a Re ie al-
Augmen ed Gene a ion app oach in he con ex o he Open
Web Index. A special pu pose and e ical sea ch engine
c ea ed wi h da a om he OWI is in eg a ed wi h a ame-
wo k ha p ocesses he e ie ed da a using di e en kinds
o LLMs.
Fu u e wo k will include use s udies ha in es iga e he
use ulness and accep ance o his app oach. Fu he mo e,
di e en con igu a ions will be c ea ed and es ed, in o de
o be e unde s and hei bene i s o he use . In pa icula ,
he bene i o end-use s o summa isa ions and e- ankings
will be in es iga ed.
ACKNOWLEDGEMENTS
This wo k has ecei ed unding om he Eu opean
Union’s Ho izon Eu ope esea ch and inno a ion p og amme
unde g an ag eemen No 101070014 (OpenWebSea ch.EU,
h ps://doi.o g/10.3030/101070014).
REFERENCES
[1]
Y. Gao e al.,Re ie al-augmen ed gene a ion o la ge lan-
guage models: A su ey, 2024.
h ps://a xi .o g/abs/
2312.10997
[2]
M. G ani ze e al., “Impac and de elopmen o an open web
index o open web sea ch,” Jou nal o he Associa ion o
In o ma ion Science and Technology, 2023.
10.1002/asi.
24818
[3]
S. Gü l, MOSAIC: Empowe ing a Modula F amewo k
o Con igu able and Tailo ed Web Sea ch based on an
Open Web Index, 2024.
h ps://diglib. ug az.a /
diplomaTheses
[4]
M. Gün he e al.,Jina embeddings 2: 8192- oken gene al-
pu pose ex embeddings o long documen s, 2023.
[5]
K. Spa ck Jones, “A s a is ical in e p e a ion o e m speci ici y
and i s applica ion in e ie al,” Jou nal o documen a ion,
ol. 28, no. 1, pp. 11–21, 1972.
[6]
Y. Zhu e al., “La ge language models o in o ma ion e ie al:
A su ey,” ACM T ansac ions on In o ma ion Sys ems, 2025.
10.1145/3748304
[7]
W. Sun e al.,Is cha gp good a sea ch? in es iga ing la ge lan-
guage models as e- anking agen s, 2024.
h ps://a xi .
o g/abs/2304.09542
[8]
J. Sun, X. Zhong, S. Zhou, and J. Han, “Dynamic ag: Le e ag-
ing ou pu s o la ge language model as eedback o dynamic
e anking in e ie al-augmen ed gene a ion,” a Xi p ep in
a Xi :2505.07233, 2025.
[9]
M. Ra hee, S. MacA aney, and A. Anand, “Guiding e ie al
using llm-based lis wise anke s,” in Eu opean Con e ence on
In o ma ion Re ie al, Sp inge , 2025, pp. 230–246.
h ps://doi.o g/10.5281/zenodo.17209496
