In o ma ion Fusion 99 (2023) 101882
A ailable online 12 June 2023
1566-2535/© 2023 The Au ho (s). Published by Else ie B.V. This is an open access a icle unde he CC BY license (h p://c ea i ecommons.o g/licenses/by/4.0/).
Con en s lis s a ailable a ScienceDi ec
In o ma ion Fusion
jou nal homepage: www.else ie .com/loca e/in us
Full leng h a icle
Explo ing he balance be ween in e p e abili y and pe o mance wi h
ca e ully designed cons ainable Neu al Addi i e Models
E o e Ma io ia,∗, José Ma ía Alonso Mo ala, Albe Ga b
aCen o Singula de In es igacion en Tecnoloxias In elixen es (CiTIUS), Uni e sidade de San iago de Compos ela, San iago de Compos ela, Spain
bU ech Uni e si y, U ech , The Ne he lands
ARTICLE INFO
Keywo ds:
Gene alised addi i e models
Explainable A i icial In elligence
In e p e able modelling
Neu al addi i e models
In e p e abili y
Explainabili y
ABSTRACT
The in e p e abili y o an in elligen model au oma ically de i ed om da a is a p ope y ha can be ac ed
upon wi h a se o s uc u al cons ain s ha such a model should adhe e o. O en hese a e in con as wi h
he ask objec i e and i is no s aigh o wa d how o explo e he balance be ween model in e p e abili y
and pe o mance. In o de o allow an in e es ed use o join ly op imise pe o mance and in e p e abili y,
we p opose a new o mula ion o Neu al Addi i e Models (NAM) which can be subjec o a numbe o
cons ain s. Acco dingly, ou app oach p oduces a new model ha is called Cons ainable NAM (o jus
CNAM in sho ) and i allows he speci ica ion o di e en egula isa ion e ms. CNAM is di e en iable
and is buil in such a way ha i can be ini ialised as a solu ion o an e icien ee-based GAM sol e
(e.g., Explainable Boos ing Machines). F om his local op imum he model can hen explo e solu ions wi h
di e en in e p e abili y-pe o mance adeo s acco ding o di e en de ini ions o bo h in e p e abili y and
pe o mance. We empi ically benchma k he model on 56 da ase s agains 12 models and obse e ha on
a e age he p oposed CNAM model anks on he Pa e o on o op imal solu ions, i.e., models gene a ed by
CNAM exhibi a good balance be ween in e p e abili y and pe o mance. Mo eo e , we p o ide wo illus a i e
examples which a e aimed o show s ep by s ep how CNAM wo ks well o sol ing classi ica ion asks, bu
also how i can yield insigh s when conside ing eg ession asks.
1. In oduc ion
Today’s socie y is accumula ing mo e and mo e da a whose use
can op imise exis ing p ocesses and b ing weal h and knowledge. This
mo i a es a need o ools ha can p ocess hese da a acco ding o he
speci ic asks a hand.
Machine Lea ning (ML) and A i icial In elligence (AI) ecen ly
gained lo s o success hanks o g ea e pe o mance wi h espec o
adi ional app oaches on many benchma k p oblems [1,2]. The ML
and AI app oach consis s in speci ying he end goal and cons ain s o
be sa is ied and he p og amme au oma ically adjus i sel o sa is y
hose. Un o una ely hese models, while e ec i e, a e ypically no
unde s andable by humans. This is because hei main job is only o
maximise p edic i e gene alisa ion, wi hou aking in o accoun he
in elligibili y o he model i sel .
The lack o in e p e abili y o some AI-based sys ems is an issue
o a numbe o applica ions o which quali y assu ance, us , legal
liabili y and adhe ence o e hical p inciples is a mus . Mo i a ed by
his, a lo o wo k has been done in he ield o eXplainable A i icial
In elligence (XAI) [3,4], a collec i e e o o explain he beha iou
∗Co esponding au ho .
E-mail add ess: [email p o ec ed] (E. Ma io i).
and p edic ions o ML models o allow debugging, o e sigh , audi ing,
knowledge disco e y, and sa e democ a isa ion o hese new powe ul
in o ma ion-p ocessing echnologies in many domains [5,6].
In his pape we de ine explainabili y as he abili y o a sys em o
gi e an explana ion o he use , ha is a epo o (pa o ) he causal
easoning ha lead o a pa icula ou come. We e e o in e p e abili y
ins ead o he less demanding p ope y o a sys em o being inspec able
in i s pa s in a meaning ul way. In his sense, in e p e abili y is a
p e equisi e o explainabili y.
XAI has e ol ed in di e en di ec ions based on he equi emen s
s a ed by he s akeholde s o he explainable sys ems. Fo example,
in o de o build ools ha a e as widely applicable as possible some
esea che s ha e p io i ised he de elopmen o ‘‘black-box explain-
e s’’ [7]. In his con ex , no assump ion is made on he model and we
a e only le wi h he inpu –ou pu ela ionship. In ha case we ha e
o ely on pos -hoc app oaches. These ypically in ol e app oxima ing
he o iginal model in an app op ia e neighbou hood o da a wi h an
in e p e able model (a so-called whi e-box) and hen inspec ing ha
o p o ide explana ions, as sugges ed by [8,9] ( his app oach is also
h ps://doi.o g/10.1016/j.in us.2023.101882
Recei ed 17 Oc obe 2022; Recei ed in e ised o m 2 May 2023; Accep ed 6 June 2023
In o ma ion Fusion 99 (2023) 101882
2
E. Ma io i e al.
known as su oga ing). The su oga ion pa adigm is a e y powe ul
one as i allows in p inciple o explain any black-box model, e en deep
neu al ne wo ks, as long as he su oga ing models a e powe ul enough
o ac like hem. In i ue o his, i we wan o enhance he explana ion
o black boxes, we i s need good in e p e able models by design so
ha we can la e use hem o su oga ing.
To pu i in o he e ms:
•we a e in e es ed in explaining black box models as hese a e
widely used;
•in e p e abili y is a p e equisi e o explainabili y, by he a gu-
men s a ed abo e;
•we should aim o models which a e in e p e able because hese
make ideal su oga e models o explaining o he models.
This opens up he ques ion o wha i eally means o a model o
be in e p e able. While wi h an impe a i e-s yle app oach (whe e he
p og amme is ha d-coded by a human) he in e p e abili y is kep
in each sub-componen , wi h he ML-s yle app oach (as i is usually
employed) he only solu ions explo ed a e hose ha sol e he ask
ega dless o he inal o m o he model. A way o le e aging he
powe o he ML app oach is o guide he sea ch o models no only
based on hei pe o mance bu also on some o he a ge s ha e lec
hei in e p e abili y. In his sense we can imagine ha an app op ia e
egula isa ion e m in o he a ge loss could cons ain he op imisa ion
o bo h dimensions (i.e., in e p e abili y and pe o mance).
Jus as he app op ia e measu e o p edic i e pe o mance a ies
om ask o ask, he app op ia e cons ain s o in e ep e abili y a e
also applica ion-dependen (usually penalise complexi y and a ou
spa se ep esen a ions) and should i e a i ely be e ined wi h domain
expe s [10]. Gene alised Addi i e Models (GAM) and Neu al Ad-
di i e Models (NAM) a e well-s udied classes o models ha ha e
powe ul p edic i e pe o mance while e aining an in e p e able s uc-
u e. This wo k add esses a new o mula ion o i ing GAM, named
Cons ainable Neu al Addi i e Model (CNAM), such ha app op ia e
in e p e able cons ain s can be en o ced o join ly maximise pe o -
mance and in e p e abili y. In o de o benchma k his model agains
o he app oaches we use a no el in e p e abili y me ic ha is called
SHAP-Leng h [11], which exploi s he well-known SHapley Addi i e
exPlana ions (SHAP) i s in oduced by [9].
The main con ibu ions in his wo k a e as ollows:
•a no el o mula ion o NAM, CNAM, such ha in e p e ablil y
cons ain s can be en o ced and join ly op imised alongside ask-
ela ed pe o mance me ics;
•an ex ensi e benchma k o CNAM on 56 bina y classi ica ion
da ase s agains 13 di e en models;
•an illus a i e use-case o CNAM on a classi ica ion da ase ;
•an illus a i e use-case o CNAM on a eg ession da ase .
The es o he manusc ip is o ganised as ollows. In Sec ion 2, we
discuss ela ed wo k in he ield. In Sec ion 3, we desc ibe he s uc u e
and p ope ies o CNAM. In Sec ion 4, we expe imen i s wi h a bench-
ma k s udy o explici ly explo ing CNAM in e p e abili y-pe o mance
adeo on bina y classi ica ion asks and hen go deepe wi h wo
illus a i e use cases: an as ophysics classi ica ion ask and a socio-
economic eg ession ask. Finally in Sec ion 6, we d aw conclusions
and delinea e u u e wo k.
2. Rela ed wo k
The his o y o lea ning om da a can be aced back o ea ly
wo k in he 19 h cen u y whe e, mo i a ed by he desi e o p edic ing
as onomical da a and minimising econs uc ion e o s, heo e ical
ounda ions and closed- o m solu ions o Linea Models (LM) we e de-
eloped, which minimised he Mean Squa ed E o (MSE) as p oposed
by [12,13]. The a ge 𝑦is modelled as 𝑦 om a se o ea u es 𝑥𝑖and
he ask is o ind a se o eal coe icien s 𝛽𝑖such ha
𝑦 =∑
𝑖
𝛽𝑖𝑥𝑖
and he MSE= 1∕𝑁∑𝑁
𝑗(𝑦𝑗−𝑦𝑗)2, is minimised.
Much la e , [14] uni ied some sca e ed iews o modelling a ce ain
a ge wi h a LM and in oduced Gene alised Linea Models (GLM),
whe e he a ge 𝑦is ans o med wi h he so-called link unc ion 𝑔(⋅)
wi h s a ic coe icien s 𝛼𝑖such ha
𝑔(𝑦) = ∑
𝑖
𝛼𝑖𝑥𝑖
This new modelling allows (among o he hings) o ha e linea
classi ie s by choosing he logis ic sigmoid as he link unc ion, leading
o wha is known as Logis ic Reg ession (LR).
A u he gene alisa ion o LM was de eloped a Bell Labs by [15,
16] whe e he condi ion o ha ing a s a ic coe icien 𝛼𝑖is elaxed
and he ela ionship is allowed o be a non-linea unc ion 𝑓𝑖(⋅)o he
uni a ia e ea u e such ha
𝑔(𝑦) = ∑
𝑖
𝑓𝑖(𝑥𝑖)
This new o mula ion is appealing because i is mo e exp essi e han
GLM while keeping a ela i ely simple and unde s andable s uc u e.
When he model is i ed one can indeed easily isualise each 𝑓𝑖
(also called a shape unc ion) as a unc ion o he 𝑥𝑖 alues, na u ally
p o iding bo h a global iew o he beha iou o he model on a da ase
and a local explana ion o he p edic ion o a single da a poin .
F om a p ac ical poin o iew, ini ially he 𝑓𝑖we e based on
Reg ession Splines o deg ee 𝑑o he o m 𝑓𝑖(𝑥𝑖) ∶= ∑𝑑
𝑘=1 𝛽𝑘𝑏𝑗(𝑥𝑖)
(Spline-GAM) o o he Ke nel Expansions o he ea u e 𝑥𝑖. On he one
hand, his allowed he injec ion o expe knowledge when designing
he model. On he o he hand, he i ing p ocedu e was slow and
some imes did no con e ge p ope ly. These i ing me hods we e hen
shown o be ou pe o med by [17] wi h a p ocedu e o bagging and
boos ing bina y decision ees leading o a class o models called
Explainable Boos ing Machines (EBM). Fo op imisa ion o speed and
memo y EBMs comp ess he ep esen a ion o he unc ion 𝑓𝑖wi h a
lookup able, a da a s uc u e ha bins he ea u e alues and maps
each bin o he heigh o he shape unc ion.
EBMs we e hen u he gene alised o allow also pai wise in e ac-
ions (EB2M) o he o m 𝑓(𝑥𝑖, 𝑥𝑗)by [18] and p o ed o be o alue
wi h an applica ion in heal hca e [19].
A pa allel de elopmen has a emp ed o model he shape unc ions
wi h neu al ne wo ks. [20] pionee ed he wo k wi h he so-called
Gene alised Addi i e Neu al Ne wo ks (GANN), whe e each 𝑓𝑖was
ep esen ed as a small neu al ne wo k. The op imisa ion p ocess ol-
lowed an i e a i e app oach and did no make use o backp opaga ion.
GANN we e success ully used o example wi h he aim o imp o ing
he pe o mance o c edi sco ing applica ions [21]. Recen ly [22]
p oposed Neu al Addi i e Models (NAM), a mode n ein e p e a ion
o GANN wi h mo e neu ons, a new ac i a ion unc ion ExU and a
sophis ica ed aining p ocedu e ha included d opou , weigh decay,
ou pu penal y and ea u e d opou . They epo ed compe i i e ask
pe o mance compa ed o EBMs and o he models on ou da ase s. I
is wo h no ing ha NAM can be a building block o o he models,
o example as he backbone o au oencode s. The main d awback o
NAM is he need o a ca e ul se up o many hype pa ame e s and he
long aining ime ha is equi ed o he con e gence o hese models.
Mo eo e wi h hei o mula ion i is mo e di icul o in oduce expe
knowledge in he sys em and o explici ly equi e a pa o a shape
unc ion o ha e a speci ic o m.
In an e o o pu he human in he loop, [23] buil GAM-Change ,
a amewo k whe e i ed GAM can be inspec ed and changed when
deemed app op ia e. This enables a domain expe o in e ac wi h he
sys em and gi e some ools on how o co ec he model whe e ew
In o ma ion Fusion 99 (2023) 101882
3
E. Ma io i e al.
Table 1
Sho summa y o ela ed models in he li e a u e. CNAM is he new model p oposed in his wo k.
LM LR GLM Spline-GAM EBM EB2M NAM CNAM*
Classi ica ion asks x x x x x x x
Reg ession asks x x x x x x x
Can be backbone o Au oencode s x x x
Can use p io knowledge on a ge x x x x x x
Can use p io knowledge on da a x x
Pai wise in e ac ions x
A bi a y ini ialisa ion x x x x x
Cons ainable x x x
Explici ly balancing x
in e p e abili y-pe o mance
Fig. 1. Schema ic iew o he s uc u e o CNAM ( o classi ica ion) and how cons ain s a e inse ed. Fo eg ession he model is he same wi h he di e ence o no ha ing a
logis ic sigmoid non-linea i y a he end.
s a is ics a e a ailable bu human knowledge can be o help. While he
use does ecei e eedback on he pe o mance o he modi ied model,
he e is cu en ly no way o ine- uning he changed model keeping
ixed he human in e en ions.
Wi h he aim o o e coming d awbacks o cu en s a e o he a
models, in his wo k we in oduce he new CNAM, a way o building
GAM based on neu al ne wo ks bu wi h a speci ic s uc u e such ha i
is possible o ini ialise each shape unc ion as an a bi a y cu e and o
en o ce cons ain s on pa s o he ne wo k. This is impo an because
i allows us o:
1. le e age he powe and speed o EBM which quickly con e ges
o a good solu ion;
2. le e age he exp essi eness o di e en iable p og amming,
opening he possibili y o join ly op imising pe o mance and
in e p e abili y- ela ed me ics.
To conclude his sec ion, Table 1 summa ises he main p ope ies o
he mos ou s anding ela ed models in he li e a u e. As we will see in
he es o he pape , CNAM is a p omising ool o human-in- he-loop
solu ions and can in p inciple be in eg a ed in o GAM-Change enabling
a comple e eedback-loop be ween a human and a machine.
3. Model s uc u e
We can g aphically ep esen CNAM as illus a ed in Fig. 1. The
impo an no el y o CNAM is he cons uc ion o he shape unc ions
𝑓𝑖as a sum o ca e ully ini ialised di e en iable e sions o he uni
s ep, which we will call Di e en iable S ep (DS). I is wo h no ing ha
only a ew shape unc ions a e expec ed o ake non-ze o alues, hus
e ec i ely disca ding some ea u es om he inal sco e by Regula isa-
ion (Few Fea u es Selec ed). The pa ame isa ion o he DS is wha will
allow CNAM o be ini ialised o any a bi a y shape, in pa icula his
p ope y will be used o ini ialise he model o he solu ion o EBM
(mo e on his in Sec ion 3.2).
Once ini ialised, he s uc u e o CNAM is anspa en enough ha
each componen has a clea in e p e a ion and his acili a es he
o mal de ini ion o app op ia e egula ise s o he p oblem a hand.
As in oduced in Sec ion 1, ha ing a model ha is lexible enough
o en o ce di e en cons ain s (e.g., spa si y) is key o a ge ing
speci ic in e p e abili y needs o di e en use cases. An enume a ion
o di e en desi able cons ain s will be he objec i e o he ollowing
sec ion.
3.1. Desi able cons ain s
We now p oceed o lis desi able cons ain s ha could be use ul
in di e en si ua ions. These can be app op ia e when p io knowledge
exis s and can help he model o gene alise be e whe e da a is sca ce:
•Mono onici y o he shape unc ions (e.g., in a banking con ex
we migh wan o en o ce ha he bigge he ‘‘Loan amoun
e m’’ he lowe should be he p obabili y o g an ing he loan).
Mo e o mally: o any 𝑥𝑖,𝑥𝑗such ha 𝑥𝑖≤𝑥𝑗we ha e ha
𝑓(𝑥𝑖)≤𝑓(𝑥𝑗)(posi i e mono onici y) o 𝑓(𝑥𝑖)≥𝑓(𝑥𝑗)(nega i e
mono onici y).
•Smoo hness o he shape unc ions. This could be use ul as a
smoo h unc ion is simple o desc ibe compa ed o a jagged one
(jagged ep esen a ions a e also mo e likely o be in luenced by
noise in he da a). A use migh be in e es ed o explo e i a
smoo h ela ionship be ween he ea u e and i s a ge easonably
model he ask a hand. Mo eo e , since a smoo h cu e is easie
o desc ibe in wo ds han a jagged one, such cons ain becomes
hen use ul when he objec i e is o use na u al language in o de
o p o ide explana ions [24,25].
In o ma ion Fusion 99 (2023) 101882
4
E. Ma io i e al.
Fig. 2. The e ec o changing he pa ame e s o he di e en iable s eps and how hey can o m a 𝑓𝑖.
•Few shape unc ions (use ul o p oblems whe e he numbe o
ea u es is so la ge ha would be imp ac ical o manually assess
e e y shape unc ion). In o he wo ds his en ails ha ing a spa se
ep esen a ion ollowing he p inciple o Ockam’s azo : ‘‘i wo
models desc ibe he same e en , he model wi h less assump ions
(i.e., he model ha elies on ewe a iables) is p e e able’’. I is
possible o ma hema ically en o ce some spa seness o a linea
combina ion o e ms 𝑥pa ame ised by he do p oduc wi h
𝑤 esul ing in 𝑤.𝑥 by minimising he 𝐿1No m o 𝑤, ha is:
𝑚𝑖𝑛 ∑𝑖|𝑤𝑖|
•Local eezing some pa o he shape unc ion (e.g., o help
op imise ai ness measu es). Fo example i we had a ca ego ical
a iable desc ibing some p o ec ed a ibu e one migh wan o
en o ce ha he sco e o he p o ec ed a ibu e ma ch some
o mal desc ip ion o ai ness.
I is impo an o no e ha he desi able cons ain s lis ed abo e a e
no an exhaus i e lis and a e con ex -dependen . Fu he mo e, some
o hese cons ain s may con lic wi h maximising ask pe o mance
o e en among hemsel es. Fo ins ance, while en o cing mono onici y
o a shape unc ion may imp o e he model’s in e p e abili y, i may
nega i ely impac i s ask pe o mance. Simila ly, en o cing spa si y o
shape unc ions may help wi h in e p e abili y, bu a he cos o some
loss in p edic i e accu acy.
Ano he app oach ha could be conside ed is o measu e he ag ee-
men o ea u e impo ances as indica ed by a human expe wi h wha
he model cu en ly p edic s. While his could help he model quickly
con e ge o a well-gene alising solu ion, i should be kep in mind ha
human anking could in oduce biases ha a e no p esen in he da a.
De eloping a sui able me ic o measu ing such ag eemen is a opic
o u u e esea ch.
Despi e hese conside a ions, he main mo i a ion behind his no el
app oach is ha once he desi able cons ain s a e de ined, he op i-
misa ion p ocedu e can explo e di e en solu ions ha aim o sa is y
all he gi en cons ain s. In he ollowing sec ion, we will discuss
how o p ecisely de ine each componen o CNAM and how o de ine
app op ia e egula ise s ha en o ce he desi able cons ain s.
3.2. Model cons uc ion
CNAM speci ies he s uc u e o how each shape unc ion can be
cons uc ed in a a he s aigh o wa d way. In ui i ely, we de elop he
idea ha each shape 𝑓𝑖is a sum o 𝑗s ep unc ions, an app oxima ion
o indica o unc ions
𝑓𝑖(𝑥𝑖) = ∑
𝑗
𝑠𝑡𝑒𝑝𝑗(𝑥𝑖)
This way he model can be ew i en as
𝑔(𝑦) = ∑
𝑖∑
𝑗
𝑠𝑡𝑒𝑝𝑗(𝑥𝑖)
As in oduced p e iously, we wan 𝑠𝑡𝑒𝑝(⋅)( he DS) o be a so e -
sion o an indica o unc ion ha adds o sub ac s a speci ic (lea nable)
quan i y a a ce ain posi ion in he shape unc ion. We pa ame ise
DS wi h h ee pa ame e s: 𝑝𝑜𝑠𝑥,𝑝𝑜𝑠𝑦,𝑠𝑝𝑒𝑒𝑑. DS can hen be de ined as
ollows:
𝑠𝑡𝑒𝑝(𝑥𝑖) ∶= 𝑝𝑜𝑠𝑦∗𝜎(𝑠𝑝𝑒𝑒𝑑 ∗ (𝑥𝑖−𝑝𝑜𝑠𝑥))
whe e
•𝜎(⋅)is he Logis ic Sigmoid unc ion, ha is: 𝜎(𝑥) = 1
1+𝑒−𝑥;
•𝑝𝑜𝑠𝑥is he posi ion on 𝑥whe e he shape is cen ed (see
Fig. 2(a));
•𝑝𝑜𝑠𝑦is he maximum heigh o he s ep (see Fig. 2(b));
•𝑠𝑝𝑒𝑒𝑑 is a pa ame e ha speci ies how quickly he s ep g ows
(see Fig. 2(c)).
In o ma ion Fusion 99 (2023) 101882
5
E. Ma io i e al.
When many ca e ully ini ialised DS a e added up hey can o m a
cu e o a bi a y shape (see Fig. 2(d)).
Then, wi hou loss o gene ali y, we can agg ega e all he shape
unc ions 𝑓𝑖wi h a linea laye pa ame ised by a ec o 𝛼 o p oduce
he inal p edic ion (log-odds in case o classi ica ion), as ollows:
𝑔(𝑦) = ∑
𝑖
𝛼𝑖⋅𝑓𝑖(𝑥𝑖)
The p e ious o mula ion can be use ul o en o cing some egula i-
sa ion like he 𝐿1no m on 𝛼𝑖in o de o a ou a spa se ep esen a ion
(i.e., when ew shape unc ions a e desi ed).
This o mula ion is well-sui ed o bina y classi ica ion asks and
eg ession asks. Howe e , we acknowledge ha ex ending ou model
o o he asks equi es addi ional conside a ions. One possible app oach
o non-bina y classi ica ion is o use a one- s- es ashion, whe e
a sepa a e model is ained o each class. This app oach inc eases
he complexi y o he sys em, bu i can be e ec i e o mul iclass
classi ica ion p oblems.
3.3. En o ceable cons ain s
Wi h his o mula ion we can now begin o impose some o he
cons ain s discussed in Sec ion 3.1 by speci ying addi ional loss e ms
o be join ly op imised o by imposing some s uc u al changes. He e’s
how he lis o possible en o ceable cons ain s enume a ed in Sec-
ion 3.1 can be implemen ed:
•Mono onici y o he shape unc ions: This can be achie ed by
imposing 𝑝𝑜𝑠𝑦and 𝑠𝑝𝑒𝑒𝑑 o be always g ea e (o smalle ) han
ze o. This can be done by applying he unc ion 𝑚𝑎𝑥(𝑥, 0), also
known as ReLU [26,27], o hem.
•Smoo hness o he shape unc ions: This can be achie ed by
incen i ising small alues o he 𝑠𝑝𝑒𝑒𝑑 pa ame e s o he a ious
DS o a gi en shape unc ion. This can be done by minimising he
𝐿2no m o he 𝑠𝑝𝑒𝑒𝑑, ha is o add o he inal loss unc ion he
componen 𝜆√∑𝑖𝑠𝑝𝑒𝑒𝑑2
𝑖. The pa ame e 𝜆con ols he amoun
o egula isa ion. No ice ha wi h his o mula ion we can ha e
smoo h unc ions ha a e s ill able o model big jumps i neces-
sa y, some hing ha spline-based GAM ailed o achie e because
he splines implici ly encode a p io o global smoo hness. On he
o he hand, en o cing an 𝐿2on he 𝑠𝑝𝑒𝑒𝑑 s ill allows local speeds
o be e y high i app op ia e.
•Few shape unc ions: his can be en o ced by imposing spa si y
(i.e., 𝐿1 egula isa ion) on he inal linea laye ha agg ega es
he shape unc ions 𝑓𝑖. Tha is, we can ep esen wi hou loss o
gene ali y he models as 𝑔(𝑥) = 𝛼1∗𝑓1(𝑥1) + ⋯+𝛼𝑝∗𝑓𝑝(𝑥𝑝)
and hen incen i ise a spa se ep esen a ion by adding o he
inal loss he 𝐿1no m o 𝛼:𝜆2∑𝑝
𝑖|𝛼𝑖|, whe e 𝜆2con ols he
amoun o egula isa ion. This is simila o wha [28] p oposed
wi h Lasso, bu i is now possible o apply i o close- o-op imal
EBM solu ions, some hing ha up o now was no possible o do.
•Local eezing: we can exclude pa s o he model om he
op imisa ion p ocess by se ing he g adien o a ce ain s ep o 0.
This p e en any u he upda es on ha pa ame e and hus any
change on ha pa o he unc ion.
3.4. Pa ame e ini ialisa ion
As al eady an icipa ed ea lie , one o he key ea u es o CNAM
is he abili y o be ini ialised as solu ions o as e sol e s (e.g. EBM)
easily. This is impo an because i we y o op imise he model om
a ypical andom ini ialisa ion (e.g., ollowing he p inciples delinea ed
by [29]) we obse ed empi ically ha he aining can be uns able and
esul in subop imal solu ions wi h espec o EBM. On he o he hand
i we op imise he model while s a ing om he solu ion ound by
EBM, hen CNAM is able o end up inding solu ions ha ha e simila
Fig. 3. Example o applying algo i hm 1 o some andom dummy da a wi h he aim
o ini ialising CNAM as a good app oxima ion o EBM.
o highe pe o mance while also op imising he in e p e abili y con-
s ain s ( hus inding a solu ion ha exhibi s a be e balance be ween
pe o mance and in e p e abili y). This will be shown la e wi h he
expe imen s in Sec ion 4.
Fo he speci ic ini ialisa ion we se he ini ial alue o speed o a
la ge alue (we ound 100 o be heu isically a good guess) o be e
app oxima e he uni s ep. The ansla ion pa ame e 𝑝𝑜𝑠𝑥can be se
as he 𝑥-posi ions o he lookup able o he shape unc ion o EBM.
The heigh o he shape unc ion (desc ibed by 𝑝𝑜𝑠𝑦) is inally se o
he 𝑦-posi ion o he EBM by i s ini ialising all he 𝑝𝑜𝑠𝑦 o 0and hen
adjus ing he 𝑝𝑜𝑠𝑦 om he igh -mos o he le -mos using Algo i hm
1. An illus a ion o how his algo i hm wo ks can be seen in Fig. 3.
Tha said, he andom ini ialisa ion can s ill be use ul o some
use-cases whe e i is no s aigh o wa d o ain an EBM (e.g. as
componen s o an Au oencode ). We s ill sugges o ini ialise he pa-
ame e 𝑝𝑜𝑠𝑥 o shape 𝑖as he quan iles o he uni a ia e dis ibu ion
o ea u e 𝑖. On he o he hand o classi ica ion and eg ession pu -
poses we consis en ly ound in ou p elimina y expe imen s ha EBM
al eady con e ges o a close- o-op imal solu ion and ini ialising CNAM
o ha as ly imp o es he inal model ac oss bo h pe o mance and
in elligibili y me ics.
Algo i hm 1 Ini ialise CNAM as EBM
Requi e: 𝑦𝐸𝐵𝑀
𝑦𝐶𝑁𝐴𝑀 ←𝑦𝐸𝐵𝑀 ∗ 0 ⊳Ini ialise all as ze o
𝑦𝐶𝑁𝐴𝑀 [−1] ←𝑦𝐸𝐵𝑀 [−1]
o 𝑘= 2, 𝑘 ≤𝐿𝑒𝑛𝑔𝑡ℎ(𝑦𝐸𝐵𝑀 )+1, 𝑘 + + do
𝑦𝐶𝑁𝐴𝑀 [−𝑘]←𝑦𝐸𝐵𝑀 [−𝑘]⊳Se he heigh o he desi ed one
𝑦𝐶𝑁𝐴𝑀 [−𝑘+ 1] ←𝑦𝐶𝑁𝐴𝑀 [−𝑘+ 1] − 𝑦𝐶𝑁𝐴𝑀 [−𝑘]⊳ ix he e ec
on he ollowing bin
end o
e u n 𝑦𝐶𝑁𝐴𝑀
3.5. Op imisa ion
Once we ha e ini ialised he CNAM pa ame e s, we can op i-
mise he inal loss, which includes bo h ask-pe o mance and ask-
in e p e abili y e ms, using a g adien -based op imise such as ADAM

In o ma ion Fusion 99 (2023) 101882
6
E. Ma io i e al.
Fig. 4. A benchma k o he pe o mance-in e p e abili y adeo s ega ding di e en models on 56 da ase s. The 𝑥-axis ep esen s he ela i e Rank o in e p e abili y (measu ed
as SHAP-Leng h) while he 𝑦-axis co esponds o he ela i e Rank o pe o mance (measu ed as ROC AUC, Accu acy, F1-Sco e). The bes models lie in he lowe -le pa o he
plo s. The do ed blue lines highligh he Pa e o on , i.e. he se o non-domina ed solu ions. Va ious ins an ia ions o CNAM on a e age lie on ha Pa e o on . In e es ingly,
e en wi hou egula isa ion, CNAM sco es be e han EBM (wi hou pai wise in e ac ions).
Fig. 5. A box plo o he esul s o he benchma k o he di e en models ac oss he di e en s a is ics.
In o ma ion Fusion 99 (2023) 101882
7
E. Ma io i e al.
Fig. 6. Pe o mance-In e p e abili y adeo on he MAGIC Telescopes da ase . CNAM
con igu a ions a e colou ed by he amoun o 𝜆1 egula isa ion, wi h blue indica ing
high egula isa ion and g een ep esen ing low egula isa ion. Mo e in e p e able
models wi h lowe SL (x-axis) and be e pe o mance (y-axis) a e si ua ed in he uppe -
le co ne o he igu e. I is impo an o no e ha 𝜆2also in luences he esul s, bu
we ha e omi ed i s isualisa ion o he sake o cla i y.
[30]. This echnique allows us o i e a i ely adjus he pa ame e s o
CNAM owa ds a solu ion ha minimises he a ge loss, e en wi hou
an explici closed- o m o mula o i . To con ol o o e i ing, we
pe iodically e alua e agains a alida ion se and employ an ea ly
s opping c i e ion [31].
4. Expe imen s
CNAM can be used bo h o bina y classi ica ion o eg ession. We
i s benchma k he model wi h di e en egula isa ion pa ame e s on
56 bina y classi ica ion da ase s (see Sec ion 4.1). Subsequen ly, we
pe o m a deep di e in o wo speci ic da ase s, a classi ica ion ask
(see Sec ion 4.2) and a eg ession ask (see Sec ion 4.3), o p o ide
a mo e in-dep h analysis o he model’s beha iou . We implemen ed
CNAM as an open-sou ce so wa e (a ailable online a h ps://gi lab.
nl4xai.eu/e o e.ma io i/cnam), using Py hon and py o ch [32] as
he deep lea ning backend in as uc u e, s uc u ing he code using
py o ch-ligh ning lib a y.
In ou expe imen s, we explo e he adeo be ween pe o mance
and in e p e abili y by a ying he weigh s o he weigh ed loss unc-
ion acco ding o he speci ic applica ion domain. Al hough we lack
heo e ical con e gence gua an ees, ou empi ical obse a ions sugges
ha by ying di e en coe icien s, we can ob ain a di e se se o solu-
ions ha e ec i ely na iga e he pe o mance-in e p e abili y adeo .
La e in he subsec ions, we demons a e his p ac ically wi h di e en
choices o egula isa ion pa ame e s as he added e m o he loss.
4.1. Bina y classi ica ion benchma k
In his sec ion we ocus ou a en ion o he ask o bina y classi-
ica ion and we benchma k CNAM agains o he classi ie s ac oss 56
abula da ase s aken om [33]. We measu e bo h ask-pe o mance
and ask-in e p e abili y sco es. The da ase s a e ela ed o di e en
domains and include bo h eal-da a as well as simula ed-da a and spans
bo h in size and class imbalance. The basic p ope ies o each da ase
a e epo ed in Table 2.
In o de o compa e he esul s we ank he models o each da ase
so ing hem wi h espec o he speci ic me ic alue hey sco ed. Once
his anking is ob ained, we a e age he ank ac oss all he da ase s. We
selec ed h ee measu es o classi ica ion pe o mance:
•Accu acy: he ac ion (pe cen age) o co ec ly classi ied in-
s ances.
•F1-Sco e: he geome ic mean o p ecision and ecall.
•ROC AUC: he a ea unde he ecei e ope a ing cha ac e is ic
cu e (i.e., unde he ROC cu e), ha is he cu e o ue
posi i es e sus alse posi i es a all classi ica ion h esholds.
Fo measu ing in e p e abili y in e ms o model complexi y we
op ed o he so-called SHAP-leng h me ic [11] which is a model-
agnos ic me ic which allows us o compa e he e ogeneous models.
This me ic e u ns he numbe o SHAP a ibu ions o each da a poin
such ha he se o a ibu ions 𝜙𝑖cap u es a gi en ac ion (we se led
a 90% he e) o he o e all explana ion mass ∑𝑖|𝜙𝑖|. Mo e o mally:
𝑆𝐿90% ∶= he smalles 𝑖such ha ∑𝑖so ed(|𝜙𝑖|)
∑𝑖|𝜙𝑖|≤0.9. This in ui i ely
cap u es he leng h o he ‘‘comp essed’’ explana ion o each p edic ion,
ha is wha a use would ha e o ead in o de o ge a ough sense
o wha is he impac o di e en ea u es. These leng hs a e hen
a e aged ac oss all he da a poin s o he da ase in o de o e u n
a single in e p e abili y sco e pe da ase -model pai . I is impo an o
no e ha SL migh no always be he ideal measu e o in e p e abili y.
Indeed, he bes in e p e abili y me ic depends on he con ex , use
p e e ence, and he speci ic domain o he ask. A model wi h a lowe
SL will ypically ha e mo e spa se explana ions, so he lowe he SL,
he less cogni i e load bu den on he use . This is a key conside a ion
when choosing an in e p e abili y me ic, as he goal is o p o ide use s
wi h concise ye meaning ul explana ions ha help hem unde s and
he model’s beha iou and decision-making p ocess.
As s a ed be o e, gi en he sco es o each model on each da ase
we can ank each model on he same da ase acco ding o he a ious
me ics. Once we ha e compu ed he anking o all he me ics o all
he models ac oss all he da ase s we can a e age hem and delinea e
he Pa e o on wi h he bes solu ions. The Pa e o on is he se o
all he Pa e o-e icien solu ions, ha is all hose solu ions in mul i-
objec i e p oblems whe e no o he solu ion is be e han hem in one
o he objec i es.
We es ed CNAM agains a pool o 12 di e en models, mos o hem
implemen ed by he sciki -lea n package [34]: Explainable Boos ing
Machines wi h pa iwise in e ac ions (EB2M) and wi hou in e ac ions
(EBM-GAM), 100 (Random Fo es wi h 100 ees), 1k (Random
Fo es wi h 1000 ees), s m (Suppo Vec o Machine), XGB (eX-
emely G adien Boos ing ees), l (Logis ic Reg ession), small- ee
(a decision ee wi h maximum ee dep h o 4), big- ee (a decision
ee uncons ained), 3-nnc, 5-nnc, 10-nnc ( espec i ely 3-, 5-, and 10-
nea es neighbou s classi ie s).
In o de o e alua e he capabili ies o CNAM we e alua ed 3 di e -
en a ia ions o he p oposed model, each wi h di e en egula isa ion
coe icien s. The egula isa ion was se up such ha a spa si y con-
s ain (𝐿1no m on he 𝛼) is added o he classical c oss en opy loss:
𝑙𝑜𝑠𝑠 =𝑐𝑟𝑜𝑠𝑠𝑒𝑛𝑡𝑟𝑜𝑝𝑦 +𝜆1∗∑𝑖|𝛼𝑖|. This way 𝜆1becomes a mac oscopic
hype pa ame e ha a ou s model simplici y a he po en ial cos o
ask pe o mance. We benchma ked CNAM wi h 𝜆1= 0,𝜆1= 0.1,
𝜆1= 1 in o de o show how CNAM beha es wi h no egula isaion,
small egula isa ion and s ong egula isa ion.
E e y model is e alua ed wi h 5- old s a i ied c oss alida ion.
Fo each single da ase he models a e o de ed om he bes o he
wo s hus p oducing a anking (lowe alues hus co espond o be e
models). The anking is hen a e aged ac oss all he da ase s and he
esul s a e epo ed in Figs. 4 and 5.
Fig. 4 epo s o he sake o cla i y and eadabili y only he mean o
he compu ed ankings. In Fig. 5 he in e es ed eade can see u he
de ails in he o m o box plo s ha help unde s and wha is he
dis ibu ion o he ankings.
The esul s sugges ha CNAM uncons ained is equi alen o (i
some imes ma ginally be e han) EBM-GAM. This makes sense as by
design CNAM is ini ialised as a good app oxima ion o EBM-GAM.
In o ma ion Fusion 99 (2023) 101882
8
E. Ma io i e al.
Table 2
Basic p ope ies o each da ase es ed in ou benchma k s udy.
Da ase # Ins ances # Fea u es Class imbalance
analca da a aids 50 4 0.000000
analca da a asbes os 83 3 0.011758
analca da a bank up cy 50 6 0.000000
analca da a boxing1 120 3 0.090000
analca da a boxing2 132 3 0.005739
analca da a c edi sco e 100 6 0.211600
analca da a cyyoung8092 97 10 0.255181
analca da a cyyoung9302 92 10 0.344518
analca da a aud 42 11 0.145125
analca da a japansol en 52 9 0.001479
analca da a lawsui 264 4 0.732840
appendici is 106 7 0.364543
aus alian 690 14 0.012132
backache 180 32 0.521605
biomed 209 8 0.079691
b eas 699 10 0.096375
b eas cance 286 9 0.164507
b eas cance wisconsin 569 30 0.064940
b eas w 699 9 0.096375
buggyC x 690 15 0.012132
bupa 345 5 0.000412
clean1 476 168 0.016966
cle e 303 13 0.007940
colic 368 22 0.068053
co al 160 6 0.015625
c edi a 690 15 0.012132
c x 690 15 0.012132
ge man 1000 20 0.160000
glass2 163 9 0.004554
hea c 303 13 0.007940
hea h 294 13 0.077792
hea s a log 270 13 0.012346
house o es 84 435 16 0.051795
hunga ian 294 13 0.077792
i ish 500 5 0.012544
labou 57 16 0.088950
lupus 87 3 0.038182
molecula biology p omo e s 106 57 0.000000
monk1 556 6 0.000000
monk2 601 6 0.098895
monk3 554 6 0.001577
mux6 128 6 0.000000
pa i y5 32 5 0.000000
pima 768 8 0.091254
p nn c abs 200 7 0.000000
p nn syn h 250 2 0.000000
p o b 672 9 0.111111
sahea 462 9 0.094470
sona 208 60 0.004530
spec 267 22 0.345762
spec 349 44 0.207560
h eeO 9 512 9 0.004944
ic ac oe 958 9 0.094181
o e 435 16 0.051795
wdbc 569 30 0.064940
xd6 973 9 0.114332
Ins ead, when a egula isa ion o he o m discussed abo e is applied,
we can see how he di e en models ac ually loose classi ica ion pe -
o mance bu gain in in e p e abili y, e u ning models on he Pa e o
on o he bes adeo s. This is impo an because i is an indica ion
ha he model canno only e ain compe i i e ask pe o mance bu can
success ully explo e he adeo be ween compe ing cons ain s in a
way ha he esul lie on he se o Pa e o-e icien solu ions. Mo eo e ,
CNAM eg 1 u ns up as he second bes om he poin o iew o
in e p e abili y (see he plo ela ed o ‘‘Explana ion Leng h’’ on he
bo om igh side o Fig. 5), only behind small- ee while i is much
be e om he pe o mance iewpoin (i.e., CNAM eg 1 is always
be e anked han small- ee in he o he pic u es in Fig. 5).
Finally, i is wo h no ing ha in a speci ic applica ion a use should
expe imen wi h di e en egula isa ion pa ame e s (and also speci ic
egula isa ions i o he speci ic domain) acco ding o his o he needs.
4.2. Illus a i e example on how o add ess a classi ica ion ask: MAGIC
da ase
In o de o gain mo e in ui ion o he beha iou o CNAM on a
speci ic da ase we p o ide as an applica ion he use o CNAM o
he MAGIC Telescopes classi ica ion da ase [35], a eal-wo ld da ase
( aken om he UCI eposi o y [36]) ha is much la ge han all o
hose included in he se o he p e ious benchma k s udy. In his
da ase he ask is o dis inguish be ween wo di e en kind o pa icles
(gamma-like s had ons) gi en a ep esen a ion o he image cap u ed
by he elescope. F om a physical poin o iew, a pa icle en e s he
a mosphe e and p oduce an ellip ic lash o ligh ha is cap u ed by he
came a o he elescope. The image is hen desc ibed by he ollowing
10 nume ical ea u es, some o which desc ibe an ellipsoidal shape
i ed on he pixels:
In o ma ion Fusion 99 (2023) 101882
9
E. Ma io i e al.
Fig. 7. Visual ep esen a ion o each shape unc ion o CNAM on he MAGIC Telescopes da ase . Each shape unc ion demons a es he con ibu ion o a gi en ea u e alue o
he inal p edic ion, which is he sum o he pa ial sco es. The blue solid line ep esen s he mos in e p e able CNAM ins ance (CNAM bes SL), iden i ied as ha ing he sho es
Shap Leng h (SL), while he do ed ed line indica es he bes -pe o ming ins ance (CNAM bes ROC AUC), which has he highes AUC (A ea Unde The Cu e) ROC (Recei e
Ope a ing Cha ac e is ics) cu e sco e among models ained wi h a ying lambda1 and lambda2 egula isa ion pa ame e s. The da a dis ibu ion is isualised as a his og am in
he backg ound, wi h he coun s epo ed on he igh -axis o he plo .
• Leng h: majo axis o ellipse
• Wid h: mino axis o ellipse
• Size: log 10 o he sum o con en o all pixels (pho on coun )
• Conc: he a io o sum o wo highes pixels o e Size
• Conc1: he a io o highes pixel o e Size
• Asym: dis ance om highes pixel o cen e, p ojec ed on o majo
axis
• M3Long: 3 d oo o hi d momen along majo axis
• M3T ans: 3 d oo o hi d momen along mino axis
• Alpha: angle o majo axis wi h ec o o o igin
• Dis : dis ance om o igin o cen e o ellipse
The da ase has 19020 ins ances (12332 gamma and 6688 had on),
wi h a class imbalance o 0.542. In his ask he Accu acy me ic is no
meaning ul as classi ying a had on (backg ound) as a gamma (signal)
is wo se han ice e sa (classi ying he signal as backg ound). Fo
compa ing di e en classi ie a mo e sensible me ic is ROC AUC, as
i allows he e alua ion o he pe o mances a di e en classi ying
h esholds.
Fo illus a i e pu poses we will i CNAM wi h di e en egula i-
sa ion pa ame e s, namely 𝜆1(encou aging spa si y, ha is he shape
unc ions a e mo e o en o 0) and 𝜆2(encou aging smoo h shapes),
d awing hem andomly om a log-uni o m dis ibu ion 𝑌= log(𝑋)
whe e 𝑋=𝑈(−6,1). Each candida e con igu a ion is e alua ed wi h a
5- old s a i ied c oss- alida ion wi h he ROC AUC sco e (pe o mance)
and 𝑆𝐿90% (in e p e abili y). Fo compa ison we will also i o he
compe i i e models: EBM-GAM (as i is CNAM ini ialisa ion), EB2M and
Random Fo es wi h 1000 ees (as bo h models demons a ed high
pe o mances on he benchma k s udy). In Fig. 6 is shown how each
model sco es in he pe o mance-in e p e abili y space.