Antithrombotic therapy recommendations in the European Society of Cardiology Guidelines : how robust are the randomized controlled trials underpinning them?

An i h ombo ic The apy Recommenda ions in he Eu opean Socie y o Ca diology Guidelines: How Robus A e he Randomized Con olled T ials Unde pinning Them? Ca a ina M. dos San os1Luísa P ada2Cláudio Da id2,3 João Cos a2,3,4 Joaquim J. Fe ei a2,3 Faus o J. Pin o5Daniel Caldei a2,5 1Faculdade de Medicina, Uni e sidade de Lisboa, Lisboa, Po ugal 2Labo a o y o Clinical Pha macology and The apeu ics, Faculdade de Medicina,Uni e sidadedeLisboa,Lisboa,Po ugal 3Ins i u o de Medicina Molecula , Faculdade de Medicina, Uni e sidade de Lisboa, Lisboa, Po ugal 4Cen o de Es udos de Medicina Baseada na E idência, Faculdade de Medicina,Uni e sidadedeLisboa,Lisboa,Po ugal 5Se iço de Ca diologia, Hospi al Uni e si á io de San a Ma ia (CHULN), CAML, Cen o Ca dio ascula da Uni e sidade de Lisboa—CCUL, Faculdade de Medicina, Uni e sidade de Lisboa, Lisboa, Po ugal TH Open 2021;5:e125–e133. Add ess o co espondence Daniel Caldei a, MD, PhD, Cen o Ca dio ascula da Uni e sidade de Lisboa—CCUL, CAML, Faculdade de Medicina, Uni e sidade de Lisboa, A enue, P o . Egas Moniz, Lisboa, 1649-028, Po ugal (e-mail: [email p o ec ed]om). Keywo ds ►an icoagulan agen s ►an ipla ele agen s ►ca dio ascula sys em ►fib inoly ic he apy ►heal h planning ecommenda ions Abs ac In oduc ion C i icisms ha e been aised agains he sole use o p- alue in in e p e - ing esul s om andomized con olled ials (RCTs). Addi ional ools ha e been sugges ed, like he agili y index (FI), a measu e o a ial’s obus ness/ agili y, and de i a i e measu es. The FI is he minimum numbe o pa ien s who would ha e o be con e ed om none en s o e en s, in he g oup wi h he leas e en s, o a esul o lose s a is ical significance. Objec i e This s udy aimed o e alua e RCT suppo ing Eu opean Socie y o Ca diol- ogy (ESC) guidelines ega ding an i h ombo ics, using he FI and FI- ela ed measu es. Me hods FI, agili y quo ien (FQ), and FI minus LTF los o ollow-up (FI LTF) we e calcula ed o he RCT unde pinning ecommenda ions ega ding an i h ombo ic he apy om he upda ed ESC guidelines. LTF was compa ed wi h FI.Resul s we e calcula ed o he o al g oup o s udies, as pe guideline and as pe ecommenda ion ype. Resul s O e all, 61 s udies we e included. The median FI was 24.5 (in e qua ile ange [IQR]: 9.0–60.0) and median FQ was 0.0035 (IQR: 0.0019–0.0056). Median FI LTF was 2.0 (IQR: 0.0–38.0). Twen y (32.8%) o he s udies had one p ima y o main sa e y ou come wi h LTF exceeding FI. Pe iphe al a e ial disease guideline and ch onic co ona y synd ome guideline had he lowes (2.5; IQR: 1.8–3.3) and he highes (48.5; IQR: 23.8–73.0) FI, espec i ely. Conclusion The median FI sugges s obus ness o clinical ials e alua ing an i h ombo ic d ugs ci ed in he guidelines, bu abou one- hi d o hem had LTF la ge han FI. This emphasizes he need o assessing ials’ obus ness when cons uc ing guidelines. ecei ed June 8, 2020 accep ed a e e ision Janua y 18, 2021 DOI h ps://doi.o g/ 10.1055/s-0041-1725043. ISSN 2512-9465. © 2021. The Au ho (s). This is an open access a icle published by Thieme unde he e ms o he C ea i e Commons A ibu ion License, pe mi ing un es ic ed use, dis ibu ion, and ep oduc ion so long as he o iginal wo k is p ope ly ci ed. (h ps://c ea i ecommons.o g/licenses/by/4.0/) Geo g Thieme Ve lag KG, Rüdige s aße 14, 70469 S u ga , Ge many THIEME O iginal A icle e125 Published online: 2021-04-14 In oduc ion An i h ombo ic he apy, comp ising an icoagulan , an ipla ele , and fib inoly ic d ugs, is he cu en key ea men o some o he majo ca dio ascula diseases. Decisions ega ding he ea men o hese condi ions a e ou inely made in acco dance o guidelines, which a e buil based on andomized con olled ials (RCTs), when a ailable. Mos o en ( hough no always), hese RCTs display s a is ically significan esul s, a concep based on p- alue ( he chance o ob aining esul s a leas as ex eme p o ided he null hypo hesis is ue)o <0.05. Fo long, he p- alue has ecei ed c i icism such as he a bi a iness o i s cu -o o significance, he ac ha i depends on he selec ed s a is ical es and, no less impo an ly, ha i s ue meaning is o en misunde s ood due o i s complexi y.1,2 Consequen ly, mo emen s ha e a isen claiming ha o he measu es, wi h di e en in o ma ion, a he han o alongside he heo e ical p- alue h eshold o 0.05, should be epo ed.3 The agili y index (FI) isone o hose measu es.In oducedin 1990 by Feins ein4and b ough back in 2014 by Walsh e al,5i is a ool o assessing a ial’s obus ness. I can be defined as he minimum numbe o pa ien s who would ha e o be con e ed om none en s o e en s, in he g oup wi h he leas e en s, o he esul s o lose hei s a is ical significance. Thelowe i is, he less obus o mo e agile a s udy is conside ed.5–7The FI ga e ise o o he ools.The agili yquo ien (FQ) is he a iobe ween FI and sample size, allowing he e alua ion o a s udy’s agili y in ela ion o i s numbe o pa icipan s. A highe FQ ep esen s mo e obus ou comes.6I is use ul o compa e obus ness be ween clinical ials o di e en dimensions, whe e he sole use o he FI may cause misin e p e a ions. Fo example, i bo h s udy A and s udy B ha e an FI o 10, i migh be emp ing o hink bo h s udies a e equally obus . Howe e , i s udy A has a sample size o 100 and s udy B o 1,000, FQ o s udy A is 0.1 whe eas FQ o s udy B is 0.01. Nei he FI no FQ has s ic cu -o s unde which hey mus be analyzed. Ins ead, hey a e ools ha mus be in e p e ed a ligh o each s udy’s cha ac e is ics. Hence, FI is o en compa ed wi h he numbe o pa ien s los o ollow-up (LTF). Ha ing an LTF which exceeds he FI in a ce ain RCT migh be a wa ning sign o agili y. The e o e, he di e ence be ween FI and LTF (FI LTF) can also be used as a measu emen o assessing agili y.8The highes his alue is, he mo e obus is he s udy. The ma e o how obus a s udy is should be pa icula ly impo an when i suppo s guideline ecommenda ions. In his in es iga ion, we p opose o assess he obus ness o he ou comes o RCT unde pinning he ecommenda ions ega d- ing an i h ombo ic he apy in he mos ecen e sions o he Eu opean Socie y o Ca diology (ESC) guidelines9 h ough he FI and ela ed measu emen s. The ESC guidelines we e chosen because o hei impo ance o p ac icing physicians. Ma e ials and Me hods Iden ifica ion o S udies We pe o med a comp ehensi e sea ch h ough he ESC web si e sec ion “Guidelines & Scien ificDocumen s”in Sep embe 2019. The sea ch was upda ed in Decem- be 2020 acco ding o newly published guidelines. All he la es e sions o he guidelines we e sc eened and hose men ioning an i h ombo ic he apy (ei he an i- pla ele ,an icoagulan ,o fib inoly ic) we e selec ed. We su eyed each o he selec ed guidelines, o iden i y e e y ecommenda ion le el o e idence (LOE) A o B ( he ones which may be suppo ed by RCT) ega ding an i h ombo ic he apy. Thei ci a ions we e looked up on PubMed. We pe o med a p ima y analysis o i les and abs ac s. All RCTs ha seemed o fi he inclusion c i e ia we e submi - ed o a seconda y ull- ex analysis. Inclusion c i e ia we e (1) RCT which assessed an i h ombo ic he apy in a leas one a m; (2) 1:1 andom alloca ion a io; (3) wo pa allel a ms, wo-by- wo ac o ial design o mo e han wo pa allel a ms, i he ecommenda ion ocused only on wo o hem; (4) a leas one dicho omous p ima y ou - come o main sa e y ou come as s a is ically significan (p<0.05 o a 95% confidence in e al [CI] ha excluded ze o,ass a edineach ial) o anullhypo hesis ha no di e ence exis ed. Since publicly a ailable da a we e employed, ins i u ional e iew boa d app o al was no applicable. Da a Ex ac ion Fi s , we e ie ed all ecommenda ions on an i h ombo ic he apy om he ESC guidelines, i s LOE and class o ecom- menda ion. Then, om each co esponding RCT, da a was ex ac ed on o a p epilo ed o m (Mic oso Excel sp eadshee ). Da a collec ion ocused on p ima y and main sa e y ou comes. I included s udy iden ifica ion, con ol, in e en ion, popula- ion, sample size, con ol and expe imen al g oup sizes, ou - come desc ip ion, numbe o e en s in he con ol and expe imen al g oups, p- alue, CI, and o al LTF. S udy Ou comes The p ima y ou come o his in es iga ion was he agili y/ obus ness o RCT unde pinning he ecommenda- ions om ESC guidelines ega ding an i h ombo ic he apy, assessed h ough he FI, FQ, and FI LTF. Calcula ing FI, FQ, and FI LTF The FI was calcula ed o each ou come using an online calcula- o (a ailable a : h ps://clincalc.com/S a s/F agili yIndex.aspx) which ollows he me hod desc ibed by Walsh e al; adding an e en om he g oup wi h he smalles numbe o e en s and sub ac ing a none en om he same g oup, so as o keep he o al numbe o pa icipan s cons an . Then, a wo-sided Fish- e ’s exac es is ecalcula ed. The p ocess is au oma ically epea ed by he calcula o un il he p- alue is 0.05 o highe .5,10 FQ was calcula ed by di iding each FI o he espec i e sample size.6,7 FI LTF was calcula ed by pe o ming a egula sub ac- ionbu , i he esul was nega i e (i.e., i heLTFou weighed he FI), i was conside ed as ze o.8 S a is ical Analysis FI, FQ, and LTF median and in e qua ile anges (IQRs) we e calcula ed o he whole g oup o included s udies, as pe TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al.e126 guideline, as pe class o ecommenda ion, and as pe LOE. To a oid o e aluing s udies epea edly ci ed in guidelines, we excluded he epe i ions unde he same guideline opic o he pu poses o global analysis and analysis pe guideline. LTF was compa ed wi h he FI o each ou come. We also calcula ed FI LTF o he comple e g oup o s udies, as well as i s median and IQR. Ca ego ical alues we e s a ed as coun s and pe cen ages. Spea man’s co ela ion (R) was used o assess he ela ionship be ween FI and sample size, FI and ecalcula ed p- alue, FI and e en a e, and FI LTF and ecalcula ed p- alue. p-Values o he co ela ions we e calcula ed h ough a wo- ailed S uden ’s - es . All s a is i- cal analysis was done h ough he Mic oso Excel sp ead- shee , apa om calcula ion o p- alues. These we e calcula ed h ough he ClinCalc online calcula o , employing Fishe ’s exac es .10 Resul s Selec ion o T ials and Da a Analysis A o al o 18 ESC guidelines11–28 we e ini ially iden ified as men ioning in any way an i h ombo ic d ugs, wi h 244 co esponding ecommenda ions. This ansla ed in o a sum o 269 s udies. One hund ed and fi y-fi e we e pa al- lel-a m RCT o which 83 had wo a ms (o we e wo-by- wo ac o ial ials o ials wi h mo e han wo a ms o which jus wo conce ned he ecommenda ion unde which hey we e ci ed) and a leas one s a is ically significan p ima y o main sa e y ou come. A e excluding 22 ials which p esen ed only nonin e io i y analyses, we we e le wi h a final sample o 61 s udies, wi h 109 co esponding ecom- menda ions om 12 guidelines.11–20,26,27 Reasons o exclu- sion can be ound in ►Fig. 1. Fig. 1 Flow diag am o included and excluded s udies. ESC, Eu opean Socie y o Ca diology; RCT, andomized con olled ials. TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al. e127 The median sample size o he s udies included was 2,524 (IQR: 855–10,253). The de ails om he s udies and end- poin s a e p esen ed in ►Table 1. The e was a o al o 109 ecommenda ions in ou analysis (►Table 2). Mos s udies analyzed we e used o suppo ecommenda ions class I and LOE A. FI,LTF,andFILTF The median FI o all 61 ials was 24.5 (IQR: 9.0–60.0). Cha ac e is ics o each included s udy, as well as espec i e FI, FQ, and FI LTF can be ound in he ►Supplemen a y Tables S1 and S2. The median FI and IQR as pe guideline is p esen ed in ►Fig. 2A. The guideline on pe iphe al a e ial disease15 had he lowes FI (2.5; IQR: 1.8–3.3). The ch onic co ona y synd ome guideline20 had he highes FI (48.5; IQR: 23.8–73.0; ►Table 3). Resul s o FI median and IQR as pe class o ecommen- da ion and LOE can be ound in ►Fig. 2B. Recommenda ions class III had he lowes FI (7.0; IQR: 4.0–8.0). Six een (26.2%) o he 61 ials did no disclose numbe o LTFs. Fo he o ali y o s udies which did, median LTF was 14.0 (IQR: 7.0–139.0). Median LTF was 13.0 (IQR: 7.0–16.3) o class I; 37.0 (IQR: 6.0–255.0) o class IIa; 139.0 (IQR: 10.0–255.0) o class IIb; and 5.5 (IQR: 3.0–19.3) o class III. LOE A had a median LTF o 14.0 (IQR: 9.0–255.0); LOE B had also a median o 14.0 (IQR: 6.0–44.0). Twen y (32.8%) s udies had one p ima y ou come o main sa e y ou come in which he LTF exceeded he FI. Fou (6.6%) o he 61 ials had a p ima y ou come o main sa e y ou come wi h a FI o 0. ►Fig. 3 shows he equencies o FI, LTF, and FI LTF. Median FI LTF was 2.0 (IQR: 0.0–38.0). Bu 45.0% o he esul s included had a FI LTF o 0. Co ela ions we e R¼0.77 be ween FI and p- alue (p<0.001), R¼0.42 be ween FI and sample size (p<0.001), R¼0.26 be ween FI and e en a e (p<0.001), and R¼0.34 be ween FILTF and p- alue (p<0.001; ►Fig. 4). F agili y Quo ien Rega ding he FQ, i s median o he o al o s udies was 0.0035 (IQR: 0.0019–0.0056). Fo class I, i is 0.0041 (IQR: 0.0019–0.0058); 0.0035 (IQR: 0.0026–0.0094) o class IIa; 0.0026 (IQR: 0.0018–0.0036) o class IIb; and 0.0019 (IQR: 0.0017–0.0030) o class III. Recommenda ions LOE A had a FQ median o 0.0039 (IQR: 0.0019–0.0059) and LOE B had 0.0028 (IQR: 0.0018–0.0041). The guideline on al ula hea disease had he highes FQ and he one on myoca dial in a c ion wi h ST ele a ion had he lowes FQ. Discussion Ou esea ches ablished he agili y o ial ou comes om 61 RCTs suppo ing ecommenda ions ega ding an i h ombo ic he apy om he upda ed e sions o ESC guidelines. Ou median FI was 24.5 (IQR: 9.0–60.0) which is highe han alues epo ed in p e ious s udies in he ca dio ascula field.29,30 The pe iphe al a e y disease guideline15 had he lowes FI (2.5; IQR: 1.8–3.3)which sugges s he RCTs unde pinning i a e mo e agile. Fo he analysis o hisguideline,only wo s udies we e included, due o es ic ions inhe en o he FI me hod. One o he s udies included was CAPRIE (clopidog el e sus aspi in in pa ien s a isk o ischaemic e en s),31 a ial well known o he agili y o i s esul s, wi h a bo de line s a is i- cally significan p- alue o 0.043 o i s main ou come. The o he s udy included by Donaldson e al32 had a sample size o 65 o i s main ou come (and a p- alue calcula ed by us o 0.003). Hence, we can he e see in p ac ice ha bo h bo de line p- alues and a small sample size con ibu e o a low FI. The ch onic co ona y synd ome guideline,20 on he o he hand, had he highes FI (48.5; IQR: 23.8–73.0). Likely, he high sample sizes o he s udies included o his guideline we e he de e mining ac o o his high agili y index. O he six s udies, fi e had mo e han 1,000 pa icipan s and Table 1 Cha ac e is ics o andomized con olled ials ci ed in he guidelines Cha ac e is ics n(%)/median (IQR) Numbe o ials 61 Sample size 2,524.0 (855.0–10,252.8) Numbe o con ol pa ien s 1,270.0 (428.5–5,117.3) Numbe o in e en ion pa ien s 1,254.0 (426.5–5,135.5) Numbe o pa ien s LTF 13.0 (3.5–39.5) Recalcula ed p- alue a 62 b (100) <0.001 23 (37.1) 0.01–0.001 17 (27.4) 0.05–0.01 18 (29.0) 0.05 4 (6.5) Abb e ia ions: IQR, in e qua ile ange; LTF, los o ollow-up. No e: Numbe s epo ed as o al (%) o median (IQR). a p-Values calcula ed using Fishe ’sexac es . b One s udy wi h ac o ial wo-by- wo design was coun ed wice, since he wo pai s o a ms we e analyzed as indi idual s udies. Table 2 Numbe o andomized con olled ials suppo ing di e en classes o ecommenda ion and le els o e idence To al numbe o ecommenda ions Recommenda ions ¼109 RCT ¼77 a Class I 52 34 Class IIA 27 19 Class IIB 24 18 Class III 6 6 LOE A 63 49 LOE B 46 28 LOE C 0 b 0 Abb e ia ions: LOE, le el o e idence; RCT, andomized con olled ials. a Numbe o s udies suppo ing each class/LOE. S udies we e coun ed mo e han once when hey suppo ed ecommenda ions wi h di e en class/LOE. b Since we only included RCT, he e a e no ecommenda ions LOE C. TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al.e128 wo o hese had o e 15,000. The smalles sample size in his g oup was o 563. The median FI LTF in ou analysis was 2.0 (IQR: 0.0–- 38.0), meaning ha in hal o he s udies, he numbe o LTF pa ien s is supe io , equal, o e y close o he numbe o pa ien s whose ou come would ha e o change o ende hese ials’ esul s nons a is ically significan . The in e p e- a ion o hese findings, as well as he po ion esul wi h an FI LTF alue o 0, is limi ed by he ac ha ou analysis included se e al epe i ions, as well as by he ac ha i uses o al a he han in e en ion o LTF con ol. None heless, i may help in e p e he o e all obus ness, i we conside he numbe o imes, a s udy is ci ed in he guidelines is di ec ly p opo ional o i s ela i e impo ance in hei building. Addi ionally, in ou in es iga ion, 20 o he 61 ials (32.8%) had a p ima y o main sa e y ou come in which he LTF ou weighed he FI. Conclusions de i ed om ou - comes whe e he LTF ma ches o exceeds he FI should be aken wi h cau ion. I may ha e been ha hose pa ien s anished om some un o una e wis o ai h ( he figu a i e slip on a banana peel), o ha hey ac ually expe ienced he s udy’s ou come, o bo h. The compa ison o FI and LTF is, he e o e, much mo e aluable han classi ying an FI as high o low, a poin o di e ence om he p- alue. Ne e heless, we mus keep in mind, i is unlikely ha all pa ien s los du ing ollow-up would ha e u ned ou o be e en s om he s udy a m wi h he lowes e en a e. Mos likely, hey we e dis ibu ed be ween he wo g oups and some o hem would end up su e ing he s udy ou come while o he s would no , had hey emained h oughou he whole leng h o he ial. Bu since we canno gua an ee ha his was he case, we ha e o admi he possibili y o he esul s being changed by he LTF pa ien s, especially in hose s udies whe e he LTF la gely ou weighs he FI. Ano he poin o in e es is ou comes wi h an FI o 0. We epo ed a o al o ou (6.6%) s udies wi h a p ima y o main sa e y ou come wi h an FI o 0, meaning ha , wi hou changing he numbe o e en s, nons a is ically significan esul s would ha e been ob ained had he choice o ano he s a is ical es . Co espondingly, on ►Table 1, we epo ed ou s udies wi h a p- alue o 0.05, de e mined h ough Fishe ’s exac es be o e calcula ing he FI. The smalles numbe o pa icipan s epo ed in his g oup o RCT was 840. O he o he h ee s udies, one had a sample size o 900 and wo had sample sizes o e 1,000. Conside ing none o hese numbe s is small enough o ende Fishe ’s exac es , he only s a is ical es sui able, he au ho s’choice o using o he es s in he s a is ical analysis can be easonable. Ou analysis also sough o de e mine he FQ o each ou come. The median FQ was 0.0035 (IQR: 0.0019–0.0056), deno ing ha he e would be no s a is ical significance i 0.35 in 100 pa ien s had expe ienced a di e en e en . The guideline on al ula hea disease had he highes FQ (0.0835; IQR: 0.0835–0.0835). Fo he analysis o his guide- line, only one s udy fi he inclusion c i e ia. This s udy, by Dewilde e al,33 sco ed an FI o 47, gi ing his guideline Fig. 2 F agili y Index pe Guideline and pe Type o Recommenda ion. (A) F agili y index pe guideline. (B) F agili y index pe ype o ecommenda ion. Resul s p esen ed as median and in e qua ile ange. AF, a ial fib illa ion; CCS, ch onic co ona y synd ome; CV P e , ca dio ascula p e en ion; DAPT, dual an ipla ele he apy; DM, diabe es melli us; FI, agili y index; HCM, hype ophic ca diomyopa hy; LOE, le elo e idence;MR,myoca dial e ascula iza ion;n, numbe o s udies suppo ing each guideline; NSTEMI, non-ST ele a ion myoca dial in a c ion; N , numbe o s udies suppo ing all guidelines; PAD, pe iphe al a e ial disease; PE, pulmona y embolism; STEMI, ST ele a ion myoca dial in a c ion; VHD, al ula hea disease.  N di e s om he sum o all N because some s udies appea in mo e han one guideline. TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al. e129 he second highes FI. This, along wi h a ela i ely small sample size compa ing o o he included s udies (563 pa ien s alloca ed) is likely why his guideline had he high- es FQ. Bo h he FI and FQ displayed a dec easing endency om ecommenda ions om class I o hose class III which sugges s ecommenda ions in a o o a ce ain p ac ice a e gene ally suppo ed by mo e obus ials han hose agains i . In he case o ecommenda ions, class III, mean e idence Table 3 FI, FQ, and LTF dispe sion as pe guideline, class o ecommenda ion, and LOE Guideline FIQ1FIQ2FIQ3FQQ1 FQQ2 FQQ3 LTFQ1LTFQ2LTFQ3 AF (n¼11) 8.5 17.0 50.0 0.0041 0.0082 0.0146 3.0 6.5 7.8 CCS (n¼6) 23.8 48.5 73.0 0.0033 0.0049 0.0287 6.8 27.5 44.0 CV p e (n¼4) 24.0 35.0 64.0 0.0019 0.0035 0.0048 13.0 14.0 255.0 DAPT (n¼13) 17.0 26.0 64.0 0.0019 0.0035 0.0049 13.0 14.0 255.0 DM (n¼7) 12.3 18.0 28.3 0.0013 0.0022 0.0030 14.0 91.5 255.0 HCM (n¼1) 17.0 17.0 17.0 0.0023 0.0023 0.0023 43.0 43.0 43.0 MR (n¼20) 4.5 20.0 60.0 0.0019 0.0039 0.0051 10.0 14.0 80.3 NSTEMI (n¼21) 13.0 34.0 62.5 0.0019 0.0035 0.0050 11.5 16.0 44.0 PAD (n¼2) 1.8 2.5 3.3 0.0154 0.0308 0.0462 10.5 21.0 31.5 PE (n¼7) 2.0 3.0 8.0 0.0030 0.0033 0.0119 5.0 7.0 14.0 STEMI (n¼12) 12.0 24.0 68.0 0.0012 0.0019 0.0044 7.0 14.0 42.0 VHD (n¼1) 47.0 47.0 47.0 0.0835 0.0835 0.0835 2.0 2.0 2.0 Class I (n¼34) 4.0 35.0 68.0 0.0019 0.0041 0.0058 7.0 13.0 16.3 Class IIa (n¼19) 14.8 26.0 47.0 0.0026 0.0035 0.0094 6.0 37.0 255.0 Class IIb (n¼18) 14.8 23.5 34.0 0.0018 0.0026 0.0036 10.0 139.0 255.0 Class III (n¼6) 4.0 7.0 8.0 0.0017 0.0019 0.0030 3.0 5.5 19.3 LOE A (n¼49) 10.0 26.0 64.0 0.0019 0.0039 0.0059 9.0 14.0 255.0 LOE B (n¼28) 4.0 14.0 34.0 0.0018 0.0028 0.0041 6.0 14.0 44.0 To al (N ¼61) a 9.0 24.5 60.0 0.0019 0.0035 0.0056 7.0 14.0 139.0 Abb e ia ions: AF, a ial fib illa ion; CCS, ch onic co ona y synd ome; Class, class o ecommenda ion; CV p e , ca dio ascula p e en ion; DAPT, dual an ipla ele he apy; DM, diabe es melli us; FI, agili y index; FQ, agili y quo ien ; HCM, hype ophic ca diomyopa hy; LOE, le el o e idence; LTF, los o ollow-up; MR, myoca dial e ascula iza ion; n, numbe o s udies suppo ing each guideline/class/LOE; NSTEMI, non-ST ele a ion myoca dial in a c ion; N , numbe o s udies suppo ing all guidelines; PAD, pe iphe al a e ial disease; PE, pulmona y embolism; Q1/Q2/Q3, qua ile 1/2/3; STEMI, ST ele a ion myoca dial in a c ion; VHD, al ula hea disease. a N di e s om hesumo allnbecause some s udies appea in mo e han one guideline. Fig. 3 F equencies o di e en ou comes. (A) F equencies o agili y indices, (B) pa ien s los o ollow-up and (C) agili y index minus los o ollow-up. The X-axis ep esen s he FI (A), he LTF (B)and heFILTF (C).TheY-axis ep esen s henumbe o imeseach aluewasen e ed o ou global analysis (as desc ibed in he sec ion Ma e ials and Me hods—S a is ical Analysis). FI, agili y index; FI LTF, FI minus los o ollow-up. TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al.e130 o ha m (since we only included s a is ically significan esul s) in hese ials is agile and he e may be ins ead jus a lack o benefi in he in e en ion. Simila ly o p e ious findings by Gaudino e al,30 he e was a conside able nega i e co ela ion be ween FI and p- alue (R¼0.77,p<0.001).Also, heFIshowedamode a eposi i e co ela ion o sample size (R¼0.42, p<0.001) which is in ag eemen wi h he alues epo ed by Khan e al (R¼0.32)29 and Gaudino e al (R¼0.35).30 The inc ease in he size o samples seems hen o be a candida e o fixing he agili y p oblemwhen designinga ial. Al houghi is ue ha au ho s walk a fine line when ying o balance a emp s o make a s udy as obus as possible, while espec ing e hical p inciples which s a e ha a hypo hesis should be es ed on as ew pa ien s as possible, i is also ue ha agile s udies wi h un eliable esul s which do no p o ide good quali y e idence a e, hemsel es, e hically censu able. Fu he mo e, hey call o addi ional s udies on he same opic, equi ing, in he end, mo epa icipan s hani wouldha e, asingle mo e obus ial been pe o med om he beginning. Limi a ions Ou s udy has some limi a ions, he main one being no including sys ema ic e iews wi h me a-analysis which a e a g ea pa o he e idence behind ecommenda ions. Addi- ionally, due o he cons ain s imposed by he FI me hod i sel , only 61 RCTs we e eligible o analysis. Finally, since ials a e powe ed o p ima y ou comes, we decided o lea e ou seconda y ones. I is also impo an o emphasize ha we selec ed only ials e e ed in he guidelines, which means we migh be a isk o s udy selec ion bias. E en hough i is known ha guideline ecommenda ions a e inc easing, LOE A (highe le el) and class-I o -III ecommenda ions a e dec eas- ing. The e o e, i isimpo an oha e ools o c i icallyapp aise he e idence in he se ing o guidelines.9 Fig. 4 Co ela ion be ween agili y index and ial cha ac e is ics. (A) Co ela ion be ween agili y index and p- alue (R¼0.77), (B) agili y index and sample size (R¼0.42), (C) agili y index and e en a e (R¼0.26), and (D) agili y index minus los o ollow-up and p- alue (R¼0.34). FI, agili y index; FI LTF, FI minus los o ollow-up. TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al. e131 Besides he es ic ions which pa ially shaped ou s udy, o he s a e wo hy o no e: he FI may no be sui able o ime- o-e en ou comes, pa icula ly when he numbe o e en s in he con ol and expe imen al g oups is simila , bu he e is a ma ked di e ence in hei imings.5This is mos ly impo an o ials in he a ea o oncology; addi ionally, since i is no a measu e o e ec (much like he p- alue) i canno be used on i s own o in e p e he esul o a ial.34 Finally, some in es iga ions35 ha e shown s ong co ela ion be ween he FI and p- alue, leading some au ho s o s a e his may be a supe fluous ool.34 Conclusion The FI has no come o eplace s a is ical significance. In ac , i is an absolu e measu e, like o he s which exis (as he numbe needed o ea /ha m) ha aids physicians in be e unde s anding he obus ness o ials beyond ela i e isks and p- alues. The esul s o ou analysis show ha mos o he s a is ically significan s udies ci ed in guidelines o add ess clinical ecommenda ions ha e a good agili y index. This means ha mo e han a ew addi ional e en s a e equi ed o cause loss o s a is ical significance. In ou iew, he FI, as he mos in ui i e and hus a s udied agili y ool he e p esen ed, should be aken in o accoun , when applicable, in he c ea ion o u u e ecommenda ions o clinical p ac ice guidelines, alongside he p- alue and CI. Conflic o In e es D.G.C. has pa icipa ed in educa ional mee ings and/o a ended a con e ences o symposia (including a el, accommoda ion and/o hospi ali y) wi h B is ol-Mye s Squibb, Baye , Boeh inge Ingelheim, Daiichi Sankyo, Me ck Se ono, Fe e , Pfize , No a is and Roche. J.J.C.F. had speake and consul an ees wi h G unen hal, Funda- ção MSD (Po ugal), TEVA, MSD, Alle gan, Med onic, GlaxoSmi hKline, No a is, Lundbeck, Sol ay, BIAL, Me ck-Se ono, Me z, Ipsen, Biogen, Acadia, Alle gan, Abb- ie, Suno ion-Pha maceu icals. F.J.d.C.A.P. had consul an and speake ees wi h As a Zeneca, Baye , BMS, Boeh- inge Ingelheim and Daiichi Sankyo. The emaining au ho s ha e no hing o decla e. Re e ences 1Dick F, Te aea ai H. Significance and limi a ions o he p Value. Eu J Vasc Endo asc Su g 2015;50(06):815 2Wasse s ein RL, Laza NA. The ASA’s s a emen on p- alues: con ex , p ocess, and pu pose. Am S a 2016;70(02):129–133 3Am hein V, G eenland S, Mcshane B. Re i e s a is ical significance. Na u e 2019;567:7–9 4Feins ein AR. The uni agili y index: an addi ional app aisal o “s a is ical significance” o a con as o wo p opo ions. J Clin Epidemiol 1990;43(02):201–209 5Walsh M, S ina han SK, McAuley DF, e al. The s a is ical signifi- cance o andomized con olled ial esul s is equen ly agile: a case o a F agili y Index. J Clin Epidemiol 2014;67(06):622–628 6Tignanelli CJ, Napoli ano LM. The agili y index in andomized clinical ials as a means o op imizing pa ien ca e. JAMA Su g 2019;154(01):74–79 7Del Paggio JC, Tannock IF. The agili y o phase 3 ials suppo ing FDA-app o ed an icance medicines: a e ospec i e analysis. Lance Oncol 2019;20(08):1065–1069 8Mazzina i G, Ball L, Se pa Ne o A, e al. The agili y o s a is ically significan findings in andomised con olled anaes hesiology ials: sys ema ic e iew o he medical li e a u e. B J Anaes h 2018;120(05):935–941 9Caldei aD,Ma ques A, AlmeidaJ,e al.Recommenda ionsandle els o e idence in he guidelines o he Eu opean Socie y o Ca diology: cu en scena io and ime ends. Eu Hea J Ca dio asc Pha mac- o he 2020;6(02):122–124 10 F agili y Index Calcula o . Accessed Janua y 18, 2020 a : h ps:// clincalc.com/S a s/F agili yIndex.aspx 11 Baumga ne H, Falk V, Bax JJ, e al; ESC Scien ific Documen G oup. 2017 ESC/EACTS Guidelines o he managemen o al ula hea disease. Eu Hea J 2017;38(36):2739–2791 12 Colle JP, Thiele H, Ba ba o E, e al; ESC Scien ific Documen G oup. 2020 ESC Guidelines o he managemen o acu e co ona y syn- d omes in pa ien s p esen ing wi hou pe sis en s -segmen ele a- ion. Eu Hea J 2020 (e-pub ahead o p in ). Doi: 10.1093/eu hea j/ehaa575 13 Kons an inides SV, Meye G, Beca ini C, e al; ESC Scien ific Documen G oup. 2019 ESC Guidelines o he diagnosis and managemen o acu e pulmona y embolism de eloped in collab- o a ion wi h he Eu opean Respi a o y Socie y (ERS). Eu Hea J 2020;41(04):543–603 14 Ibanez B, James S, Agewall S, e al; ESC Scien ific Documen G oup. 2017 ESC Guidelines o he managemen o acu e myoca dial in a c ion in pa ien s p esen ing wi h ST-segmen ele a ion: The Task Fo ce o he managemen o acu e myoca dial in a c ion in pa ien s p esen ing wi h ST-segmen ele a ion o he Eu opean Socie y o Ca diology (ESC). Eu Hea J 2018;39(02):119–177 15 Aboyans V, Ricco JB, Ba elink MEL, e al; ESC Scien ificDocumen G oup. 2017 ESC guidelines on he diagnosis and ea men o pe iphe al a e ial diseases, in collabo a ion wi h he Eu opean Socie y o Vascula Su ge y (ESVS): documen co e ing a he o- scle o ic disease o ex ac anial ca o id and e eb al, mesen e ic, enal, uppe and lowe ex emi y a e iesEndo sed by: he Eu o- pean S oke O ganiza ion (ESO)The Task Fo ce o he Diagnosis and T ea men o Pe iphe al A e ial Diseases o he Eu opean Socie y o Ca diology (ESC) and o he Eu opean Socie y o Vascula Su ge y (ESVS). Eu Hea J 2018;39(09):763–816 16 Piepoli MF, Hoes AW, Agewall S, e al; ESC Scien ific Documen G oup. 2016 Eu opean Guidelines on ca dio ascula disease p e- en ion in clinical p ac ice. Eu Hea J 2016;37(29):2315–2381 17 Neumann FJ, Sousa-U a M, Ahlsson A, e al; ESC Scien ific Docu- men G oup. 2018 ESC/EACTS guidelines on myoca dial e ascu- la iza ion. Eu Hea J 2019;40(02):87–165 18 Valgimigli M, Bueno H, By ne RA, e al; ESC Scien ificDocumen G oup. 2017 ESC ocused upda e on dual an ipla ele he apy in co ona y a e y disease de eloped in collabo a ion wi h EACTS. Eu J Ca dio ho ac Su g 2018;53(01):34–78 19 Hind icks G, Po pa a T, Dag es N, e al; ESC Scien ificDocumen G oup. 2020 ESC Guidelines o he diagnosis and managemen o a ial fib illa ion de eloped in collabo a ion wi h he Eu opean Associa ion o Ca dio-Tho acic Su ge y (EACTS). Eu Hea J 2020; 00:1–126 20 Knuu i J, Wijns W, Sa as e A, e al; ESC Scien ific Documen G oup. 2019 ESC Guidelines o he diagnosis and managemen o ch onic co ona y synd omes. Eu Hea J 2020;41(03):407–477 21 Regi z-Zag osek V, Roos-Hesselink JW, Baue sachs J, e al; ESC Scien ific Documen G oup. 2018 ESC Guidelines o he manage- men o ca dio ascula diseases du ing p egnancy. Eu Hea J 2018;39(34):3165–3241 22 B ugada J, Ka i sis DG, A belo E, e al; ESC Scien ificDocumen G oup. 2019 ESC Guidelines o he managemen o pa ien s wi h sup a en icula achyca diaThe Task Fo ce o he managemen TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al.e132 o pa ien s wi h sup a en icula achyca dia o he Eu opean Socie y o Ca diology (ESC). Eu Hea J 2020;41(05):655–720 23 Mancia G, de Backe G, Dominiczak A, e al. 2007 guidelines o he managemen o a e ial hype ension. J Hype ens 2007;25(06): 1105–1187 24 K is ensen SD, Knuu i J, Sa as e A, e al; Au ho s/Task Fo ce Mem- be s. 2014 ESC/ESA Guidelines on non-ca diac su ge y: ca dio as- cula assessmen and managemen : he Join Task Fo ce on non- ca diac su ge y: ca dio ascula assessmen and managemen o he Eu opean Socie y o Ca diology (ESC) and he Eu opean Socie y o Anaes hesiology (ESA). Eu Hea J 2014;35(35):2383–2431 25 Ponikowski P, Voo s AA, Anke SD, e al; ESC Scien ificDocumen G oup. 2016 ESC Guidelines o he diagnosis and ea men o acu e and ch onic hea ailu e: The Task Fo ce o he diagnosis and ea men o acu e and ch onic hea ailu e o he Eu opean Socie y o Ca diology (ESC) de eloped wi h he special con ibu- ion o he Hea Failu e Associa ion (HFA) o he ESC. Eu Hea J 2016;37(27):2129–2200 26 Ellio PM, Anas asakis A, Bo ge MA, e al; Au ho s/Task Fo ce membe s. 2014 ESC guidelines on diagnosis and managemen o hype ophicca diomyopa hy. Eu Hea J 2014;35(39):2733–2779 27 Cosen ino F, G an PJ, Aboyans V, e al; ESC Scien ificDocumen G oup. 2019 ESC Guidelines on diabe es, p e-diabe es, and ca - dio ascula diseases de eloped in collabo a ion wi h he EASD. Eu Hea J 2020;41(02):255–323 28 Habib G, Lancello i P, An unes MJ, e al; ESC Scien ificDocumen G oup. 2015 ESC Guidelines o he managemen o in ec i e endoca di is: he Task Fo ce o he Managemen o In ec i e Endoca di is o he Eu opean Socie y o Ca diology (ESC). Endo sed by: Eu opean Associa ion o Ca dio-Tho acic Su ge y (EACTS), he Eu opean Associa ion o Nuclea Medicine (EANM). Eu Hea J 2015;36(44):3075–3128 29 Khan MS, Ochani RK, Shaikh A, e al. F agili y index in ca dio as- cula andomized con olled ials. Ci c Ca dio asc Qual Ou - comes 2019;12(12):e005755 30 Gaudino M, Hameed I, Biondi-Zoccai G, e al. Sys ema ic e alua- ion o he obus ness o he e idence suppo ing cu en guide- lines on myoca dial e ascula iza ion using he agili y index. Ci c Ca dio asc Qual Ou comes 2019;12(12):e006017 31 Gen M, Beaumon D, Blancha d J, e al. A andomised, blinded, ial o clopidog el e sus aspi in in pa ien s a isk o ischaemic e en s (CAPRIE). Hea 2006;92(01):49–51 32 Donaldson DR, Sal e MCP, Kes e RC, e al. The influence o pla ele inhibi ion on he pa ency o emo o-popli eal dac on bypass g a s. Vasc Endo ascula Su g 1985;19(04):224–230 33 Dewilde WJM, Oi bans T, Ve heug FWA, e al; WOEST s udy in es iga o s. Use o clopidog el wi h o wi hou aspi in in pa ien s aking o al an icoagulan he apy and unde going pe - cu aneous co ona y in e en ion: an open-label, andomised, con olled ial. Lance 2013;381(9872):1107–1115 34 Machado T, Dua e GS, Gonçal es N, Fe ei a JJ, Cos a J. A c i ique o he agili y index. Lance Oncol 2019;20(10):e553 35 Ca e RE, McKie PM, S o lie CB. The agili y index: A P- alue in sheep’s clo hing? Eu Hea J 2017;38(05):346–348 TH Open Vol. 5 No. 2/2021 © 2021. The Au ho (s). An i h ombo ic Guidelines Robus ness San os e al. e133