HBM performance on FPGAs

HBM pe o mance on FPGAs Elias Pe domo∗†, Te esa Ce e o∗, Xa ie Ma o ell∗† Behzad Salami∗, ∗Ba celona Supe compu ing Cen e , Ba celona, Spain †Uni e si a Poli ` ecnica de Ca alunya, Ba celona, Spain E-mail: {elias.pe domo, e esa.ce e o, xa ie .ma o ell, behzad.salami}@bsc.es Keywo ds—HBM, FPGA, pe o mance, pseudo-channel, mic o- swi ches I. EXTENDED ABSTRACT Main memo y access has become an inc easing pe o - mance bo leneck o adi ional and High-Pe o mance Com- pu ing (HPC) applica ions. High Bandwid h Memo y (HBM) eme ged as an al e na i e o con en ional DRAMs, o e ing highe bandwid h, lowe powe , and highe in eg a ion capa- bili ies o mee he demands o con empo a y applica ions. The ansi ion o mos ad anced FPGAs om DDR o HBM con i ms his pa adigm shi . Howe e , use s ace subs an ial challenges due o he sca ci y o echnical documen a ion on maximizing HBM ea u es when using FPGAs. We add essed he knowledge gap o HBM cha ac e is ics wi hin FPGAs, aiming o s anda dize i s u iliza ion in he complex HPC domain. Ou Memo y Sandbox enables analysis wi hin and ac oss HBM pseudo-channels. We show ha HBM achie es 99.99% o i s nominal peak bandwid h wi h long sequen ial memo y accesses. Howe e , we obse e a pe o - mance d op o 0.17% wi h educed bu s size and andom da a access pa e ns. Ou s udy spo ligh s he necessi y o me iculous manage- men o concu en accesses and s a egic da a placemen in HBM, o e ing c i ical conside a ions o op imizing HBM pe o mance in FPGA-based sys ems. A. HBM in he cu en compu e a chi ec u e en i onmen Cus om ha dwa e – om wo ks a ions o PCs– has expe i- enced emendous imp o emen s in he pas decades. Howe e , while he speed o comme cial mic op ocesso s has inc eased by app oxima ely 70% e e y yea , he speed o commodi y DRAM has imp o ed by only a ound 50% in he pas decade. As a esul , compu e sys ems a e expe iencing di icul ies in achie ing high p ocessing e iciency[1]. The adi ional app oach o boos ing pe o mance in sys- ems, pa icula ly hose a he edge whe e huge amoun s o da a need o be p ocessed locally o egionally; consis s o adding mo e compu a ional capabili y in o a chip and b inging mo e memo y on-chip. Bu ha app oach no longe scales since we a e now ge ing o he bounda ies o he i ec a o on Neumann a chi ec u es, Moo e’s Law and Denna d scaling. Consequen ly, enginee s ha e begun ocusing on sol ing he bo leneck be ween p ocesso s and memo ies by u ning ou new a chi ec u al designs a a a e no one would ha e an ici- pa ed be o e. An al e na i e, based on ecen echnological ad ances, is mo ing p ocessing elemen s close o, o e en in o he 42% 260% 200% 50% Powe consump ion pe chip is educed o e 68% wi h HBM2 Memo y Bandwid h pe Chip inc eases 530% wi h HBM2 Bandwid h (GB/s) Ra io [mW/Gbps/Pin) Memo y Ve sions Memo y Ve sions 256 192 160 128 96 64 32 0 GDDR5 HBM1 HBM2 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 GDDR5 HBM1 HBM2 Fig. 1. Bandwid h and powe consump ion compa ison. memo y. This solu ion looks o a oid he penal y o epli- ca ing p ocessing elemen s, which p o ides an accep able adeo . When u ilizing wide sho buses (HBM being he mos common example [2], [3]), designe s a oid he penal y o going ou side he die o access o memo y and eco e some o he pe o mance adeo s. HBM sys ems can o e - come all DRAM challenges as an enable o a chi ec u es o high-pe o mance and/o low-powe compu ing, while i s low speed/pin consump ion also imp o es powe e iciency [19], [20](Fig. 1). This end is ollowed by Xilinx, one o he wo ma ke gian s in he a ea o FPGAs and he leade in adap i e compu ing. Xilinx is i mly commi ed o a ansi ion o HBM memo y as a solu ion o memo y bo lenecks, as demons a ed in ecen yea s. In Oc obe 2018, Xilinx launched he Al eo U200 wi h no HBM memo y [4]. Only 1 mon h a e , in No embe 2018 he new Al eo U280 al eady included 8GB o HBM2 and hal ed DDR capaci y [5]. Thei las Al eo Da a cen e ca d elease, he Al eo U55C, doubled HBM capaci y and escinded he DDR memo y banks’ use [6]. B. HBM pe o mance analysis To shed some ligh on he in insic de ails o HBM, we de eloped he Memo y Sandbox ool p o iding highe con igu abili y, mo e con ol o e measu emen s, and u he insigh s (i.e. clock cycles o each memo y ansac ion) han he cu en HBM moni o o e ed by Xilinx. Ou con igu able en i onmen is s uc u ed in wo main pieces: a on -end piece as a use in e ace o se ing up he expe imen s o be execu ed, and a back-end piece composed o a se o ha dwa e IPs o un he expe imen s in he FPGA, acco ding o he da a in oduced in he on end. Thus, he mos ele an IP we ha e de eloped is a highly Con igu able Pa e n Gene a o , which mimics p ocesso h eads da a eques s wi h sequen ial and pseudo- andom memo y access pa e ns. 0 50 100 150 200 250 300 350 400 450 500 WR RD WR RD WR RD WR RD WR RD WR RD WR RD RCB BRC BRGCG RBC RGBCG RBC ue RBC alse Add ess Mapping Policies 32 PsCHs (Read and W i e) Theo e ical Maximum 460.8GB/s 58.35% 61.26% 63.36% 63.76% 44.41% 49.23% 93.50% 99.99% 94.49% 99.99% 94.82% 99.99% 74.39% 74.39% de aul ID ea u e 0 5 10 15 20 25 30 35 40 WR RD WR RD WR RD WR RD BRC RBC RCB RCBI Add ess Mapping Policies 2 DDR4 modules (Read and W i e) Th oughpu (GB/s) Theo e ical Maximum 38.4GB/s 7.98% 11.81% 8.28% 11.98% 77.23% 86.18% 93.82% 93.82% DDR HBM (a) (b) Fig. 2. Th oughpu Resul s o (a)DDR4 and (b) HBM Add ess Mapping Policies. An ini ial analysis o ypical memo y access pa e ns allows us o implemen benchma ks o e eal he subjacen cha ac e is ics o HBM and DDR in FPGAs. Fo his pu pose, we emula e he Repe i i e Sequen ial T a e sal (RST) a ypical sequen ial access pa e n widely used in FPGA p og amming and spa se accesses wi h pseudo- andom accesses. The i s scena ios in end o s ess HBM and DDR o measu e he ac ual h oughpu peak (bandwid h) when using ou Memo y Sandbox. Fo his pu pose, we pe o m sequen ial accesses (RST) in e ically a ached pseudo-channels o banks. We enabled ou s anding ansac ions and bu s sizes we e se o he maximum (16 and 256 bea s, espec i ely). Add ess mapping policies mic obenchma ks esul s a e shown in Fig. 2. Mos mode n compu e applica ions equi e la ge amoun s o memo y access. In HBM, as each pseudo-channel has a size o 256MB, mul iple pseudochannels will likely be accessed by mos applica ions. F om he p e ious expe imen s, we know ha he pe o mance o a single pseudo-channel is he esul o any add ess mapping policy in Fig. 3 di ided by he o al amoun o pseudochannels (32). Fig. 3 shows he esul s o accessing di e en HBM pseudo-channels emula ing a single- h eaded p ocessing elemen connec ed o AXI Po 0. These expe imen s a e pe o med wi h a sequen ial access pa e n (RST), a bu s size o 16 and RBC ue as add ess mapping policy, which o e s he bes pe o mance o his ype o access pa e n acco ding o ou expe imen s. Two main conclusions can be d awn om hese expe imen s: • Pseudo-channels on he same mic o-swi ch show he same pe o mance ega dless o he AXI po accessing hem. • Th oughpu expe iences an a e age deg ada ion o 50% i he p ocessing elemen pe o ms memo y accesses ou side he pseudo-channel o which i is di ec ly connec ed. This pe - o mance loss is he same o he adjacen mic o-swi ch o he u hes one. The e is no linea deg ada ion. The pe o mance is ei he he same o he 4 pseudo-channels wi hin he same mic o-swi ch o 50% in he o he 28 pseudo-channels. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 WR RD Ps-CH in di e en mic o-swi ches Theo e ical Maximum 14.4GB/s Th oughpu (GB/s) µSW0 µSW1 µSW2 µSW3 µSW4 µSW5 µSW6 µSW7 Fig. 3. Th oughpu Resul s o HBM accessing di e en mic o-swi ches. C. Conclusion HBM appea s as a solu ion being in eg a ed in o FPGAs o ace he memo y wall issue and la ge companies a e al eady commi ed o i s use. As expec ed, he h oughpu pe o mance was mo e han 12 imes be e when using all 32 pseudo- channels in he HBM in pa allel han when using he 2 memo y banks in he DDR. The di e en add ess mapping policies, he bu s size, accesses wi hin a mic o-swi ch o ex e nal ones, and he andomiza ion o he add ess can ha e a huge impac on he HBM h oughpu . REFERENCES [1] Todd Ca l Mow y, “Tole a ing la ency h ough so wa e-con olled da a p e e ching,” PhD Thesis, S an o d Uni e si y, Ma . 1994. [2] Copy igh © 2022 Samsung ALL igh s ese ed, “Nex -le el pe o mance,” Samsung HBM, Tech. Rep., 2022. [Online]. A ailable: h ps://www.samsung.com/semiconduc o /d am/hbm/ [3] M. Ujald´ on, “HPC Accele a o s wi h 3D Memo y,” 2016 IEEE In l Con- e ence on Compu a ional Science and Enginee ing (CSE) and IEEE In l Con e ence on Embedded and Ubiqui ous Compu ing (EUC) and 15 h In l Symposium on Dis ibu ed Compu ing and Applica ions o Business Enginee ing (DCABES), pp. 320–328, Aug. 2016, doi: 10.1109/CSE- EUC-DCABES.2016.203. [4] Xilinx Inc., “ALVEO™ P oduc Selec ion Guide Da ashee ,” Xilinx Inc., Tech. Rep. XMP451 ( 1.7), 2021. [Online]. A ailable: h ps://www.xilinx.com/suppo /documen a ion/selec ion- guides/al eo-p oduc -selec ion-guide.pd [5] ——, “Xilinx Ex ends Da a Cen e Leade ship wi h New Al eo U280 HBM2 Accele a o Ca d,” Xilinx Inc., Tech. Rep., No . 2018. [Online]. A ailable: h ps://www.xilinx.com/news/p ess/2018/xilinx-ex ends-da a- cen e -leade ship-wi h-new-al eo-u280-hbm2-accele a o -ca d-dell-emc- i s - o-quali y-al eo-u200.h ml [6] Xilinx Inc., “Xilinx Launches Al eo U55C, I s Mos Powe ul Accele a o Ca d E e , Pu pose-Buil o HPC and Big Da a Wo kloads,” Xilinx Inc., Tech. Rep., No . 2021. [Online]. A ailable: h ps://www.xilinx.com/news/p ess/2021/xilinx-launches-al eo-u55c-i s- mos -powe ul-accele a o -ca d-e e -pu pose-buil - o -hpc-and-big-da a- wo kloads.h ml Elias Pe domo ecei ed a B.Eng. deg ee in Au oma- ion Enginee ing in 2012, and a M.Sc. deg ee in Digi al Sys ems in 2018 bo h om he Technological Uni e si y o Ha ana, Cuba. In addi ion, he ecei ed a M.Sc. Ad anced Mic oelec onic Sys ems Engi- nee ing om he Uni e si y o B is ol, UK in 2019. Since 2020, he has been wo king wi h FPGA Team o he Ba celona Supe compu ing Cen e (BSC) as well a PhD s uden a he Compue A chi ec u e De- pa men o he Uni e si a Poli ecnica de Ca alunya (UPC), Ba celona, Spain. His cu en esea ch in- e es s include Embedded Sys ems, RTL and RISCV SoC design, FPGAs, He e ogeneous Compu ing, au oma ic design gene a ion and memo y man- agemen o HPC and p og amming models.