Accelerating Signal Processing with Tensor Cores

Accelerating Signal Processing with Tensor Cores John W. Romein NL-RSE / HW Accel NL meetup July 7th 2025 2 Outline •astronomy & radio telescopes •tensor cores •signal processing with tensor cores –correlation –beam forming •miscellaneous 3 Astronomy & Radio Telescopes 4 Challenges in astronomy •fast radio bursts •dark matter, dark energy •when did the first stars start to shine? •how do supermassive black holes and galaxy clusters shape the universe? •habitability around low-mass stars? •directly detect exoplanets? •... build telescopes→ 5 Why observing in radio? Gamma rays, X-rays and ultraviolet light blocked by the upper atmosphere (best observed from space). Visible light observable from Earth, with some atmospheric distortion. Most of the infrared spectrum absorbed by atmospheric gases (best observed from space). Radio waves observable from Earth. Long-wavelength radio waves blocked. 0 % 50 % 100 % 0.1 nm 1 nm 10 nm 100 nm 1 µm 10 µm 100 µm 1 mm 1 cm 10 cm 1 m 10 m 100 m 1 km Wavelength Atmospheric opacity credits: NASA •atmosphere transparent •some phenomena only visible at radio wavelengths •H I line 6 Problem •long wavelengths •high-resolution images large telescopes→ Solution •combine data from many receivers •use earth rotation 7 LOFAR •LOw Frequency ARray •largest low-freq telescope worldwide •construction started 2005 8 LOFAR receivers •LBA: 10–90 MHz •HBA: 110–250 MHz –tile: 16 receivers 9 LOFAR station •group of 96 LBAs + 48 (96) HBAs •combines signals of selection of 48 LBAs or HBAs 16 GPU Tensor Cores •hardware matrix-multiplication units –limited-precision input data –much faster than regular GPU cores •accelerates deep learning •in recent GPUs 17 GPU Tensor Cores •tensor core compute power increases rapidly source: NVIDIA tensor cores 18 GPU tensor cores •warp (= 32 thread) performs matrix multiplication in one go •decompose large matrix multiplications x B CA = + 32 bitsD16/8/4 bits 16/8/4 bits 32 bits 19 Cheap & expensive tensor cores •“cheap” tensor cores –available in every GPU •“expensive” tensor cores –more powerful –work quite differently –not portable across generations –only available in systems > €35,000 •confusingly: both called 4th (Ada, Hopper) or 5th (Blackwell) gen 20 Cheap & expensive tensor cores •“cheap” tensor cores –available in every GPU •“expensive” tensor cores –more powerful –work quite differently –not portable across generations –only available in systems > €35,000 •confusingly: both called 4th (Ada, Hopper) or 5th (Blackwell) gen 21 Programming tensor cores •through libraries (cuBLAS, cuDNN, …) •directly –CUTLASS •template library •little support for complex numbers –WMMA •AMD: similar API •limited fp8, int4 support •not optimal for Hopper, Blackwell –PTX (assembly) 22 Programming tensor cores •through libraries (cuBLAS, cuDNN, …) •directly –CUTLASS •template library •little support for complex numbers –WMMA •AMD: similar API •limited fp8, int4 support •not optimal for Hopper, Blackwell –PTX (assembly) we use 23 WMMA example load_matrix_sync(a_frag, &a[…][…], K); load_matrix_sync(b_frag, &b[…][…], N); fill_fragment(c_frag, 0); mma_sync(d_frag, a_frag, b_frag, c_frag); // d=a*b+c store_matrix_sync(&d[…][…], d_frag, …); 24 Use tensor cores for signal processing? •yes, if: –algorithm translates to matrix-matrix multiplications •correlator: ✔ •beam former: ✔, ✘ •FIR filter: (in practice) ✘ •non-uniform Fourier transform: ✔ •FFT: ✘ (radix 8: ✔) –operates on few bits ✔ 25 Tensor-Core Correlator 32 Runtime compilation •compile GPU code at runtime –target GPU known –many variables constants→ –improves code readability greatly better performance 33 Tensor-Core Correlator: performance NVIDIA A100 PCIe 40 GB 34 Tensor-Core Correlator: energy efficiency NVIDIA A100 PCIe 40 GB 35 Optimizing for energy efficiency 8-bit input 0 64 128 192 256 320 384 448 512 576 #receivers 0 100 200 300 400 500 performance (teraops/s) Grace Hopper A100 0 64 128 192 256 320 384 448 512 576 #receivers 0 200 400 600 800 1000 energy efficiency (gigaops/joule) Grace Hopper A100 36 Optimizing for energy efficiency •clock frequency tuning –faster ≠ energy efficient 1.98 1.68 1.38 1.08 GPU clock frequency 0 100 200 300 400 500 performance (teraops/s) 1.98 1.68 1.38 1.08 GPU clock frequency 0 200 400 600 GPU power (Watt) 1.98 1.68 1.38 1.08 GPU clock frequency 0.0 0.5 1.0 energy efficiency (teraops/joule) 37 Tensor-Core Beam Former special thanks to Leon Oostrum, Bram Veenboer, Ronald Rook, Pieter Kruizinga, Michael Brown 38 Beam Forming •combines antenna signals –directional sensitivity •many application domains •samples matrix * weights matrix 39 Beam Forming with Tensor Cores •yes, if: –algorithm translates to matrix-matrix multiplications –multiple simultaneous beams ✔, ✘ –weights constant over (some) time ✔, ✘ –operates on few bits ✔ 40 Tensor-Core Beam Former generic tensor-core beam-forming library ultrasound interface radio-astronomical interface 41 Medical ultrasound use case •blood flow in mouse brain •real time •huge matrix with 1-bit samples 48 The I/O challenge ●GPU correlators: >100x performance in 10 years ●exploring techniques for 100x I/O performance ●1.2 Tb/s packet processing on Grace Hopper with DPDK GPUdev1 1200 Gb/s packet generator correlator 1) J.W. Romein, “Breaking the I/O Barrier: 1.2 Tb/s Ethernet Packet Processing on a GPU”, Euro-Par'25, Dresden, Germany, August 25-29. 2025 49 Studying energy efficiency •PowerSensor 31 –measures GPU power use –20 kHz –optimize for energy efficiency 1) S. v.d. Vlugt et al. “PowerSensor3: A Fast and Accurate Open Source Power Measurement Tool”. IEEE ISPASS'25, Genth, Belgium, May 11-13, 2025 50 Take away •tensor cores great for correlation, beam forming –~10x performance & energy efficiency –hide complexity in libraries •rethink your own algorithms 51 Acknowledgements •This work was funded by –Netherlands eScience Center (PADRE, RECRUIT) –EU H2020 FETHPC (DEEP-EST, Grant Agreement nr. 754304) –NWO Netherlands Foundation for Scientific Research (DAS-5, DAS-6) –ESO (ALMA GPU Correlator Study) –EU HORIZON INFRA-TECH (RADIOBLOCKS, Grant Agreement nr. 101093934)