4 ms·
what's the memory bandwidth? IIRC that is the limiting factor in LLM hardware today
by throwawaymaths 1y ago
what's the memory bandwidth? IIRC that is the limiting factor in LLM hardware today
- walterbell 1y agoSlide 21, https://files.futurememorystorage.com/proceedings/2024/20240805_PDSC2_Intro_to_HBM_Greenberg.pdf https://files.futurememorystorage.com/proceedings/2024/20240... TPUv3 TPUv4 HBM2 BW 900 GB/s 1200 GB/s