3 ms·
[stub]
by tomhow 1y ago
[stub]
- qat321 1y ago[flagged]
- charleshong 1y ago[dead]
- gfhsad 1y agoWhenever I see '17x faster than experts,' I read 'the experts didn't actually try very hard on the baseline.'
- charleshong 1y agoWell, most of our results are not 17x. But still (IMO) solid across the board! Also, the 17x came from a pretty obscure fusion optimization that isn't called out anywhere in the documentation (we had to run the profiler to see what was actually going on). Wouldn't be surprised if whoever within AWS wrote the kernel didn't know about that optimization.
- snklt 1y ago17x is a wild improvement regardless of the baseline. Impressive results.
- taqpos 1y ago[flagged]
- CobbledSteel 1y agoI'd argue the logic goes the other way, if all it takes to get high performant kernels is to rent a GPU farm, that seems to undercut the years and millions of engineering hours required to build the NVIDIA SW infrastructure. High hopes for smaller players now
- archipelago123 1y agoThe fact that nobody cared to optimize kernels for these hardware platforms proves Nvidia's CUDA moat, especially now that squeezing performance has become so important for serving inference. Hardware ISA is broken => nobody knows how to program the hardware => unoptimized kernels => nobody will use your hardware. Also, bad baselines present opportunities for LLMs to optimize for. Indeed, the kernel that achieved a 17X speedup seems to be a conv1d, which AWS could not care less about optimizing.
- deleted 1y ago[deleted]
- quc1k 1y ago[flagged]
- kap901 1y ago[flagged]
- pakt1 1y ago[flagged]
- matll 1y ago[dead]
- incin1234 1y ago[dead]
- chanwutk 1y ago[dead]
- melissapan 1y agoADRS <> Compiler: what if your “compiler” could think?
- dksgmlwo 1y ago[flagged]
- maltese669 1y ago[dead]
- yrh 1y ago[dead]
- karek 1y ago[dead]
- bvcasd 1y ago[dead]
- dataeaa 1y ago[flagged]
- mavt6 1y ago[dead]
- dfdsfds 1y ago[dead]
- maven5t 1y ago[dead]
- abmfy 1y ago[dead]
- andy_yang 1y ago[dead]