4 ms·
I wonder if with the recent research coming to light about the effectiveness of using 3 state weights do they plan to release something along those lines? Can
by margorczynski 3y ago
I wonder if with the recent research coming to light about the effectiveness of using 3 state weights do they plan to release something along those lines?
Can they really compete in FLOP/$ with the likes on Nvidia even if it is a more bespoke architecture than a modified GPU?
- mastax 3y agoNvidia has insane margins so it isn't hard* for a well-funded player to beat them in FLOPS/$. The hard part is the compiler toolchains, the libraries, the documentation, getting models to run fast. * Everything is relative. It's hard to make even a simple microcontroller - but also it isn't hard, you know?
- transitionnel 3y ago"Life in 1 comment"
- VHRanger 3y ago> Nvidia has insane margins so it isn't hard* for a well-funded player to beat them in FLOPS/$ It's hard when you can't buy TSMC capacity
- dannyw 3y agoI’d wait till the dust settles. Maybe 2bit encoding (-1, 0, 0.5, 1) would be easier to design hardware for; as 0.5 can be a bitwise shift.
- imtringued 3y agoNobody cares about FLOPs anymore when it comes to transformer based architectures and considering the transformer architecture is applicable to almost any usecase beyond LLMs, memory bandwidth and memory capacity are the most important metrics. Grayskull has enough 16bit float performance to outrun its memory bandwidth even with 1 ternary bit quantization.