3 ms·
With the recent breakthrough in ternary weight models it'll be interesting to see how needed this kind of bruteforce power will be.
by hresvelgr 3y ago
With the recent breakthrough in ternary weight models it'll be interesting to see how needed this kind of bruteforce power will be.
- throwaway4good 3y agoYou would still use floating point for training afaik.
- Aardwolf 3y agoWhy not have both software and hardware improvements? This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented
- TeMPOraL 3y agoYeah, there's this weird notion that OSS work on running LLMs on consumer hardware, and researching more efficient architectures, is somehow in opposition or competition to the Evil AI Wielding Corporations. It's not. Like always, whatever tool an individual has, a corporation can afford to use more of it, and use it better.
- hresvelgr 3y ago> This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented I didn't assert anything here, I merely pondered. However, lets pretend that I did say that we shouldn't need heavy GPUs anymore. I don't think that's an entirely bad take because it would be fantastic if we could run larger models with more parameters on lower spec hardware. Especially phones that aren't flagship models.
- Aardwolf 3y agoSure, the analogy was a bit exaggerated perhaps :) I also of course didn't necessarily want to imply whether it's actually a good idea to have a 1000W hogging monster card, vs e.g. putting more memory on consumer cards