5 ms·
George Hotz already implemented LLaMA 7B and 15B on Twitch yesterday on GPU in Tunygrad llama branch: https://github.com/geohot/tinygrad/tree/llama https://git
by xiphias2 4y ago
George Hotz already implemented LLaMA 7B and 15B on Twitch yesterday on GPU in Tunygrad llama branch:
https://github.com/geohot/tinygrad/tree/llama https://github.com/geohot/tinygrad/tree/llama
The only problem is that it's swapping on 16GB Macbook, so you need at least 24GB in practice.
- jalino23 4y agothat is so impressive!
- apetresc 4y agoCan you link to the Twitch stream VoD?
- vopi 4y agohttps://www.twitch.tv/georgehotz/videos?filter=archives&sort=time https://www.twitch.tv/georgehotz/videos?filter=archives&sort... although, there is a VOD channel on YT that might be better.
- yieldcrv 4y agowhy does it say video unavailable?
- stagger87 4y agoI'm pretty sure he only makes his past twitch streams available to subscribers. It's weird they are available on Youtube, maybe he doesn't know that's something he can change.
- jonas21 4y agoI don't think he runs the YouTube channel - it's managed by some of his fans. But like what's he gonna do about it? Send them a takedown notice?
- botdan 4y agoNot sure about his more recent content but he used to have a policy that the VODs could be uploaded in full as long as they were uncut and not modified. If you go watch some older VODs he says so at the beginning.
- vopi 4y agoHmm. I'm not sure. Presumably this will be updated soon: https://www.youtube.com/@geohotarchive/videos https://www.youtube.com/@geohotarchive/videos
- Mistletoe 4y agoThis is such a refreshing and neat way to use Twitch.
- attractivechaos 4y agoBoth are very impressive. A nice thing about Gerganov's implementation is that it is written in almost pure C. Arguably easier for deployment.
- AlecSchueler 4y agoWhat does almost mean in this case?
- bigstumpy 4y agoThere is also C++
- codetrotter 4y agoIow it probably wouldn’t compile with an actual C only compiler, but by and large it looks more like C than like C++?
- deleted 4y ago[deleted]
- BoberMod 4y agoThere is also a gpu-acelerated fork of the original repo https://github.com/remixer-dec/llama-mps https://github.com/remixer-dec/llama-mps
- xiphias2 4y ago> For 7B model, it always goes above 32gb of RAM, That's double of what Tinygrad uses
- DennisAleynikov 4y agoTinygrad is using openCL right?
- lostmsu 4y agoIt's not tinygrad really, it is PyTorch.
- capableweb 4y agoUnfamiliar with the domain, but is it? Looking through the changes (https://github.com/geohot/tinygrad/compare/master...llama https://github.com/geohot/tinygrad/compare/master...llama) it seems to not mention pytorch at all but tinygrad + numpy.
- diimdeep 4y agoThere are two coding sessions streamed so far, very interesting to watch George Hotz | Programming | can we fit a LLaMA inside a tinygrad? https://www.youtube.com/watch?v=0kRDs9BW2NU https://www.youtube.com/watch?v=0kRDs9BW2NU George Hotz | Programming | ChatLLaMA: get in losers we're building a chatbot https://www.youtube.com/watch?v=nctqc8FBJ2U https://www.youtube.com/watch?v=nctqc8FBJ2U