4 ms·
It makes me wonder how long it will be until open-source locally-run AI chatbots reach the GPT-4 level. Five years? Ten?
by causality0 3y ago
It makes me wonder how long it will be until open-source locally-run AI chatbots reach the GPT-4 level. Five years? Ten?
- swyx 3y agoits more a question of hardware progress than software probably. theres a minimum level for these behaviors to emerge
- crooked-v 3y agoThe real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.
- brucethemoose2 3y agoNot anymore. Llama.cpp and kobold.cpp run well with low vram and non Nvidia GPUs. Huggingface is full of finetunes now, and I believe a 33b model can be finetuned on a single 3090. Llama.cpp is developing some kind of training, but I have no idea what the requirements will be.
- causality0 3y agoTo me, GPT's real "secret sauce" is how it primes the model for interactive prompting instead of just text completion.
- ynniv 3y agoThere are "chat" and "instruct" variants of most models now.
- lettergram 3y agoIt depends on the tasks, I’d argue some of the open source alternatives are at the gpt-4 level already, particularly for code generation. That said, I suspect summarization, translation, etc will take time. I’d suspect under 1 year.
- worldsayshi 3y ago>I’d argue some of the open source alternatives are at the gpt-4 level already, particularly for code generation. Like which one?
- brucethemoose2 3y agoLarge context Chronos 33b (and some mixes) for roleplaying type chat. And there are some very new 65b finetunes I have not tried.
- vorpalhex 3y agoThe open source bot I played with last week (Wizard Vicuna uncensored) was 85% of gpt 3.5 on a VERY hard use case (fiction stories). Maybe a year before we are the level of stablediffusion?
- deleted 3y ago[deleted]
- eob 3y agoWhen it happens, I think it’ll happen on our phones first.
- koheripbal 3y agoGiven the fundamental hardware matrix operations, that seems unlikely, unless it's something very scaled down.