3 ms·
Yeah, that's possible now. You don't actually need a GPU to run inference against some of the smaller language models - I've heard reports of them running on a
by simonw 4y ago
Yeah, that's possible now. You don't actually need a GPU to run inference against some of the smaller language models - I've heard reports of them running on a RaspberryPi.
The trick for the moment is to skip Python though. lambda.cpp and its many variants are the ones that I've heard working best.
I suggest starting with LLaMA 7B or Alpaca. More notes here: https://simonwillison.net/tags/homebrewllms/ https://simonwillison.net/tags/homebrewllms/
This one is the easiest to get working I think: https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all
- ineedasername 4y agoThanks! I think have my weekend free time planned out now. Or a least a Saturday. I don't do AI at work but playing with code on my time off, even for fun, is a little too work like to really relax. I needs me some mindless entertainment too.