3 ms·
I think the difficulty will be teaching tricks to the AI, or making them output actions in the right format. Pygmalion + Tavern/Kobold can run a very convincin
by jeroenhd 4y ago
I think the difficulty will be teaching tricks to the AI, or making them output actions in the right format.
Pygmalion + Tavern/Kobold can run a very convincing chatbot with personality on about 10GiB of VRAM, even better with more. The less VRAM you dedicate, the easier the model runs.
For about a thousand dollars you can get an NVIDIA A4000 with 16GiB of VRAM. The GPU isn't particularly fast, but VRAM seems much more important with these models anyway. Gaming GPUs (which perform worse on ML tasks but are still equipped with plenty of VRAM) cost more but are already in the hands of high-end home desktop users.
The hardware is ready and available for those wanting to get started. All you need to do is find a way to connect the output of the AI into some kind of smart home system+interact with cron jobs. A few weekends with the Home Assistant API may be enough to get most things done already.
The only thing I haven't seen open source AIs do is look for information online, like Bing Chat does.