4 ms·
I hosted my own AI using SillyTavern + text-generation backend (e.g. oogabooga / KoboldAI), just need the power needed to crank the LLM.
by mcbuilder 3y ago
I hosted my own AI using SillyTavern + text-generation backend (e.g. oogabooga / KoboldAI), just need the power needed to crank the LLM.
- brucethemoose2 3y agoKoboldcpp (with ngrok if you need it) is another excellent self hosting solution. 13b will work on 16GB RAM, and 33b on 32GB RAM, with pretty much any dGPU for a little acceleration and RAM offloading. Doubly so if you host it as an AI Horde node (so you have priority access to many models through the web browser).