3 ms·
> Not a big fan of how server-centric the LLM landscape is. That's just not true. You can get ooba[1] running in no time, which is 100% made for desktop usage.
by redox99 3y ago
> Not a big fan of how server-centric the LLM landscape is.
That's just not true. You can get ooba[1] running in no time, which is 100% made for desktop usage. There's also koboldcpp and other solution also made for desktop users. In fact, most LLM communities are dominated by end users who run these LLMs on their desktops to roleplay.
AMD being awful is orthogonal here.
[1] https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui