3 ms·
LLMs are difficult to run and very few people have the hardware to run a big enough model to be useful. This means you have to integrate LLMs over an API, which
by Dr4kn 2y ago
LLMs are difficult to run and very few people have the hardware to run a big enough model to be useful. This means you have to integrate LLMs over an API, which costs you money. LLMs are to expensive for a free browser to just give away free usage. Implementing APIs that you can login with an account and use a specific LLM is a lot of work and difficult to communicate the privacy nightmare that comes with sending that data to some LLM.
You don't have to cater to every user. Focusing on things you do better for a huge portion of potential users is a much better decision. If features turn out to be so useful that everyone needs to integrate them to not be left behind you have to do it. LLMs are not there yet. They are to unreliable, expensive and difficult to run locally. In a few years when most users might be able to run decent models locally you can reevaluate that decision