4 ms·
Yeah its an awkward position, as self-hosting is going to be insanely expensive unless you have a substantial userbase to amortize the costs over. At least for
by ted537 1y ago
Yeah its an awkward position, as self-hosting is going to be insanely expensive unless you have a substantial userbase to amortize the costs over. At least for a model comparable to GPT-4o or deepseek.
But at least if you use an API in the same region as your customers, court order shenanigans won't get you caught between different jurisdictions.
- 999900000999 1y agoIdeally smaller models will get better. For most tasks I don't need the best model in existence, I just need good enough. A small law firm using LLMs for summaries can probably do it on prem and hire a smart college student to setup a PC to do it. The problem is that's still more difficult ( let's say our hypothetical junior IT only makes 60k a year) than just sending all your private business information to some 3rd party API. You can then act shocked and concerned when your 3rd party leaks the data.