4 ms·
Ask HN: Is anybody selling API access to HuggingFace models?
I've used the OpenAI and GCP chat-bison APIs with great success, but I'd like to put my money into more ethical companies with free-as-in-freedom-er models.
I don't want to fickle around with buying or renting GPUs or installing random LLM scripts that don't work 90% of the time.
I also don't want to pay $XX/month subscriptions, I want clean API billing like OpenAI and GCP.
What exists? Is anybody doing anything?
- smoldesu 3y agoMany of the "free as in freedom" models are not trained or licensed for commercial use. For example, selling access to a LLaMA API would be a direct violation of the license the model was provided with.
- version_five 3y agoI don't think you mean free as in freedom. Maybe freemium, I can't think of a closer short description. They don't cost anything but you're not allowed to use them for most stuff either. Otoh, there are plenty of Apache and MIT licensed models. For example Open LLaMA and Falcon, Dolly, etc. HF lets you filter by license.
- smoldesu 3y agoYou're right. In fairness, the actual code for training LLaMA is "FOSS" technically. It's just that the model license (and weights afaik) are noncommercially licensed. I guess it would be cool to see a paid Falcon/Dolly API, but in my experience those models get smoked by a good LLaMA 7b tuning. You would really have to hate OpenAI to seek out a non-GPT3 solution here...
- version_five 3y agoSort of related, I'm surprised there are no "edge" GPUs available in the style of cloudflare workers or AWS lambda. I heard that cloudflare was going to do something like this but it never materialized. It would be great (as an alternative to API access) to be able to pay for a worker GPU that you only used when doing inference. I know there are technical challenges to that, I wonder of it's still too hard to make something responsive enough to be useful?
- ilaksh 3y agoWhat about replicate.com and RunPod?
- version_five 3y agoThanks! I was looking for something like this last year and couldn't find anything I'm glad to see progress in this space!
- brucethemoose2 3y agoThis is not really an answer, but I really like the "AI horde" API setup. https://aihorde.net/ https://aihorde.net/ Namely, you have a pool of gen AI volunteer workers constantly chewing through API requests. Host requests are prioritized, but the "downtime" of your AI worker instance is spent fulfilling other requests, which earns karma that can be used to prioritize requests from other hosts should you need more throughput. Economically, this is way more efficient than self hosting or "renting" an instance that will be idle between requests, and you aren't paying a massive tax for a proprietary API.