4 ms·

We seriously need a service that is as cheap and fast as the OpenAI/Anthropic APIs but allow us to run the various community-fine-tuned versions of Mixtral and

by dtquad 2y ago

We seriously need a service that is as cheap and fast as the OpenAI/Anthropic APIs but allow us to run the various community-fine-tuned versions of Mixtral and LLaMA 3 that are not/less censored.

cmcollier 2y ago
Here's one I've used: * https://www.together.ai/ https://www.together.ai/ Here are all the models: * https://docs.together.ai/docs/chat-models https://docs.together.ai/docs/chat-models * https://docs.together.ai/docs/language-and-code-models https://docs.together.ai/docs/language-and-code-models
SparkyMcUnicorn 2y ago
I believe openrouter.ai has a few.
whoistraitor 2y ago
You can use HuggingFace, PerplexityAI, Anyscale, TogetherAI. There are a bunch of options. Many offer OpenAI compatible APIs too.
Tiberium 2y ago
https://openrouter.ai https://openrouter.ai is exactly that, and they even support payments with crypto.
- jsemrau 2y ago
  >and they even support payments with crypto. What a value proposition! ;-)
  - immibis 2y ago
    The true meaning of BlockchAIn.
- csomar 2y ago
  Coinbase commerce? That's hardly crypto and I am not sure if it's better or worse than traditional payments.
camel_Snake 2y ago
I addition to those mentioned by others, there is featherless.ai
rthnbgrredf 2y ago
Such services already exists. I don't want to promote any in particular, but if you do a research on pay-as-you-go inference of e.g. mixtral or llama3 you will find offerings that offer an API and charge just cents for XY amount of tokens, exactly as OpenAI does.
Grimblewald 2y ago
or crowd run, like through petals or kobald's ai horde.