4 ms·
We offer Llama-2-70B-chat api[0] at Deep Infra. $1 per 1M tokens, support streaming and OpenAI compatible endpoint[1]. 0. https://deepinfra.com/meta-llama/Llam
by yessen 3y ago
We offer Llama-2-70B-chat api[0] at Deep Infra. $1 per 1M tokens, support streaming and OpenAI compatible endpoint[1].
0. https://deepinfra.com/meta-llama/Llama-2-70b-chat-hf https://deepinfra.com/meta-llama/Llama-2-70b-chat-hf
1. https://deepinfra.com/docs/advanced/openai_api https://deepinfra.com/docs/advanced/openai_api
- cjonas 3y agoNice, awesome that you can just repoint an OpenAI client to this service. I'll give it a try. Do you have future plans to support fine-tuning via pay-per-token billing? Related: Found an interesting Reddit post (rare) discussing how OpenAI manages to do this by loading the LoRA layer at runtime. https://www.reddit.com/r/MachineLearning/comments/15zl7ul/d_how_do_you_think_open_ai_hosts_all_these_fine/ https://www.reddit.com/r/MachineLearning/comments/15zl7ul/d_...