3 ms·
Many of the open source inference solutions you'd use for llama ship with an openai-compatible interface (e.g. oobabooga/text-generation-web-ui, huggingface/tex
by spmurrayzzz 3y ago
Many of the open source inference solutions you'd use for llama ship with an openai-compatible interface (e.g. oobabooga/text-generation-web-ui, huggingface/text-generation-inference, etc). Probably can just change the API host and it would work.