3 ms·
Oh, and some companies offer APIs that match the OpenAI API and there are some open-source projects that do this for llama running locally. Since those would be
by jackmpcollins 3y ago
Oh, and some companies offer APIs that match the OpenAI API and there are some open-source projects that do this for llama running locally. Since those would be compatible with the openai python package they will work with magentic too - though some of these do not support function calling.
See for example Anyscale Endpoints https://app.endpoints.anyscale.com/landing https://app.endpoints.anyscale.com/landing
and https://github.com/AmineDiro/cria https://github.com/AmineDiro/cria
- michaelmior 3y agoThere's also LocalAI[0] which allows the use of local LLMs with an OpenAI compatible API. [0] https://github.com/go-skynet/LocalAI https://github.com/go-skynet/LocalAI
- jackmpcollins 3y agoThanks for sharing! LocalAI supports function calling[0] so this should work for most or all features of magentic - I'm interested to see if concurrent requests work. I will test this out. [0] https://localai.io/features/openai-functions/ https://localai.io/features/openai-functions/