5 ms·
Another abstraction layer library is: https://github.com/BerriAI/litellm https://github.com/BerriAI/litellm For me the killer feature of a library like this wo
by zby 3y ago
Another abstraction layer library is: https://github.com/BerriAI/litellm https://github.com/BerriAI/litellm
For me the killer feature of a library like this would be if it implemented function calling. Even if it was for a very restricted grammar - like the traditional ReAct prompt:
Solve a question answering task with interleaving Thought, Action, Observation usteps. Thought can reason about the current situation, and Action can be three types:
(1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search.
(2) Lookup[keyword], which returns the next sentence containing keyword in the current passage.
(3) Finish[answer], which returns the answer and finishes the task.
After each observation, provide the next Thought and next Action. Here are some examples:
One parameter functions would be enough for the start (but maybe something that would work with multi-line strings - this line oriented grammar is kind of restricted).
- iamjackg 3y agoLocalAI can do that: https://github.com/mudler/LocalAI https://github.com/mudler/LocalAI https://localai.io/features/openai-functions/ https://localai.io/features/openai-functions/
- kcorbitt 3y agoThe problem is that most non-OpenAI models haven't actually been fine-tuned with function calling in mind, and getting a model to output function-calling-like syntax without having been trained on it is quite unreliable. There are a few alternatives that have been (OpenHermes 2.5 has some function calling in its dataset and does a decent job with it, and the latest Claude does as well), but for now it just doesn't work great. That said, it's not that hard to fine-tune a model to understand function calling -- we do that as part of all of our OpenPipe fine tunes, and you can see the serialization method we use here: https://github.com/OpenPipe/OpenPipe/blob/main/app/src/modelProviders/fine-tuned/serializers.ts#L12 https://github.com/OpenPipe/OpenPipe/blob/main/app/src/model... It isn't particularly difficult, and I'd expect more general-purpose fine-tunes will start doing something similar as they get more mature!
- stevenhuang 3y agoCheckout this recently released function calling model, apparently it beats gpt4 by 7% https://huggingface.co/Nexusflow/NexusRaven-V2-13B https://huggingface.co/Nexusflow/NexusRaven-V2-13B
- zby 3y agoReAct was used before OpenAI introduced functions. It might be less reliable - but you can add error checking and retry.