7 ms·
Gorilla: Large Language Model Connected with APIs
- Aditya_Garg 3y agoHeads up, your discord link is broken
- jeron 3y agoworking for me...
- edmundsauto 3y agoHow does this compare to LangChain?
- TechBro8615 3y agoI don't know how its performance compares, but its architecture is completely different: LangChain is a "normal" software library, but Gorilla is itself an LLM: > Gorilla is a LLM that can provide appropriate API calls. It is trained on three massive machine learning hub datasets: Torch Hub, TensorFlow Hub and HuggingFace. We are rapidly adding new domains, including Kubernetes, GCP, AWS, OpenAPI, and more. Zero-shot Gorilla outperforms GPT-4, Chat-GPT and Claude. Gorilla is extremely reliable, and significantly reduces hallucination errors. My reading of that abstract is that it's an LLM that outputs API calls instead of natural language (or maybe it still outputs natural language, but it can use API calls during inference? I didn't read very far), whereas LangChain is simply a software library. In theory, you could probably get Gorilla to output LangChain "API" (function) calls...
- valine 3y ago"We release Gorilla, a finetuned LLaMA-based model that surpasses the performance of GPT-4 on writing API calls" Sounds like it's another LLaMA variant specifically fine tuned for API calls.
- shishirpatil 3y agoGood point. This was the original release, we now also have Apache-2.0 licensed models finetuned on MPT-7B and Falcon-7B!
- Kerbonut 3y agoThere is Open Llama 7B which is Apache 2.0 licensed, please consider checking it out https://github.com/openlm-research/open_llama https://github.com/openlm-research/open_llama
- csjh 3y agoStill outputs natural language. Example from their colab: Input: I would like to translate from English to Chinese. Output: <<<domain>>>: Natural Language Processing Text2Text Generation <<<api_call>>>: M2M100ForConditionalGeneration.from_pretrained('facebook/m2m100_1.2B') <<<api_provider>>>: Hugging Face Transformers <<<explanation>>>: 1. Import M2M100ForConditionalGeneration and M2M100Tokenizer from the transformers library. 2. Load the pre-trained M2M100 model and tokenizer using the from_pretrained() method. The model is trained to translate text from English to Chinese, among other languages. 3. Encode the input text in English using the tokenizer. 4. Generate the translation using the model.generate() method. 5. Decode the output tokens using the tokenizer to obtain the translated text in Chinese. 6. Print the translated text.
- shishirpatil 3y agoHi @csjh, we trained to model to also additionally output additional context so it would be useful for a downstream task. We wrapped the API call with special decorator so it's easier to just regex. Would you like to just have the API instead? Happy to release an API only model if there is wider interest - It's a strictly easier task for Gorilla LLM :)
- shishirpatil 3y agoLangchain is a terrific project that tries to teach agents how to use tools using prompting. Our take on this is that prompting is not scalable if you want to pick between 1000s of APIs. So Gorilla is a LLM that can pick and write the semantically and syntactically correct API for you to call! A drop in replacement into Langchain!
- sroussey 3y agoCurrently released weights: https://huggingface.co/gorilla-llm https://huggingface.co/gorilla-llm
- deleted 3y ago[deleted]
- shishirpatil 3y agoHi HN, I'm one of the lead authors on the paper! Gorilla is an open source effort and we would love to hear from the community. Let us know if you have any questions or suggestions!!
- joshuanapoli 3y agoIs Gorilla helpful for private APIs? Is it addressing the same need that OpenAI's new "function-calling" feature?
- gsharma 3y agoCongratulations on shipping! This is pretty cool. Building next-gen Zapier on top of this would be a great use of this. There is a finite number of public APIs (100K?) which keeps the problem manageable. IMO, adding support for custom/private APIs (something like OpenAI functions) will make this a very powerful tool.
- shishirpatil 3y agoThanks for the kind words @gsharma! Private APIs is something we are definitely thinking about.
- CyrsBel 3y agoOutperforms ChatGPT and GPT-4 in generating api calls? Or in every type of querying? It's exciting that this is open, looking forward to trying all of this from end to end!
- absk82 3y agoYour license says it can be used commercially by anyone while Llama's license says it can only be used for research purpose. Isn't your license bound by the Llama usage license ?
- rahimnathwani 3y agoAIUI it uses the Llama architecture, but not Facebook's Llama weights. It uses MPT-7B, which was trained from scratch: https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b
- swyx 3y agothe code is open source but not the weights. as far as i can tell.
- shishirpatil 3y agoHi swyx, the weights are also open sourced at https://huggingface.co/gorilla-llm https://huggingface.co/gorilla-llm Let us know if you are unable to access them.
- shishirpatil 3y agoYes we have three set of models. One based on llama - which you are right, cannot be used commercial. We have two additional models based on MPT-7 base and Falcon-7B which can be used commercially with no obligations!
- Kerbonut 3y agoThere is Open Llama 7B available that is Apache 2.0 licensed. Would you consider fine tuning that one as well for a commercial use of this with Llama model?
- shishirpatil 3y agoYes, when we released the initial set of models Open Llama was still at 600B checkpoint and not finished training yet. It's an easy port :)
- tianjunz 3y agoHey everyone, I am one of the authors of the Gorilla project. Super excited to see how the project grows! We have released LLaMA based, MPT based (Apache 2.0) and Falcon based (Apache 2.0) models so far. Something cooler is coming soon!
- tianjunz 3y agoWe named the project Gorilla cause it is an cute animal that use tools !
- random5245 3y agoIs this module raw uncensored ?
- jarulraj 3y agoNeat idea, @shishirpatil! We are developing EvaDB [1] for shipping simpler, faster, and cost-effective AI apps. Can you share your thoughts on transforming the output of the Gorilla LLM to functions in EvaDB apps -- like this function that uses the HuggingFace API -- https://evadb.readthedocs.io/en/stable/source/tutorials/07-object-segmentation-huggingface.html#register-hugging-face-segmentation-model-as-an-user-defined-function-udf-in-evadb https://evadb.readthedocs.io/en/stable/source/tutorials/07-o...? [1] https://github.com/georgia-tech-db/eva https://github.com/georgia-tech-db/eva
- arbuge 3y agoIn the colab example it appears you are using the openai python library but with the gorilla model instead of openai's models. That works? How do you set that up? # Query Gorilla server def get_gorilla_response(prompt="I would like to translate from English to French.", model="gorilla-7b-hf-v0"): try: completion = openai.ChatCompletion.create( model=model, messages=[{"role": "user", "content": prompt}] ) return completion.choices[0].message.content except Exception as e: raise_issue(e, model, prompt)
- lt 3y agothey point openai.api_base to their server that implements the same API
- arbuge 3y agoAh, I missed that. Thanks.
- OkGoDoIt 3y agoThat’s clever. Do other LLM API’s do that?
- dygd 3y agoYesterday there was a "Launch HN" thread for credal.ai [0] and I noticed that they use the same openai.api_base trick [1]. [0] https://news.ycombinator.com/item?id=36326525 https://news.ycombinator.com/item?id=36326525 [1] https://credalai.notion.site/Drop-In-APIs-3a45d32405c347e8bfd0a570afb61e8b https://credalai.notion.site/Drop-In-APIs-3a45d32405c347e8bf...
- anonzzzies 3y agoIt would take you (or gpt) 3 seconds to write an openai compatible wrapper; the inference api is trivial for all LLMs.
- fareesh 3y agoWhat's a good/affordable GPU to run these projects locally? It seems like building anything on top of these runs into either a big GPU cost for yourself or a big compute cost if you scale for others.
- Tostino 3y ago3090 can run 30b parameter models, 2x can run 65b parameter models. 4090 can run the same, very slightly faster for much more money.
- thisisit 3y agoPreviously discussed: https://news.ycombinator.com/item?id=36073241 https://news.ycombinator.com/item?id=36073241
- lmeyerov 3y agoAt first I was excited -- this is the second time i'm.seeing this advertised, and we are thinking through reliable API call-out strategies for louie.ai -- but then I got confused by the paper: Is this really just tested against 95 API calls, and I'm guessing largely from just a small number of libriaries like pytorch? More importantly, if anywhere near true, is there any reason to (so far) use this for use cases like OpenAI's around calling generic OpenAPI style libs (zapier scenario), known specific tools, or random python libs not in that dataset? I'm really thinking 3 scenarios for our users: -- python libraries we know they'll want to use ahead of time, like pandas and pygraphistry -- Same for CLI, like AWS and az, and OpenAPI from and index -- Long-tail that we don't expect, esp in python + js, so on the fly, with limited time budget for inspecting GitHub/Google/etc So far, we generally find auto approaches too unreliable for non-hobbyists, and have to tune a bunch for each tool and database we teach louie. This line of research is def interesting to us...
- sfriedr 3y agoCongratulation, great paper! It should have been put on HN earlier ;) I have a few questions: * you say (page 4): "We then perform standard instruction finetuning on the base LLaMA-7B model" Could you perhaps provide a reference to the _exact_ finetuning approach you used? I'm afraid different groups of people have a different notion of "standart" (see for example pages 131-155 from https://arxiv.org/abs/2302.08575 https://arxiv.org/abs/2302.08575 for various fine-tuning approaches) and without knowing exactly how fine-tuning was carried out, it can be very difficult reproduce your research and results exactly. * the idea of using AST Sub-Tree Matching is nice. Could you please let me know which function in which file from your GitHub repository this is implemented in? Again, great job on publishing this paper! --- Best regards, friederrr.org
- data_maan 3y agoSeems @shishirpatil ran out of steam answering questions. Too bad.
- data_maan 3y ago(Or maybe the questions were too tricky and he wasn't able to answer, heh)
- shishirpatil 3y agoHaha, was busy yesterday! Or was I? :P
- shishirpatil 3y agoThanks @sfriedr We generate self-instruct data and then fine tune the base model with perplexity loss. The self-instruct data is https://github.com/ShishirPatil/gorilla/tree/main/data/apibench https://github.com/ShishirPatil/gorilla/tree/main/data/apibe... Thank you! Yes, the code can be found here: https://github.com/ShishirPatil/gorilla/tree/main/eval/eval-scripts https://github.com/ShishirPatil/gorilla/tree/main/eval/eval-... Hope this helps. Let me know if you have any follow-ups!
- sublimefire 3y agoMy issue with this is that it needs to be retrained on a regular basis to make sure latest APIs are included. There needs to be a long term assessment to understand its viability in a commercial setting. Otherwise we'll jump in and after 6 months it will begin producing out of date suggestions for some edge cases. And then again if you need to support an old API how can you be sure it will produce the scoped results?