4 ms·
In the colab example it appears you are using the openai python library but with the gorilla model instead of openai's models. That works? How do you set that u
by arbuge 3y ago
In the colab example it appears you are using the openai python library but with the gorilla model instead of openai's models. That works? How do you set that up?
# Query Gorilla server
def get_gorilla_response(prompt="I would like to translate from English to French.", model="gorilla-7b-hf-v0"):
try:
completion = openai.ChatCompletion.create(
model=model,
messages=[{"role": "user", "content": prompt}]
)
return completion.choices[0].message.content
except Exception as e:
raise_issue(e, model, prompt)
- lt 3y agothey point openai.api_base to their server that implements the same API
- arbuge 3y agoAh, I missed that. Thanks.
- OkGoDoIt 3y agoThat’s clever. Do other LLM API’s do that?
- dygd 3y agoYesterday there was a "Launch HN" thread for credal.ai [0] and I noticed that they use the same openai.api_base trick [1]. [0] https://news.ycombinator.com/item?id=36326525 https://news.ycombinator.com/item?id=36326525 [1] https://credalai.notion.site/Drop-In-APIs-3a45d32405c347e8bfd0a570afb61e8b https://credalai.notion.site/Drop-In-APIs-3a45d32405c347e8bf...
- anonzzzies 3y agoIt would take you (or gpt) 3 seconds to write an openai compatible wrapper; the inference api is trivial for all LLMs.