4 ms·

Glad to see open source models becoming competitive and thanks for sharing the model to community! Lepton AI has hosted this model here free for everyone to giv

by ccding 3y ago

Glad to see open source models becoming competitive and thanks for sharing the model to community! Lepton AI has hosted this model here free for everyone to give it a try (and compare to the original CodeLlama 34B version): https://codellama.lepton.run https://codellama.lepton.run API is also available at https://codellama.lepton.run/api/v1 https://codellama.lepton.run/api/v1 (it's fully compatible with OpenAI's api, so just switching the `api_base` to the this url and all your existing openai client side code should continue working)

eshack94 3y ago
Hey, just an FYI, phind-codellama-34b-v2 was just released, which performs even better! Just letting you know in case you wanted to add the new model too!
rushingcreek 3y ago
I see you're using [INST] tokens. Please don't do this -- this model was not trained in this format or to be a chat model. Instead, it should be treated as a completions model (like text-davinci-003) and no system prompt should be provided. Just tell it what you want.
- bddppq 3y ago
  Thanks for pointing out. We have now updated the prompt template to follow the format.
  - eshack94 3y ago
    Hey, just an FYI, phind-codellama-34b-v2 was just released, which performs even better! Just letting you know in case you wanted to add the new model to the lepton chat you guys set up.