8 ms·

You can set a custom URL to use Ollama which is OpenAI API compatible. Llama3 8b runs quite fast for me on a M1 Max. I'll be making use of this feature I think

You can set a custom URL to use Ollama which is OpenAI API compatible. Llama3 8b runs quite fast for me on a M1 Max. I'll be making use of this feature I think (haven't tried it yet).

jchoelt 2y ago
No, unfortunately it's not possible to set a custom URL... OpenAI is either hard coded or using a configuration option that isn't exposed.
- klaustopher 2y ago
  Advanced settings -> "URL for AI API"
- jarpineh 2y ago
  It is exposed from Advanced tab under "URL for AI API". You do have to change the model name from the preselected and offered OpenAI GPT model names. Otherwise this AI API URL isn't used. Haven't yet tested with local OAI compatible model yet, though, but did manage to break it using bad address :)
  - tinio 2y ago
    How do you change the model name from the preselected and offered OpenAI GPT model names? I don't see the option to do so.
    - weikju 2y ago
      click the name instead of the dropdown arrow Note that to make use of ollama, you'll have to wait for the next release or upgrade to a beta release --- while you can change the API right now it's using a newer format for the payload, that's fixed in the beta/next release.