4 ms·

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

deleted 3y ago
[deleted]
xyc 3y ago
Not strange at all! It's a very valid ask. The focus is local AI, but GPT-3.5/GPT-4 are actually included in the app (bring your own key), although customization is limited. Planning to expose some more customizability there including API base urls / model names.
castles 3y ago
https://recurse.chat/faq/#:~:text=We%20support%20Mistral%2C%20Llava%2C%20WizardCoder%20Python%20and%20custom%20GGUF%2C%20as%20well%20as%20OpenAI%20GPT%2D3%20and%20GPT%2D4%20models https://recurse.chat/faq/#:~:text=We%20support%20Mistral%2C%...
- christiangenco 3y ago
  ...how did you highlight a specific sentence like that?
  - sandyarmstrong 3y ago
    Looks like a Chromium-specific feature: https://web.dev/articles/text-fragments https://web.dev/articles/text-fragments Pretty cool. Doesn't work on Firefox.
    - QuinnyPig 3y ago
      It just worked on Safari on iOS. That’s pretty impressive.
      - svat 3y ago
        https://caniuse.com/url-scroll-to-text-fragment https://caniuse.com/url-scroll-to-text-fragment -- yes Safari supports it
  - deleted 3y ago
    [deleted]
iansinnott 3y ago
You could try out Prompta [1], which I made for this use case. Initially created to use OpenAI as a desktop app, but can use any compatible API including Ollama if you want local completions. [1]: https://github.com/iansinnott/prompta https://github.com/iansinnott/prompta
- CGamesPlay 3y ago
  This one doesn't seem to support system prompts, which are absolutely essential for getting useful output from LLMs.
  - derwiki 3y ago
    Can you speak more to this? I get useful output from LLMs all the time, but never use system prompts. What am I missing?
    - CGamesPlay 3y ago
      Sure, I use one system prompt template to make ChatGPT be more concise. Compare these two: https://sharegpt.com/c/fEZKMIy https://sharegpt.com/c/fEZKMIy vs https://sharegpt.com/c/S2lyYON https://sharegpt.com/c/S2lyYON I use similar ones to get ChatGPT to be more thorough or diligent as well. From my limited experience with local models, this type of system prompting is even more important than with ChatGPT 4.
      - addandsubtract 3y ago
        Is there a difference in using a system prompt and just pasting the "system prompt" part at the beginning of your message?
        CGamesPlay 3y ago
        Haven't tested, but having it built-in is more convenient, and convenience is why I'm using these tools in the first place (as a replacement for StackOverflow, for example).
longnguyen 3y ago
Shameless plug: if you need multiple AI Service Provider, give BoltAI[0] a try. It’s native (not Electron), and supports multiple services: OpenAI, Azure OpenAI, OpenRouter, Mistral, Ollama… It also allows you to interact with LLMs via multiple different interfaces: Chat UI, a context-aware called AI Command and an Inline mode. [0]: https://boltai.com https://boltai.com

deleted 3y ago

[deleted]

xyc 3y ago

Not strange at all! It's a very valid ask. The focus is local AI, but GPT-3.5/GPT-4 are actually included in the app (bring your own key), although customization is limited. Planning to expose some more customizability there including API base urls / model names.

castles 3y ago

https://recurse.chat/faq/#:~:text=We%20support%20Mistral%2C%20Llava%2C%20WizardCoder%20Python%20and%20custom%20GGUF%2C%20as%20well%20as%20OpenAI%20GPT%2D3%20and%20GPT%2D4%20models https://recurse.chat/faq/#:~:text=We%20support%20Mistral%2C%...

christiangenco 3y ago
...how did you highlight a specific sentence like that?
- sandyarmstrong 3y ago
  Looks like a Chromium-specific feature: https://web.dev/articles/text-fragments https://web.dev/articles/text-fragments Pretty cool. Doesn't work on Firefox.
  - QuinnyPig 3y ago
    It just worked on Safari on iOS. That’s pretty impressive.
    - svat 3y ago
      https://caniuse.com/url-scroll-to-text-fragment https://caniuse.com/url-scroll-to-text-fragment -- yes Safari supports it
- deleted 3y ago
  [deleted]

iansinnott 3y ago

You could try out Prompta [1], which I made for this use case. Initially created to use OpenAI as a desktop app, but can use any compatible API including Ollama if you want local completions. [1]: https://github.com/iansinnott/prompta https://github.com/iansinnott/prompta

CGamesPlay 3y ago
This one doesn't seem to support system prompts, which are absolutely essential for getting useful output from LLMs.
- derwiki 3y ago
  Can you speak more to this? I get useful output from LLMs all the time, but never use system prompts. What am I missing?
  - CGamesPlay 3y ago
    Sure, I use one system prompt template to make ChatGPT be more concise. Compare these two: https://sharegpt.com/c/fEZKMIy https://sharegpt.com/c/fEZKMIy vs https://sharegpt.com/c/S2lyYON https://sharegpt.com/c/S2lyYON I use similar ones to get ChatGPT to be more thorough or diligent as well. From my limited experience with local models, this type of system prompting is even more important than with ChatGPT 4.
    - addandsubtract 3y ago
      Is there a difference in using a system prompt and just pasting the "system prompt" part at the beginning of your message?
      - CGamesPlay 3y ago
        Haven't tested, but having it built-in is more convenient, and convenience is why I'm using these tools in the first place (as a replacement for StackOverflow, for example).

longnguyen 3y ago

Shameless plug: if you need multiple AI Service Provider, give BoltAI[0] a try. It’s native (not Electron), and supports multiple services: OpenAI, Azure OpenAI, OpenRouter, Mistral, Ollama… It also allows you to interact with LLMs via multiple different interfaces: Chat UI, a context-aware called AI Command and an Inline mode. [0]: https://boltai.com https://boltai.com