4 ms·

I'm actually working on an open source product to solve this. For a long while I've wanted a good "pro" UI that can connect to multiple different llm APIs Con

I'm actually working on an open source product to solve this. For a long while I've wanted a good "pro" UI that can connect to multiple different llm APIs Convenient editing and branching is one of the items in my roadmap already, what else do you think I could include?

lumenwrites 2y ago
Good history search (including non "main" conversation branches) and convenient conversation management (bookmarking, folders, maybe something smarter) would be great. Also, maybe some convenient way to create message templates? I don't know how I'd implement this, I just know that I often write one long prompt that I reuse multiple times, with multiple minor tweaks/edits, and it'd be amazing to have a convenient tool to manage that. Also, good mobile/tablet support, convenient to use and without bugs (as I happen to spend most of my time writing prompts on my ipad, but that's just me). If you already have a demo - please share a link, I'd be happy to beta test it and maybe become one of the early customers.
- namanyayg 2y ago
  wow, reading your comment is a great mood boost for me because these are literally the exact features I want from my llm chat experience. It's great to see someone with the exact same problem set. I just followed you on Twitter (I'm @NamanyayG there as well), I'll definitely ping you when I have something to test.
  - handfuloflight 2y ago
    [dead]
TobTobXX 2y ago
You might be looking for "LLM Web-UI"s. I searched for a while until I found this thread with recommendations: https://old.reddit.com/r/LocalLLaMA/comments/1847qt6/llm_webui_recommendations/ https://old.reddit.com/r/LocalLLaMA/comments/1847qt6/llm_web...
- namanyayg 2y ago
  Thanks for the resource! I've seen some of them and the main issue I had is I don't want to self host anything, and I want to use latest third party models as soon as they are released. Maybe something like what I'm talking about exists already, but I think I'll still try and make my own open source version to fulfill my personal requirements.
  - calum-bird 2y ago
    We (disclosure: founder) do something similar at Trelent[1] but with an emphasis on security. Paid accounts can use OpenAI & Anthropic models, free ones just OpenAI. We have 3.5 sonnet live already. If you want to try it out lmk! Also totally respect building your own open-source :) [1]: https://trelent.com https://trelent.com
    - namanyayg 2y ago
      wow Trelent looks cool, how does ZDR negotiation work exactly? What do you offer to the provider that allows you ZDR?
      - calum-bird 2y ago
        So typically these providers only offer ZDR to "managed" customers, after a lengthy application process. For example, on Azure, "managed" means companies with >$1m, possibly more now, in annual spend. They don't want to waste their time going through this long application process with smaller companies, so we take some of that weight off their shoulders. They get the same revenue at the end of the day, so in many ways it groups smaller companies' LLM spend and sends it straight to their bottom line, and they still get to claim their rolling out AI "responsibly". Once one provider is cracked, the others fall as well, as these AI companies are all competing viciously for customers. Et voila, ZDR across multiple providers for the small(er) companies out there :)

lumenwrites 2y ago

Good history search (including non "main" conversation branches) and convenient conversation management (bookmarking, folders, maybe something smarter) would be great. Also, maybe some convenient way to create message templates? I don't know how I'd implement this, I just know that I often write one long prompt that I reuse multiple times, with multiple minor tweaks/edits, and it'd be amazing to have a convenient tool to manage that. Also, good mobile/tablet support, convenient to use and without bugs (as I happen to spend most of my time writing prompts on my ipad, but that's just me). If you already have a demo - please share a link, I'd be happy to beta test it and maybe become one of the early customers.

namanyayg 2y ago
wow, reading your comment is a great mood boost for me because these are literally the exact features I want from my llm chat experience. It's great to see someone with the exact same problem set. I just followed you on Twitter (I'm @NamanyayG there as well), I'll definitely ping you when I have something to test.
- handfuloflight 2y ago
  [dead]

TobTobXX 2y ago

You might be looking for "LLM Web-UI"s. I searched for a while until I found this thread with recommendations: https://old.reddit.com/r/LocalLLaMA/comments/1847qt6/llm_webui_recommendations/ https://old.reddit.com/r/LocalLLaMA/comments/1847qt6/llm_web...

namanyayg 2y ago
Thanks for the resource! I've seen some of them and the main issue I had is I don't want to self host anything, and I want to use latest third party models as soon as they are released. Maybe something like what I'm talking about exists already, but I think I'll still try and make my own open source version to fulfill my personal requirements.
- calum-bird 2y ago
  We (disclosure: founder) do something similar at Trelent[1] but with an emphasis on security. Paid accounts can use OpenAI & Anthropic models, free ones just OpenAI. We have 3.5 sonnet live already. If you want to try it out lmk! Also totally respect building your own open-source :) [1]: https://trelent.com https://trelent.com
  - namanyayg 2y ago
    wow Trelent looks cool, how does ZDR negotiation work exactly? What do you offer to the provider that allows you ZDR?
    - calum-bird 2y ago
      So typically these providers only offer ZDR to "managed" customers, after a lengthy application process. For example, on Azure, "managed" means companies with >$1m, possibly more now, in annual spend. They don't want to waste their time going through this long application process with smaller companies, so we take some of that weight off their shoulders. They get the same revenue at the end of the day, so in many ways it groups smaller companies' LLM spend and sends it straight to their bottom line, and they still get to claim their rolling out AI "responsibly". Once one provider is cracked, the others fall as well, as these AI companies are all competing viciously for customers. Et voila, ZDR across multiple providers for the small(er) companies out there :)