5 ms·

Since we are on HN here, I can highly recommend open-webui with some OpenAI-compatible provider. I'm running with Deep Infra for more than a year now and am ver

Since we are on HN here, I can highly recommend open-webui with some OpenAI-compatible provider. I'm running with Deep Infra for more than a year now and am very happy. New models are usually available within one or two days after release. Also have some friends who use the service almost daily.

unquietwiki 2y ago
I'm using open-webui at home with a couple of different models. gemma2-9b fits in VRAM on a NV 3060 card + performs nicely.
- mdp2021 2y ago
  > performs nicely Do you have rough indication of token/s ?
- zakki 2y ago
  What is the memory of your NV3060? 8GB?
  - ngvjmfgb 2y ago
    12GB (edit: that is what mine is)
l72 2y ago
I too run openweb-ui locally and use deepinfra.com as my backend. It has been working very well, and I am quite happy with deepinfra's pricing and privacy policy. I have set up the same thing at work for my colleagues, and they find it better than openai for their tasks.
- jychang 2y ago
  Yeah, openweb-ui is the best frontend for API queries. Everything seems to work well. I've tried LibreChat before, but the app is terrible at generating titles for chats instead of leaving it as "New Chat". Also it lacks a working Code Interpreter.
deleted 2y ago
[deleted]
totetsu 2y ago
And it’s quite easy to set up a Cloudflare tunnel to make your open-webui instance accessible online too just you
- simonw 2y ago
  ... or a TailScale network. I've been leaving open-webui running on my laptop on my desk and then going out into the word and accessing it from my phone via TailScale, works great.
  - wkat4242 2y ago
    Yeah this sounds like the more secure option, you don't want to be dependent on a single flaw in a web service
  - totetsu 2y ago
    I would use tail scale. But I specifically want to use open web-ui from a place I can’t install a Tailscale client
    - fragmede 2y ago
      where's that?
wkat4242 2y ago
Yeah OpenWebUI is great with local models too. I love it. You can even do a combo, send the same prompt to local and cloud and even various providers and compare the results.
eurekin 2y ago
I've tried using it, but it's browser tab seems to peg one core to 100% after some time. Anyone else experienced it?
indigodaddy 2y ago
Can open-webui update code on your local computer ala cursor etc?
- cess11 2y ago
  It has a module system so maybe it can but it seems more people are using Aider or Continue for that. There's a bit of stitching things together regardless of whether you show your project to some SaaS or run local models but if you can manage a Linux system it'll be easy. Personally I heavily dislike the experience though, so I might not be the best one to answer.

unquietwiki 2y ago

I'm using open-webui at home with a couple of different models. gemma2-9b fits in VRAM on a NV 3060 card + performs nicely.

mdp2021 2y ago
> performs nicely Do you have rough indication of token/s ?
zakki 2y ago
What is the memory of your NV3060? 8GB?
- ngvjmfgb 2y ago
  12GB (edit: that is what mine is)

l72 2y ago

I too run openweb-ui locally and use deepinfra.com as my backend. It has been working very well, and I am quite happy with deepinfra's pricing and privacy policy. I have set up the same thing at work for my colleagues, and they find it better than openai for their tasks.

jychang 2y ago
Yeah, openweb-ui is the best frontend for API queries. Everything seems to work well. I've tried LibreChat before, but the app is terrible at generating titles for chats instead of leaving it as "New Chat". Also it lacks a working Code Interpreter.

deleted 2y ago

[deleted]

totetsu 2y ago

And it’s quite easy to set up a Cloudflare tunnel to make your open-webui instance accessible online too just you

simonw 2y ago
... or a TailScale network. I've been leaving open-webui running on my laptop on my desk and then going out into the word and accessing it from my phone via TailScale, works great.
- wkat4242 2y ago
  Yeah this sounds like the more secure option, you don't want to be dependent on a single flaw in a web service
- totetsu 2y ago
  I would use tail scale. But I specifically want to use open web-ui from a place I can’t install a Tailscale client
  - fragmede 2y ago
    where's that?

wkat4242 2y ago

Yeah OpenWebUI is great with local models too. I love it. You can even do a combo, send the same prompt to local and cloud and even various providers and compare the results.

eurekin 2y ago

I've tried using it, but it's browser tab seems to peg one core to 100% after some time. Anyone else experienced it?

indigodaddy 2y ago

Can open-webui update code on your local computer ala cursor etc?

cess11 2y ago
It has a module system so maybe it can but it seems more people are using Aider or Continue for that. There's a bit of stitching things together regardless of whether you show your project to some SaaS or run local models but if you can manage a Linux system it'll be easy. Personally I heavily dislike the experience though, so I might not be the best one to answer.