3 ms·

Given you're running a 3090 24gb, go with Oobabooga/Sillytavern, and don't come here for advice on this stuff, go to https://www.reddit.com/r/LocalLLaMA/ https:

Given you're running a 3090 24gb, go with Oobabooga/Sillytavern, and don't come here for advice on this stuff, go to https://www.reddit.com/r/LocalLLaMA/ https://www.reddit.com/r/LocalLLaMA/, they usually have a "best current local model" thread pinned, and you're less likely to get incorrect advice.

pluc 3y ago
Non-cancer link: https://old.reddit.com/r/LocalLLaMA/ https://old.reddit.com/r/LocalLLaMA/ (though that won't work on VPNs now) (I just got a little emotional because these are things we used to say on reddit and now we say them about reddit. How the mighty have fallen)
- notjulianjaynes 3y ago
  > (though that won't work on VPNs now) Did for me just now, although as of a week or two ago reddit has been blocking many of my attempts to access through a VPN old. or not. Usually need to reconnect about 3 or 4 times before a page will load.
- tommy_axle 3y ago
  VPNs were being blocked over the weekend (I checked after seeing https://news.ycombinator.com/item?id=39883747 https://news.ycombinator.com/item?id=39883747) but as of today within the last hour or so it seems like it works again for both old.redddit.com and regular reddit.com so something changed.
- Cheezmeister 3y ago
  RFC: Is there some straightforward way to use a Pi-hole like setup to 302 redirect `reddit.com` to `old.reddit.com`? I'm sure there are myriad browser extensions that will do it at the DOM level, but that's such a heavy-handed solution, and also lol I'm not putting an extension on the cartesian product of all my browsers on all my machines in the service of dis-enshittifying one once-beloved social network.
  - joenot443 3y ago
    Most straightforward way is to login once and then turn on the "Opt out of redesign" option. Done. I can't imagine any type of Pi-hole setup which would be faster, seems like a "holding a hammer" kind of solution.
    - stuaxo 3y ago
      A nice thing about not being logged in is that it is less addictive as a lot of the front page is less relevant.
    - deleted 3y ago
      [deleted]
  - shepherdjerred 3y ago
    I doubt you could do this at the DNS level. Reddit probably loads static assets from `reddit.com`. You could perhaps intercept the HTTP requests, but that would require decrypting SSL, so you'd have to MITM yourself if you wanted to do it at the network level. You'd replace the DNS request for `reddit.com` to some device that intercepts the traffic. That device would redirect the HTTP request to `old.reddit.com` if applicable; static assets would be routed to `reddit.com`. I don't know how things like HSTS and certificate pinning fit into an idea like this. Doing this at the device level is probably easiest. I've used Redirector [0] in the past and it works well. [0]: https://github.com/einaregilsson/Redirector https://github.com/einaregilsson/Redirector
    - njitram 3y ago
      I found this extension that makes it even easier, and you can configure it to go to either old.reddit.com or new.reddit.com (instead of going to 3rd generation even worse sh.reddit.com) without having to configure Redirector: https://chromewebstore.google.com/detail/ui-changer-for-reddit/bfcldjodnnkndfccfjndmdlppfkmccgh https://chromewebstore.google.com/detail/ui-changer-for-redd...
      - lavezzi 3y ago
- _zoltan_ 3y ago
  [flagged]
- razodactyl 3y ago
  I miss when redditors were really nice xD That was a while ago now. (Or maybe people just make me more nervous these days lol)
aliasaria 3y ago
If you are looking for a downloadable app as an alternative to Oobabooga / Sillytavern, we built https://github.com/transformerlab/transformerlab-app https://github.com/transformerlab/transformerlab-app as a way to easily download, interact, and even train local llms.
- Der_Einzige 3y ago
  The reason everyone loves oobabooga is that it’s made with a maximalist design paradigm. Everyone in the VC world misunderstands why oobabooga is successful and tries to embrace not maximalism. Your example product to benchmark yourself against is blender, if you want to serious compete against oobabooga. You need maximalism
  - CuriouslyC 3y ago
    I wouldn't say maximalism so much, as Oobabooga was designed to emulate Automatic1111, where it's a good enough tool by itself, but then it has a strong extension ecosystem, and it can be utilized through other tools via web service. It's not great at any one thing but it's "good enough" pretty much everywhere.
    - Der_Einzige 3y ago
      I agree with you re:extensions. I know something of the extension ecosystem for it - I built one - https://github.com/Hellisotherpeople/llm_steer-oobabooga https://github.com/Hellisotherpeople/llm_steer-oobabooga Oobabooga the closest thing we have to maximalism. It has exposure for by far the largest number of parameters/settings/backends compared to all others. My main point is that the world yearns for a proper "Photoshop for text" - and no one has even tried to make this (closest is oobabooga). All VC backed competitors are not even close to the mark on what they should be doing here.
brushfoot 3y ago
That doesn't seem to be the case, at least right now. There's no pinned post as far as I can tell, and searching the sub for `best model` returns mostly questions.
- CuriouslyC 3y ago
  It might be because a lot of people use chatbot arena leaderboard for that purpose now, check it out here and look for local models: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
  - polycaster 3y ago
    Looks interesting, but I see no way to filter local models. There are only listings of popular SaaS models. Could be because I’m on mobile?
    - CuriouslyC 3y ago
      The local models should be colored differently and have licenses/etc that should tip you off. You might need to scroll down the list to see one though.
downrightmike 3y ago
Just let reddit die already, the VPN blocking is just bullshit.
fsckboy 3y ago
>Given you're running a 3090 ... "best current local model" thread pinned, and you're less likely to get incorrect advice why do you think my info about the 3090 https://en.wikipedia.org/wiki/IBM_3090 https://en.wikipedia.org/wiki/IBM_3090 is going to be anything less than up-to-date?
- m463 3y ago
  ibm 3090 makes 3 slots seem quite svelte. on the other hand, 24gb of late 80's memory... how many acres of raised floor data center would that take?

pluc 3y ago

Non-cancer link: https://old.reddit.com/r/LocalLLaMA/ https://old.reddit.com/r/LocalLLaMA/ (though that won't work on VPNs now) (I just got a little emotional because these are things we used to say on reddit and now we say them about reddit. How the mighty have fallen)

notjulianjaynes 3y ago
> (though that won't work on VPNs now) Did for me just now, although as of a week or two ago reddit has been blocking many of my attempts to access through a VPN old. or not. Usually need to reconnect about 3 or 4 times before a page will load.
tommy_axle 3y ago
VPNs were being blocked over the weekend (I checked after seeing https://news.ycombinator.com/item?id=39883747 https://news.ycombinator.com/item?id=39883747) but as of today within the last hour or so it seems like it works again for both old.redddit.com and regular reddit.com so something changed.
Cheezmeister 3y ago
RFC: Is there some straightforward way to use a Pi-hole like setup to 302 redirect `reddit.com` to `old.reddit.com`? I'm sure there are myriad browser extensions that will do it at the DOM level, but that's such a heavy-handed solution, and also lol I'm not putting an extension on the cartesian product of all my browsers on all my machines in the service of dis-enshittifying one once-beloved social network.
- joenot443 3y ago
  Most straightforward way is to login once and then turn on the "Opt out of redesign" option. Done. I can't imagine any type of Pi-hole setup which would be faster, seems like a "holding a hammer" kind of solution.
  - stuaxo 3y ago
    A nice thing about not being logged in is that it is less addictive as a lot of the front page is less relevant.
  - deleted 3y ago
    [deleted]
- shepherdjerred 3y ago
  I doubt you could do this at the DNS level. Reddit probably loads static assets from `reddit.com`. You could perhaps intercept the HTTP requests, but that would require decrypting SSL, so you'd have to MITM yourself if you wanted to do it at the network level. You'd replace the DNS request for `reddit.com` to some device that intercepts the traffic. That device would redirect the HTTP request to `old.reddit.com` if applicable; static assets would be routed to `reddit.com`. I don't know how things like HSTS and certificate pinning fit into an idea like this. Doing this at the device level is probably easiest. I've used Redirector [0] in the past and it works well. [0]: https://github.com/einaregilsson/Redirector https://github.com/einaregilsson/Redirector
  - njitram 3y ago
    I found this extension that makes it even easier, and you can configure it to go to either old.reddit.com or new.reddit.com (instead of going to 3rd generation even worse sh.reddit.com) without having to configure Redirector: https://chromewebstore.google.com/detail/ui-changer-for-reddit/bfcldjodnnkndfccfjndmdlppfkmccgh https://chromewebstore.google.com/detail/ui-changer-for-redd...
    - lavezzi 3y ago
_zoltan_ 3y ago
[flagged]
razodactyl 3y ago
I miss when redditors were really nice xD That was a while ago now. (Or maybe people just make me more nervous these days lol)

aliasaria 3y ago

If you are looking for a downloadable app as an alternative to Oobabooga / Sillytavern, we built https://github.com/transformerlab/transformerlab-app https://github.com/transformerlab/transformerlab-app as a way to easily download, interact, and even train local llms.

Der_Einzige 3y ago
The reason everyone loves oobabooga is that it’s made with a maximalist design paradigm. Everyone in the VC world misunderstands why oobabooga is successful and tries to embrace not maximalism. Your example product to benchmark yourself against is blender, if you want to serious compete against oobabooga. You need maximalism
- CuriouslyC 3y ago
  I wouldn't say maximalism so much, as Oobabooga was designed to emulate Automatic1111, where it's a good enough tool by itself, but then it has a strong extension ecosystem, and it can be utilized through other tools via web service. It's not great at any one thing but it's "good enough" pretty much everywhere.
  - Der_Einzige 3y ago
    I agree with you re:extensions. I know something of the extension ecosystem for it - I built one - https://github.com/Hellisotherpeople/llm_steer-oobabooga https://github.com/Hellisotherpeople/llm_steer-oobabooga Oobabooga the closest thing we have to maximalism. It has exposure for by far the largest number of parameters/settings/backends compared to all others. My main point is that the world yearns for a proper "Photoshop for text" - and no one has even tried to make this (closest is oobabooga). All VC backed competitors are not even close to the mark on what they should be doing here.

brushfoot 3y ago

That doesn't seem to be the case, at least right now. There's no pinned post as far as I can tell, and searching the sub for `best model` returns mostly questions.

CuriouslyC 3y ago
It might be because a lot of people use chatbot arena leaderboard for that purpose now, check it out here and look for local models: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
- polycaster 3y ago
  Looks interesting, but I see no way to filter local models. There are only listings of popular SaaS models. Could be because I’m on mobile?
  - CuriouslyC 3y ago
    The local models should be colored differently and have licenses/etc that should tip you off. You might need to scroll down the list to see one though.

downrightmike 3y ago

Just let reddit die already, the VPN blocking is just bullshit.

fsckboy 3y ago

>Given you're running a 3090 ... "best current local model" thread pinned, and you're less likely to get incorrect advice why do you think my info about the 3090 https://en.wikipedia.org/wiki/IBM_3090 https://en.wikipedia.org/wiki/IBM_3090 is going to be anything less than up-to-date?

m463 3y ago
ibm 3090 makes 3 slots seem quite svelte. on the other hand, 24gb of late 80's memory... how many acres of raised floor data center would that take?