4 ms·

Don't we already have small models highly distributed?

by onlyrealcuzzo 1y ago

saltysalt 1y ago
We do, but the vast majority of users interact with centralised models from Open AI, Google Gemini, Grok...
- onlyrealcuzzo 1y ago
  I'm not sure we can look forward to self-hosted models ever being mainstream. Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. I don't know how free versions are going to be smaller, run on commodity hardware, take up trivial space and ram etc, AND be substantially better
  - saltysalt 1y ago
    You make a fair point, I'm just hoping this will happen, but not confident either to be frank.
  - ryanianian 1y ago
    The "enshittification" hasn't happened yet. They'll add ads and other gross stuff to the free or cheap tiers. Some will continue to use it, but there will be an opportunity for self-hosted models to emerge.
  - deleted 1y ago
    [deleted]
  - oceanplexian 1y ago
    > I'm not sure we can look forward to self-hosted models ever being mainstream. If you are using an Apple product chances are you are already using self-hosted models for things like writing tools and don't even know it.
  - o11c 1y ago
    > Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. No, you usually only change your habit when the tools you are already using are changed without consulting you, and the statistics are then used to lie.
- raincole 1y ago
  Because small models are just not that good.
- positron26 1y ago
  The vast majority won't switch until there's a 10x use case. We know they are coming. Why bother hopping?
godshatter 1y ago
I use gpt4all and have downloaded some models. It's not that hard, and doesn't take a huge system. It's pretty easy if you use models on their list, it's a bit more work to find the right chat interactive script (forgot what they are called) for your class of model if you downloaded one that's not on the list, but not that hard. I have 8GB VRAM, and it's slow but they work. I like that it's local and private, but then I don't use them for much other than as an oddity that is fun to interact with.

Don't we already have small models highly distributed?

saltysalt 1y ago

We do, but the vast majority of users interact with centralised models from Open AI, Google Gemini, Grok...

onlyrealcuzzo 1y ago
I'm not sure we can look forward to self-hosted models ever being mainstream. Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. I don't know how free versions are going to be smaller, run on commodity hardware, take up trivial space and ram etc, AND be substantially better
- saltysalt 1y ago
  You make a fair point, I'm just hoping this will happen, but not confident either to be frank.
- ryanianian 1y ago
  The "enshittification" hasn't happened yet. They'll add ads and other gross stuff to the free or cheap tiers. Some will continue to use it, but there will be an opportunity for self-hosted models to emerge.
- deleted 1y ago
  [deleted]
- oceanplexian 1y ago
  > I'm not sure we can look forward to self-hosted models ever being mainstream. If you are using an Apple product chances are you are already using self-hosted models for things like writing tools and don't even know it.
- o11c 1y ago
  > Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. No, you usually only change your habit when the tools you are already using are changed without consulting you, and the statistics are then used to lie.
raincole 1y ago
Because small models are just not that good.
positron26 1y ago
The vast majority won't switch until there's a 10x use case. We know they are coming. Why bother hopping?

godshatter 1y ago

I use gpt4all and have downloaded some models. It's not that hard, and doesn't take a huge system. It's pretty easy if you use models on their list, it's a bit more work to find the right chat interactive script (forgot what they are called) for your class of model if you downloaded one that's not on the list, but not that hard. I have 8GB VRAM, and it's slow but they work. I like that it's local and private, but then I don't use them for much other than as an oddity that is fun to interact with.