2 ms·

>and I suspect a decade into the future, entry level hardware will be running better models than high-end graphics cards can run now, as entry-level hardware ge

>and I suspect a decade into the future, entry level hardware will be running better models than high-end graphics cards can run now, as entry-level hardware gets better and models get more efficient. I doubt it. The play seems to be: lock what was once commodity compute up into datacenters depriving us regular folk of it, then sell it back to us on subscription. Even if my #NeverSubscribe movement succeeds, all that misdirected hardware [into datacenters] won't likely be practical for home use.

curious_cat_163 1mo ago
> The play seems to be: lock what was once commodity compute up into datacenters depriving us regular folk of it, then sell it back to us on subscription. I had not considered that as a possibility. It is somewhat dark and unlikely, imho, but a real possibility. I am more inclined to believe that the fab capacity will grow over time and "commodity" compute will be available to us all again. However, I can also imagine going back to the 60s era "hyper verticalized" mainframes, in which case, the frontier labs might not just be producing models, but chips and an ecosystem around themselves and their suppliers/customers.
dlcarrier 1mo ago
I had an internship at HP in the early 2000's, and my computer was a terminal that connected to an X server in a data center. Doing everything as a service is not a new idea, and suppliers have been trying to push it for decades. The only time it really succeeds is when regulations force it. I can store files on my own computer or a home NAS just fine, but if I start a medical practice, I have pretty much no hope of being HIPAA compliant without signing up for a data hosting service. The same goes for tax preparation, banking, and several other fields. It may very well be the case in China, Europe, or Australia that regulatory restrictions force users into models-as-a-service, but in the US, regulations censoring models, even if they frame that censorship as a safety measure, won't pass constitutional muster.
queenkjuul 1mo ago
But the Chinese models exist, are competitive, and have locally useful quantized versions. If you're actually willing to pick up old DC inference hardware on the cheap you can already do a lot at home. The new hardware not so much admittedly but if the Chinese models keep getting better and running on less hardware... I don't see a reason to be quite so pessimistic.

curious_cat_163 1mo ago

> The play seems to be: lock what was once commodity compute up into datacenters depriving us regular folk of it, then sell it back to us on subscription. I had not considered that as a possibility. It is somewhat dark and unlikely, imho, but a real possibility. I am more inclined to believe that the fab capacity will grow over time and "commodity" compute will be available to us all again. However, I can also imagine going back to the 60s era "hyper verticalized" mainframes, in which case, the frontier labs might not just be producing models, but chips and an ecosystem around themselves and their suppliers/customers.

dlcarrier 1mo ago

I had an internship at HP in the early 2000's, and my computer was a terminal that connected to an X server in a data center. Doing everything as a service is not a new idea, and suppliers have been trying to push it for decades. The only time it really succeeds is when regulations force it. I can store files on my own computer or a home NAS just fine, but if I start a medical practice, I have pretty much no hope of being HIPAA compliant without signing up for a data hosting service. The same goes for tax preparation, banking, and several other fields. It may very well be the case in China, Europe, or Australia that regulatory restrictions force users into models-as-a-service, but in the US, regulations censoring models, even if they frame that censorship as a safety measure, won't pass constitutional muster.

queenkjuul 1mo ago

But the Chinese models exist, are competitive, and have locally useful quantized versions. If you're actually willing to pick up old DC inference hardware on the cheap you can already do a lot at home. The new hardware not so much admittedly but if the Chinese models keep getting better and running on less hardware... I don't see a reason to be quite so pessimistic.