12 ms·

Not sure the dial-up analogy fits, instead I tend to think we are in the mainframe period of AI, with large centralised computing models that are so big and exp

Not sure the dial-up analogy fits, instead I tend to think we are in the mainframe period of AI, with large centralised computing models that are so big and expensive to host, only a few corporations can afford to do so. We rent a computing timeshare from them (tokens = punch cards). I look forward to the "personal computing" period, with small models distributed everywhere...

sixtyj 1y ago
Dial-up + mainframe. Mainframe from POV as silos, dial-up internet as the speed we have now when looking back to 2025 in 2035.
runarberg 1y ago
I actually think we are much closer to the sneaker era of shoes, or the monorail era of public transit.
onlyrealcuzzo 1y ago
Don't we already have small models highly distributed?
- saltysalt 1y ago
  We do, but the vast majority of users interact with centralised models from Open AI, Google Gemini, Grok...
  - onlyrealcuzzo 1y ago
    I'm not sure we can look forward to self-hosted models ever being mainstream. Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. I don't know how free versions are going to be smaller, run on commodity hardware, take up trivial space and ram etc, AND be substantially better
    - saltysalt 1y ago
      You make a fair point, I'm just hoping this will happen, but not confident either to be frank.
    - ryanianian 1y ago
      The "enshittification" hasn't happened yet. They'll add ads and other gross stuff to the free or cheap tiers. Some will continue to use it, but there will be an opportunity for self-hosted models to emerge.
    - deleted 1y ago
      [deleted]
    - oceanplexian 1y ago
      > I'm not sure we can look forward to self-hosted models ever being mainstream. If you are using an Apple product chances are you are already using self-hosted models for things like writing tools and don't even know it.
    - o11c 1y ago
      > Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. No, you usually only change your habit when the tools you are already using are changed without consulting you, and the statistics are then used to lie.
  - raincole 1y ago
    Because small models are just not that good.
  - positron26 1y ago
    The vast majority won't switch until there's a 10x use case. We know they are coming. Why bother hopping?
- godshatter 1y ago
  I use gpt4all and have downloaded some models. It's not that hard, and doesn't take a huge system. It's pretty easy if you use models on their list, it's a bit more work to find the right chat interactive script (forgot what they are called) for your class of model if you downloaded one that's not on the list, but not that hard. I have 8GB VRAM, and it's slow but they work. I like that it's local and private, but then I don't use them for much other than as an oddity that is fun to interact with.
paxys 1y ago
Why would companies sell you the golden goose when they can instead sell you an egg every day?
- saltysalt 1y ago
  Exactly! It's a rent-seeking model.
  - echelon 1y ago
    > I look forward to the "personal computing" period, with small models distributed everywhere... Like the web, which worked out great? Our Internet is largely centralized platforms. Built on technology controlled by trillion dollar titans. Google somehow got the lion share of browser usage and is now dictating the direction of web tech, including the removal of adblock. The URL bar defaults to Google search, where the top results are paid ads. Your typical everyday person uses their default, locked down iPhone or Android to consume Google or Apple platform products. They then communicate with their friends over Meta platforms, Reddit, or Discord. The decentralized web could never outrun money. It's difficult to out-engineer hundreds of thousands of the most talented, most highly paid engineers that are working to create these silos.
    - saltysalt 1y ago
      I agree man, it's depressing.
    - deleted 1y ago
      [deleted]
    - NemoNobody 1y ago
      Ok, so Brave Browser exists - if you download, you will see 0 ads on the internet, I've never really seen ads on the internet - even in the before brave times. Fr tho, no ads - I'm not making money off them, I've got no invite code for you, I'm a human - I just don't get it. I've probably told 500 people about Brave, I don't know any that ever tried it. I don't ever know what to say. You're not wrong, as long as you never try to do something else.
cyanydeez 1y ago
I think we are in the dotcom boom era where investment is circular and the cash investments all depend on the idea that growth is infinite. Just a bunch of billionaires jockeying for not being poor.
gowld 1y ago
We are also in the mainframe period of computing, with large centralised cloud services.
chemotaxis 1y ago
> I look forward to the "personal computing" period, with small models distributed everywhere... One could argue that this period was just a brief fluke. Personal computers really took off only in the 1990s, web 2.0 happened in the mid-2000s. Now, for the average person, 95%+ of screen time boils down to using the computer as a dumb terminal to access centralized services "in the cloud".
- btown 1y ago
  Even the most popular games (with few exceptions) present as relatively dumb terminals that need constant connectivity to sync every activity to a mainframe - not necessarily because it's an MMO or multiplayer game, but because it's the industry standard way to ensure fairness. And by fairness, of course, I mean the optimization of enforcing "grindiness" as a mechanism to sell lootboxes and premium subscriptions. And AI just further normalizes the need for connectivity; cloud models are likely to improve faster than local models, for both technical and business reasons. They've got the premium-subscriptions model down. I shudder to think what happens when OpenAI begins hiring/subsuming-the-knowledge-of "revenue optimization analysts" from the AAA gaming world as a way to boost revenue. But hey, at least you still need humans, at some level, if your paperclip optimizer is told to find ways to get humans to spend money on "a sense of pride and accomplishment." [0] We do not live in a utopia. [0] https://www.guinnessworldrecords.com/world-records/503152-most-downvoted-comment-on-reddit https://www.guinnessworldrecords.com/world-records/503152-mo... - https://www.reddit.com/r/StarWarsBattlefront/comments/7cff0b/seriously_i_paid_80_to_have_vader_locked/dppum98/?st=JH2MUORV&sh=5997c5a5 https://www.reddit.com/r/StarWarsBattlefront/comments/7cff0b...
  - throw23920 1y ago
    I imagine there are plenty of indie single-player games that work just fine offline. You lose cloud saves and achievements, but everything else still works.
- pksebben 1y ago
  That 'average' is doing a lot of work to obfuscate the landscape. Open source continues to grow (indicating a robust ecosystem of individuals who use their computers for local work) and more importantly, the 'average' looks like it does not necessarily due to a reduction in local use, but to an explosion of users that did not previously exist (mobile first, SAAS customers, etc.) The thing we do need to be careful about is regulatory capture. We could very well end up with nothing but monolithic centralized systems simply because it's made illegal to distribute, use, and share open models. They hinted quite strongly that they wanted to do this with deepseek. There may even be a case to be made that at some point in the future, small local models will outperform monoliths - if distributed training becomes cheap enough, or if we find an alternative to backprop that allows models to learn as they infer (like a more developed forward-forward or something like it), we may see models that do better simply because they aren't a large centralized organism behind a walled garden. I'll grant that this is a fairly polyanna take and represents the best possible outcome but it's not outlandishly fantastic - and there is good reason to believe that any system based on a robust decentralized architecture would be more resilient to problems like platform enshittification and overdeveloped censorship. At the end of the day, it's not important what the 'average' user is doing, so long as there are enough non-average users pushing the ball forward on the important stuff.
dzonga 1y ago
this -- chips are getting fast enough both arm n x86. unified memory architecture means we can get more ram on devices at faster throughput. we're already seeing local models - just that their capability is limited by ram.
8ytecoder 1y ago
Funny you would pick this analogy. I feel like we’re back in the mainframe era. A lot of software can’t operate without an internet connection. Even if in practice they execute some of the code on your device, a lot of the data and the heavyweight processing is already happening on the server. Even basic services designed from the ground up to be distributed and local first - like email (“downloading”) - are used in this fashion - like gmail. Maps apps added offline support years after they launched and still cripple the search. Even git has GitHub sitting in the middle and most people don’t or can’t use git any other way. SaaS, Electron, …etc. have brought us back to the mainframe era.
- thewebguyd 1y ago
  It's always struck me as living in some sort of bizaro world. We now have these super powerful personal computers, both handheld (phones) and laptops (My M4 Pro smokes even some desktop class processors) and yet I use all this powerful compute hardware to...be a dumb terminal to someone else's computer. I had always hoped we'd do more locally on-device (and with native apps, not running 100 instances of chromium for various electron apps). But, it's hard to extract rent that way I suppose.
  - ryandrake 1y ago
    I don't even understand why computer and phone manufacturers even try to make their devices faster anymore, since for most computing tasks, the bottleneck is all the data that needs to be transferred to and from the modern version of the mainframe.
    - charcircuit 1y ago
      Consumers care about battery life.
      - galaxyLogic 1y ago
        And providers count their capacity in Giga-watts.
EGreg 1y ago
I actually don’t look forward to this period. I have always been for open source software and distributism — until AI. Because if there’s one thing worse than governments having nuclear weapons, it’s everyone having them. It would be chaos. And with physical drones and robots coming, it woukd be even worse. Think “shitcoins and memecoins” but unlike those, you don’t just lose the money you put in and you can’t opt out. They’d affect everyone, and you can never escape the chaos ever again. They’d be posting around the whole Internet (including here, YouTube deepfakes, extortion, annoyance, constantly trying to rewrite history, get published, reputational destruction at scale etc etc), and constant armies of bots fighting. A dark forest. And if AI can pay for its own propagation via decentralized hosting and inference, then the chance of a runaway advanced persistent threat compounds. It just takes a few bad apples, or even practical jokers, to unleash crazy stuff. And it will never be shut down, just build and build like some kind of kessler syndrome. And I’m talking about with just CURRENT AI agent and drone technology.
graeme 1y ago
We have a ton of good, small models. The issues are: 1. Most people don't have machines that can run even midsized local models well 2. The local models are nearly as good as the frontier models for a lot of use cases 3. There are technical hurdles to running local models that will block 99% of people. Even if the steps are: download LM Studio and download a model Maybe local models will get so good that they cover 99% of normal user use cases and it'll be like using your phone/computer to edit a photo. But you'll still need something to make it automatic enough that regular people use it by default. That said, anyone reading this is almost certainly technical enough to run a local model. I would highly recommend trying some. Very neat to know it's entirely run from your machine and seeing what it can do. LM Studio is the most brainless way to dip your toes in.
- loyalcinnamon 1y ago
  As the hype is dying down it's becoming a little bit clearer that AI isn't like blockchain and might be actually useful (for non generative purposes at least) I'm curious what counts as a midsize model; 4B, 8B, or something larger/smaller? What models would you recommend? I have 12GB of vram so anything larger than 8B might be really slow, but i am not sure
  - DSingularity 1y ago
    It can depend on your use case. Are you editing a large code base and will thus make lots of completion requests with large contexts?
  - riskable 1y ago
    My take: Large: Requires >128GB VRAM Medium: 32-128GB VRAM Small: 16GB VRAM Micro: Runs on a microcontroller or GPUs with just 4GB of VRAM There's really nothing worthwhile for general use cases that runs in under 16GB (from my testing) except a grammar-checking model that I can't remember the name of at the moment. gpt-oss:20b runs on 16GB of VRAM and it's actually quite good (for coding, at least)! Especially with Python. Prediction: The day that your average gaming PC comes with 128GB of VRAM is the day developers will stop bothering with cloud-based AI services. gpt-oss:120b is nearly as good as gpt5 and we're still at the beginning of the AI revolution.
jijji 1y ago
ollama and other peojects already make this possible
raincole 1y ago
> "personal computing" period The period when you couldn't use Linux as your main OS because your organization asked for .doc files? No thanks.
giancarlostoro 1y ago
I mean, people can self-host plenty off of a 5090, heck even Macs with enough RAM can run larger models that I can't run on a 5090.
consumer451 1y ago
I like to think of it more broadly, and that we are currently in the era of the first automobile. [0] LLMs are the internal combustion engine, and chatbot UIs are at the "horseless carriage" phase. My personal theory is that even if models stopped making major advancements, we would find cheaper and more useful ways to use them. In the end, our current implementations will look like the automobile pictured below. [0] https://group.mercedes-benz.com/company/tradition/company-history/1885-1886.html https://group.mercedes-benz.com/company/tradition/company-hi...
falcor84 1y ago
I'm not a big google fan, but I really like the "Google AI Edge Gallery" android app [0]. In particular, I've been chatting with the "Gemma-3n-E2B-it" model when I don't have an internet connection, and it's really decent! [0] https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery https://play.google.com/store/apps/details?id=com.google.ai....
js8 1y ago
Mainframes still exist, and they actually make a lot of sense from physics perspective. It's good idea to run transactions in a big machine rather than distributed, the latter is less energy efficient. I think the misconception is that things cannot be overpriced for reasons other than inefficiency.
supportengineer 1y ago
Imagine small models on a cheap chip that can be added to anything (alarm clock, electric toothbrush, car keys...)

sixtyj 1y ago

Dial-up + mainframe. Mainframe from POV as silos, dial-up internet as the speed we have now when looking back to 2025 in 2035.

runarberg 1y ago

I actually think we are much closer to the sneaker era of shoes, or the monorail era of public transit.

onlyrealcuzzo 1y ago

Don't we already have small models highly distributed?

saltysalt 1y ago
We do, but the vast majority of users interact with centralised models from Open AI, Google Gemini, Grok...
- onlyrealcuzzo 1y ago
  I'm not sure we can look forward to self-hosted models ever being mainstream. Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. I don't know how free versions are going to be smaller, run on commodity hardware, take up trivial space and ram etc, AND be substantially better
  - saltysalt 1y ago
    You make a fair point, I'm just hoping this will happen, but not confident either to be frank.
  - ryanianian 1y ago
    The "enshittification" hasn't happened yet. They'll add ads and other gross stuff to the free or cheap tiers. Some will continue to use it, but there will be an opportunity for self-hosted models to emerge.
  - deleted 1y ago
    [deleted]
  - oceanplexian 1y ago
    > I'm not sure we can look forward to self-hosted models ever being mainstream. If you are using an Apple product chances are you are already using self-hosted models for things like writing tools and don't even know it.
  - o11c 1y ago
    > Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better. No, you usually only change your habit when the tools you are already using are changed without consulting you, and the statistics are then used to lie.
- raincole 1y ago
  Because small models are just not that good.
- positron26 1y ago
  The vast majority won't switch until there's a 10x use case. We know they are coming. Why bother hopping?
godshatter 1y ago
I use gpt4all and have downloaded some models. It's not that hard, and doesn't take a huge system. It's pretty easy if you use models on their list, it's a bit more work to find the right chat interactive script (forgot what they are called) for your class of model if you downloaded one that's not on the list, but not that hard. I have 8GB VRAM, and it's slow but they work. I like that it's local and private, but then I don't use them for much other than as an oddity that is fun to interact with.

paxys 1y ago

Why would companies sell you the golden goose when they can instead sell you an egg every day?

saltysalt 1y ago
Exactly! It's a rent-seeking model.
- echelon 1y ago
  > I look forward to the "personal computing" period, with small models distributed everywhere... Like the web, which worked out great? Our Internet is largely centralized platforms. Built on technology controlled by trillion dollar titans. Google somehow got the lion share of browser usage and is now dictating the direction of web tech, including the removal of adblock. The URL bar defaults to Google search, where the top results are paid ads. Your typical everyday person uses their default, locked down iPhone or Android to consume Google or Apple platform products. They then communicate with their friends over Meta platforms, Reddit, or Discord. The decentralized web could never outrun money. It's difficult to out-engineer hundreds of thousands of the most talented, most highly paid engineers that are working to create these silos.
  - saltysalt 1y ago
    I agree man, it's depressing.
  - deleted 1y ago
    [deleted]
  - NemoNobody 1y ago
    Ok, so Brave Browser exists - if you download, you will see 0 ads on the internet, I've never really seen ads on the internet - even in the before brave times. Fr tho, no ads - I'm not making money off them, I've got no invite code for you, I'm a human - I just don't get it. I've probably told 500 people about Brave, I don't know any that ever tried it. I don't ever know what to say. You're not wrong, as long as you never try to do something else.

cyanydeez 1y ago

I think we are in the dotcom boom era where investment is circular and the cash investments all depend on the idea that growth is infinite. Just a bunch of billionaires jockeying for not being poor.

gowld 1y ago

We are also in the mainframe period of computing, with large centralised cloud services.

chemotaxis 1y ago

> I look forward to the "personal computing" period, with small models distributed everywhere... One could argue that this period was just a brief fluke. Personal computers really took off only in the 1990s, web 2.0 happened in the mid-2000s. Now, for the average person, 95%+ of screen time boils down to using the computer as a dumb terminal to access centralized services "in the cloud".

btown 1y ago
Even the most popular games (with few exceptions) present as relatively dumb terminals that need constant connectivity to sync every activity to a mainframe - not necessarily because it's an MMO or multiplayer game, but because it's the industry standard way to ensure fairness. And by fairness, of course, I mean the optimization of enforcing "grindiness" as a mechanism to sell lootboxes and premium subscriptions. And AI just further normalizes the need for connectivity; cloud models are likely to improve faster than local models, for both technical and business reasons. They've got the premium-subscriptions model down. I shudder to think what happens when OpenAI begins hiring/subsuming-the-knowledge-of "revenue optimization analysts" from the AAA gaming world as a way to boost revenue. But hey, at least you still need humans, at some level, if your paperclip optimizer is told to find ways to get humans to spend money on "a sense of pride and accomplishment." [0] We do not live in a utopia. [0] https://www.guinnessworldrecords.com/world-records/503152-most-downvoted-comment-on-reddit https://www.guinnessworldrecords.com/world-records/503152-mo... - https://www.reddit.com/r/StarWarsBattlefront/comments/7cff0b/seriously_i_paid_80_to_have_vader_locked/dppum98/?st=JH2MUORV&sh=5997c5a5 https://www.reddit.com/r/StarWarsBattlefront/comments/7cff0b...
- throw23920 1y ago
  I imagine there are plenty of indie single-player games that work just fine offline. You lose cloud saves and achievements, but everything else still works.
pksebben 1y ago
That 'average' is doing a lot of work to obfuscate the landscape. Open source continues to grow (indicating a robust ecosystem of individuals who use their computers for local work) and more importantly, the 'average' looks like it does not necessarily due to a reduction in local use, but to an explosion of users that did not previously exist (mobile first, SAAS customers, etc.) The thing we do need to be careful about is regulatory capture. We could very well end up with nothing but monolithic centralized systems simply because it's made illegal to distribute, use, and share open models. They hinted quite strongly that they wanted to do this with deepseek. There may even be a case to be made that at some point in the future, small local models will outperform monoliths - if distributed training becomes cheap enough, or if we find an alternative to backprop that allows models to learn as they infer (like a more developed forward-forward or something like it), we may see models that do better simply because they aren't a large centralized organism behind a walled garden. I'll grant that this is a fairly polyanna take and represents the best possible outcome but it's not outlandishly fantastic - and there is good reason to believe that any system based on a robust decentralized architecture would be more resilient to problems like platform enshittification and overdeveloped censorship. At the end of the day, it's not important what the 'average' user is doing, so long as there are enough non-average users pushing the ball forward on the important stuff.

dzonga 1y ago

this -- chips are getting fast enough both arm n x86. unified memory architecture means we can get more ram on devices at faster throughput. we're already seeing local models - just that their capability is limited by ram.

8ytecoder 1y ago

Funny you would pick this analogy. I feel like we’re back in the mainframe era. A lot of software can’t operate without an internet connection. Even if in practice they execute some of the code on your device, a lot of the data and the heavyweight processing is already happening on the server. Even basic services designed from the ground up to be distributed and local first - like email (“downloading”) - are used in this fashion - like gmail. Maps apps added offline support years after they launched and still cripple the search. Even git has GitHub sitting in the middle and most people don’t or can’t use git any other way. SaaS, Electron, …etc. have brought us back to the mainframe era.

thewebguyd 1y ago
It's always struck me as living in some sort of bizaro world. We now have these super powerful personal computers, both handheld (phones) and laptops (My M4 Pro smokes even some desktop class processors) and yet I use all this powerful compute hardware to...be a dumb terminal to someone else's computer. I had always hoped we'd do more locally on-device (and with native apps, not running 100 instances of chromium for various electron apps). But, it's hard to extract rent that way I suppose.
- ryandrake 1y ago
  I don't even understand why computer and phone manufacturers even try to make their devices faster anymore, since for most computing tasks, the bottleneck is all the data that needs to be transferred to and from the modern version of the mainframe.
  - charcircuit 1y ago
    Consumers care about battery life.
    - galaxyLogic 1y ago
      And providers count their capacity in Giga-watts.

EGreg 1y ago

I actually don’t look forward to this period. I have always been for open source software and distributism — until AI. Because if there’s one thing worse than governments having nuclear weapons, it’s everyone having them. It would be chaos. And with physical drones and robots coming, it woukd be even worse. Think “shitcoins and memecoins” but unlike those, you don’t just lose the money you put in and you can’t opt out. They’d affect everyone, and you can never escape the chaos ever again. They’d be posting around the whole Internet (including here, YouTube deepfakes, extortion, annoyance, constantly trying to rewrite history, get published, reputational destruction at scale etc etc), and constant armies of bots fighting. A dark forest. And if AI can pay for its own propagation via decentralized hosting and inference, then the chance of a runaway advanced persistent threat compounds. It just takes a few bad apples, or even practical jokers, to unleash crazy stuff. And it will never be shut down, just build and build like some kind of kessler syndrome. And I’m talking about with just CURRENT AI agent and drone technology.

graeme 1y ago

We have a ton of good, small models. The issues are: 1. Most people don't have machines that can run even midsized local models well 2. The local models are nearly as good as the frontier models for a lot of use cases 3. There are technical hurdles to running local models that will block 99% of people. Even if the steps are: download LM Studio and download a model Maybe local models will get so good that they cover 99% of normal user use cases and it'll be like using your phone/computer to edit a photo. But you'll still need something to make it automatic enough that regular people use it by default. That said, anyone reading this is almost certainly technical enough to run a local model. I would highly recommend trying some. Very neat to know it's entirely run from your machine and seeing what it can do. LM Studio is the most brainless way to dip your toes in.

loyalcinnamon 1y ago
As the hype is dying down it's becoming a little bit clearer that AI isn't like blockchain and might be actually useful (for non generative purposes at least) I'm curious what counts as a midsize model; 4B, 8B, or something larger/smaller? What models would you recommend? I have 12GB of vram so anything larger than 8B might be really slow, but i am not sure
- DSingularity 1y ago
  It can depend on your use case. Are you editing a large code base and will thus make lots of completion requests with large contexts?
- riskable 1y ago
  My take: Large: Requires >128GB VRAM Medium: 32-128GB VRAM Small: 16GB VRAM Micro: Runs on a microcontroller or GPUs with just 4GB of VRAM There's really nothing worthwhile for general use cases that runs in under 16GB (from my testing) except a grammar-checking model that I can't remember the name of at the moment. gpt-oss:20b runs on 16GB of VRAM and it's actually quite good (for coding, at least)! Especially with Python. Prediction: The day that your average gaming PC comes with 128GB of VRAM is the day developers will stop bothering with cloud-based AI services. gpt-oss:120b is nearly as good as gpt5 and we're still at the beginning of the AI revolution.

jijji 1y ago

ollama and other peojects already make this possible

raincole 1y ago

> "personal computing" period The period when you couldn't use Linux as your main OS because your organization asked for .doc files? No thanks.

giancarlostoro 1y ago

I mean, people can self-host plenty off of a 5090, heck even Macs with enough RAM can run larger models that I can't run on a 5090.

consumer451 1y ago

I like to think of it more broadly, and that we are currently in the era of the first automobile. [0] LLMs are the internal combustion engine, and chatbot UIs are at the "horseless carriage" phase. My personal theory is that even if models stopped making major advancements, we would find cheaper and more useful ways to use them. In the end, our current implementations will look like the automobile pictured below. [0] https://group.mercedes-benz.com/company/tradition/company-history/1885-1886.html https://group.mercedes-benz.com/company/tradition/company-hi...

falcor84 1y ago

I'm not a big google fan, but I really like the "Google AI Edge Gallery" android app [0]. In particular, I've been chatting with the "Gemma-3n-E2B-it" model when I don't have an internet connection, and it's really decent! [0] https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery https://play.google.com/store/apps/details?id=com.google.ai....

js8 1y ago

Mainframes still exist, and they actually make a lot of sense from physics perspective. It's good idea to run transactions in a big machine rather than distributed, the latter is less energy efficient. I think the misconception is that things cannot be overpriced for reasons other than inefficiency.

supportengineer 1y ago

Imagine small models on a cheap chip that can be added to anything (alarm clock, electric toothbrush, car keys...)