4 ms·

Modern LLMs could be like the equivalent of those steam powered toys the Romans had in their time. Steam tech went through a long winter before finally being fu

by deadbabe 11mo ago

Modern LLMs could be like the equivalent of those steam powered toys the Romans had in their time. Steam tech went through a long winter before finally being fully utilized for the Industrial Revolution. We’ll probably never see the true AI revolution in our lifetime, only glimpses of what could be, through toys like LLMs. What we should underscore though, is that even if there is a new AI winter, the world isn’t going back to what it was before AI. This is it, forever. Generations ahead will gaslight themselves into thinking this AI world is better, because who wants to grow up knowing they live in a shitty era full of slop? Don’t believe it.

7thaccount 11mo ago
LLMs are useful tools, but certainly have big limitations. I think we'll continue to see anything be automated that can be automated in a way that reduces head count. So you have the dumb AI as a first line of defense and lay off half the customer service you had before. In the meantime, fewer and fewer jobs (especially entry level), a rising poor class as the middle class is eliminated and a greater wealth gap than ever before. The markets are going to also collapse from this AI bubble. It's just a matter of when.
cardanome 11mo ago
The development of LLM's required access to huge amounts of decent training data. It could very well that the current generation of AI has poisoned the well for any future endeavors of creating AI. You can't trivially filter out the AI slop and humans are less likely to make their handcrafted content freely available for training. In fact violating GPL code to train models on it might be ruled to be illegal as well generally stricter rules on which data you are allowed to use for training. We might have reached a local optimum that is very difficult to escape from. There might be a long, long AI winter ahead of us, for better or worse. > the world isn’t going back to what it was before AI. This is it, forever. I feel this so much. I though my longing for the pre-smartphone days was bad but damn we have lost so much.
- esafak 11mo ago
  That would create the impetus to create a training data marketplace.
  - deadbabe 11mo ago
    Maybe the path to redemption for AI stealing jobs would be if people would have to be rehired en masse to write and produce art, about whatever they want, so that it can be used to train more advanced AI.
rnxrx 11mo ago
The development of steam technology is a great metaphor. The basic understanding of steam as a thing that could yield some kind of mechanical force almost certainly predated even the Romans. That said, it was the synthesis of other technologies with these basic concepts that started yielding really interesting results. Put another way, the advent of industrialized steam power wasn't so much about steam per se, but rather the intersection of a number of factors (steam itself obviously being an important one). This intersection became a lot more likely as the pace of innovation in general began accelerating with the Enlightenment and the ease with which this information could be collected and synthesized. I suspect that the LLM itself may also prove to be less significant than the density of innovation and information of the world it's developed in. It's not a certainty that there's a killer app on the scale of mechanized steam, but the odds of such significant inventions arguably increase as the basics of modern AI become basic knowledge for more and more people.
- orwin 11mo ago
  Its mostly metallurgy. The fact that we became so much better and precise at metallurgy enabled us to make use of steam machines. Of course a lot of stuff helped (Glassmaking, whale oil immediatly come to mind) but mostly, metallurgy.
  - int_19h 11mo ago
    I remember reading an article that argued that it was basically a matter of being path dependent. The earliest steam engines that could do useful work were notoriously large and fuel-inefficient, which is why their first application was for pumps in coal mines - it effectively made the fuel problem moot and similarly their other limitations were not important in that context, while at the same time rising wages in UK made even those inefficient engines more affordable than manual labor. And then their use in that very narrow niche allowed them to be gradually improved to the point where they became suitable for other contexts as well. But if that analogy holds, then LLM use in software development is the "new coal mines" where it will be perfected until it spills over into other areas. We're definitely not at the "Roman stage" anymore.

7thaccount 11mo ago

LLMs are useful tools, but certainly have big limitations. I think we'll continue to see anything be automated that can be automated in a way that reduces head count. So you have the dumb AI as a first line of defense and lay off half the customer service you had before. In the meantime, fewer and fewer jobs (especially entry level), a rising poor class as the middle class is eliminated and a greater wealth gap than ever before. The markets are going to also collapse from this AI bubble. It's just a matter of when.

cardanome 11mo ago

The development of LLM's required access to huge amounts of decent training data. It could very well that the current generation of AI has poisoned the well for any future endeavors of creating AI. You can't trivially filter out the AI slop and humans are less likely to make their handcrafted content freely available for training. In fact violating GPL code to train models on it might be ruled to be illegal as well generally stricter rules on which data you are allowed to use for training. We might have reached a local optimum that is very difficult to escape from. There might be a long, long AI winter ahead of us, for better or worse. > the world isn’t going back to what it was before AI. This is it, forever. I feel this so much. I though my longing for the pre-smartphone days was bad but damn we have lost so much.

esafak 11mo ago
That would create the impetus to create a training data marketplace.
- deadbabe 11mo ago
  Maybe the path to redemption for AI stealing jobs would be if people would have to be rehired en masse to write and produce art, about whatever they want, so that it can be used to train more advanced AI.

rnxrx 11mo ago

The development of steam technology is a great metaphor. The basic understanding of steam as a thing that could yield some kind of mechanical force almost certainly predated even the Romans. That said, it was the synthesis of other technologies with these basic concepts that started yielding really interesting results. Put another way, the advent of industrialized steam power wasn't so much about steam per se, but rather the intersection of a number of factors (steam itself obviously being an important one). This intersection became a lot more likely as the pace of innovation in general began accelerating with the Enlightenment and the ease with which this information could be collected and synthesized. I suspect that the LLM itself may also prove to be less significant than the density of innovation and information of the world it's developed in. It's not a certainty that there's a killer app on the scale of mechanized steam, but the odds of such significant inventions arguably increase as the basics of modern AI become basic knowledge for more and more people.

orwin 11mo ago
Its mostly metallurgy. The fact that we became so much better and precise at metallurgy enabled us to make use of steam machines. Of course a lot of stuff helped (Glassmaking, whale oil immediatly come to mind) but mostly, metallurgy.
- int_19h 11mo ago
  I remember reading an article that argued that it was basically a matter of being path dependent. The earliest steam engines that could do useful work were notoriously large and fuel-inefficient, which is why their first application was for pumps in coal mines - it effectively made the fuel problem moot and similarly their other limitations were not important in that context, while at the same time rising wages in UK made even those inefficient engines more affordable than manual labor. And then their use in that very narrow niche allowed them to be gradually improved to the point where they became suitable for other contexts as well. But if that analogy holds, then LLM use in software development is the "new coal mines" where it will be perfected until it spills over into other areas. We're definitely not at the "Roman stage" anymore.