3 ms·

I think it’s clear that LLMs cannot be the end state of this technology, and we will need systems that can reason and develop hypotheses and test them internall

I think it’s clear that LLMs cannot be the end state of this technology, and we will need systems that can reason and develop hypotheses and test them internally. These systems may benefit from more curated datasets (such as those collected before the bullshit wave began) along with real world interaction data from YouTube and robotics. Such systems could eventually be used to rank web pages for their bullshit level, which of course would present a risk of censorship but when used properly could lead to insightful data handling. It’s just really clear that a giant text averaging machine can only go so far, and while we do see some higher level emergent properties, we’re going to have to move beyond the current state of the art in the next decade, and such future systems may be much less affected by the internet’s bullshit. Even without the bullshit singularity I think such measures would be a necessity and we would see them developed soon.

pton_xd 3y ago
> It’s just really clear that a giant text averaging machine can only go so far It's not really a text averaging machine, it's a pattern matching machine. Right now the "depth" of the patterns it can match can only go so far, but in a few years with more advances in chips and memory the depth is going to increase and the patterns it can match will fan out accordingly.
- vrighter 3y ago
  it is a statistical model. If everyone is saying X and is wrong, and one guy says Y and is right, the llm will bit out X. Because that is the most probable thing in the dataset. It literally is a text averaging machine
visarga 3y ago
Not just real world data from videos. AI models need feedback from many sources: humans, code execution, web search, simulations, games, robotics, math verification, or from actual experiments in the real world. All of these are environments that can take the output of a model and do some processing and return feedback. The model can learn and search for solutions, creating its own training data as a RL agent. Since all deployed models produce some kind of effect and feedback from the world there is an opportunity there to collect data targeted on the current level of the model, the most useful kind of data. That's why I think AI will be ok even with the proliferation of bots online. It's not 100% pure synthetic data in a loop, it is a agent-environment loop. tl;dr Models learn better from their own experiences, not ours.
- noduerme 3y ago
  But bullshit hallucinogenic output and fakery at colossal scales doesn't just pollute the pool of static information available on the web. It also warps the minds of the humans you're relying on to verify reality on the next training loop. Not only that, but we can also see the emergent breed of humans who believe that writing is similar to arithmetic - an unnecessary skill that can be handed off to a calculator. Or that making a movie shouldn't require knowing anything besides asking for what you want to see. How is someone like that - someone who wants to rely on bots - going to tell a bot what is or isn't true? How can they even have a pre-pollution baseline understanding of reality? I just finished rereading Do Androids Dream for the first time in 20 years or so, and was astonished at how similar his andys really are to LLMs in the polluted / destroyed reality there. How confusing and corrupting they are to organic life. PKD describes the androif brains as neural networks with thousands of layered pathways and trillions of weighted parameters, and it's as if he was able to accurately conceive of what linguistic and "emotional" strengths and weaknesses those constructs would actually have, decades before LLMs existed. And there's this one amazing line where Deckard calls them "Life thieves". What else should we call what Sam Altman and others are building - but theft of human ingenuity, creativity, and basic reason for living, and the utter annihilation and suppression of people like this 16 year old kid who dare to hope they can contribute something more original in life than being a servant of a tech company building this shit, or a tiktok influencer who writes prompts? What should that kid hope: That their work becomes noticeable enough to be immediately stolen and their name turned into a prompt? Life thieves. Just as a further aside, I had dinner tonight with a friend who's a fairly famous animator in the commercial realm, and I brought up this post. He's just sure the kid's screwed and the genie is out and creative is basically over. He's turning to building wooden clocks. But his reaction, and the reactions I see here every time this comes up, remind me of something else. They remind me of how people react when someone is robbed. Everyone has some reason why it's bad but not that bad, it was inevitable, it'll be okay, etc. Or they go around wondering what they're going to do now. Or they paper it over with optimism. Surprisingly few people get robbed and are willing to realize they were robbed, and become wildly pissed off about it. Most people have some sort of flight reaction, as evidenced by e.g. the promotion of "prompt writing" or learning to use a paid API to do what was formerly your own creative job that now spits your own work back at you. I say sue the shit out of all these content thieves.

pton_xd 3y ago

> It’s just really clear that a giant text averaging machine can only go so far It's not really a text averaging machine, it's a pattern matching machine. Right now the "depth" of the patterns it can match can only go so far, but in a few years with more advances in chips and memory the depth is going to increase and the patterns it can match will fan out accordingly.

vrighter 3y ago
it is a statistical model. If everyone is saying X and is wrong, and one guy says Y and is right, the llm will bit out X. Because that is the most probable thing in the dataset. It literally is a text averaging machine

visarga 3y ago

Not just real world data from videos. AI models need feedback from many sources: humans, code execution, web search, simulations, games, robotics, math verification, or from actual experiments in the real world. All of these are environments that can take the output of a model and do some processing and return feedback. The model can learn and search for solutions, creating its own training data as a RL agent. Since all deployed models produce some kind of effect and feedback from the world there is an opportunity there to collect data targeted on the current level of the model, the most useful kind of data. That's why I think AI will be ok even with the proliferation of bots online. It's not 100% pure synthetic data in a loop, it is a agent-environment loop. tl;dr Models learn better from their own experiences, not ours.

noduerme 3y ago
But bullshit hallucinogenic output and fakery at colossal scales doesn't just pollute the pool of static information available on the web. It also warps the minds of the humans you're relying on to verify reality on the next training loop. Not only that, but we can also see the emergent breed of humans who believe that writing is similar to arithmetic - an unnecessary skill that can be handed off to a calculator. Or that making a movie shouldn't require knowing anything besides asking for what you want to see. How is someone like that - someone who wants to rely on bots - going to tell a bot what is or isn't true? How can they even have a pre-pollution baseline understanding of reality? I just finished rereading Do Androids Dream for the first time in 20 years or so, and was astonished at how similar his andys really are to LLMs in the polluted / destroyed reality there. How confusing and corrupting they are to organic life. PKD describes the androif brains as neural networks with thousands of layered pathways and trillions of weighted parameters, and it's as if he was able to accurately conceive of what linguistic and "emotional" strengths and weaknesses those constructs would actually have, decades before LLMs existed. And there's this one amazing line where Deckard calls them "Life thieves". What else should we call what Sam Altman and others are building - but theft of human ingenuity, creativity, and basic reason for living, and the utter annihilation and suppression of people like this 16 year old kid who dare to hope they can contribute something more original in life than being a servant of a tech company building this shit, or a tiktok influencer who writes prompts? What should that kid hope: That their work becomes noticeable enough to be immediately stolen and their name turned into a prompt? Life thieves. Just as a further aside, I had dinner tonight with a friend who's a fairly famous animator in the commercial realm, and I brought up this post. He's just sure the kid's screwed and the genie is out and creative is basically over. He's turning to building wooden clocks. But his reaction, and the reactions I see here every time this comes up, remind me of something else. They remind me of how people react when someone is robbed. Everyone has some reason why it's bad but not that bad, it was inevitable, it'll be okay, etc. Or they go around wondering what they're going to do now. Or they paper it over with optimism. Surprisingly few people get robbed and are willing to realize they were robbed, and become wildly pissed off about it. Most people have some sort of flight reaction, as evidenced by e.g. the promotion of "prompt writing" or learning to use a paid API to do what was formerly your own creative job that now spits your own work back at you. I say sue the shit out of all these content thieves.