5 ms·
The stochastic parrot claim seems to crash people’s brains because they assume this is all about words. The fact that these models very clearly learn not just p
by thom 4y ago
The stochastic parrot claim seems to crash people’s brains because they assume this is all about words. The fact that these models very clearly learn not just patterns of words, but patterns of ideas, and ideas about ideas, seems a very profound result. Personally I do remain unimpressed by how the current models ‘think’ on top of this knowledge, but I’d have to work very hard to be as cynical as to imply this was all a scam, or somehow not progress (whatever you think of the destination).
- masswerk 4y agoHum, as someone who grew up in the times of the linguistic turn, I'm somewhat missing the groundbreaking revolution here. The entire idea of "text" was based on this.
- AndrewDucker 4y ago"these models very clearly learn not just patterns of words, but patterns of ideas, and ideas about ideas" This is not at all clear to me.
- smitty1e 4y agoThe assertion seems to be that epistemology is boils down to data crunching. But is epistemology purely empirical? Color me skeptical. One feels that we're nailing the easy 3/4 of the question or so.
- AndrewDucker 4y agoI think it's worse than that. We're doing a great job of generating text which looks reasonable X% of the time. Which doesn't actually mean we've solved X% of the question - because there is no way of building on top of what we have to close the gap.
- brookst 4y agoAre you talking about people or LLMs?
- ieee2 4y agoAs an new AI image generators are not just replicating existing pictures but generating wonderful new ones, language models are not just replicating average of what they seen but are capable of generating novel ideas.
- AndrewDucker 4y agoThey're certainly capable of remixing things they've seen, and adding in randomness will add novelty. Whether that counts as "creativity" is something people can debate : - ) I think that the reason image ones have caught on better in some ways is because they don't need to be accurate. We're not asking them to understand anything, just produce images based on text prompts (which is amazing stuff all by itself).
- stuckinhell 4y agoRemixing objects and designs consistently like "van gogh" or "super mario" really implies some kind of internal model or "understanding" of the world. Image generation didn't catch on because of lack of accuracy, but because of how GOOD the results are. It's made artwork immediately accessible to the masses without the huge learning curve. That's where these AI's are going to shine very very quickly.
- AndrewDucker 4y agoOh yes - you show them X and they make a model of X. Show them enough pictures labelled "Van Gogh" and they get an idea of what "Van Gogh" looks like. They dp am awesome job of that. The problem with the text ones is that people think that showing them words mean that they make a model of the thing the words are describing, rather than of how those words go together.
- stuckinhell 4y agoI guess you are on a different level of abstraction that I am. It's clear the art AI's have a model of "van gogh's" style, and apply it to create very unique forms of art. The neural model weight's aren't storing compressed images of van gogh, but relationships and mathematical models about a concept.
- jjtheblunt 4y ago> The fact... Proof or at least citation somewhere?