3 ms·

> it is language that crystallises and captures almost all the human intelligence. This used to be a common position held by the likes of Wittgenstein, but the

by mxkopy 3y ago

> it is language that crystallises and captures almost all the human intelligence. This used to be a common position held by the likes of Wittgenstein, but the biological reality is probably different [1]. Language is probably like the shadows in the Allegory of the Cave, a representation of some thing not to be confused for the thing itself. [1]: https://mcgovern.mit.edu/2019/05/02/ask-the-brain-can-we-think-without-language/ https://mcgovern.mit.edu/2019/05/02/ask-the-brain-can-we-thi...

visarga 3y ago
Of course language relates to things, but I am thinking of language in the sense of a constantly growing corpus of text. It is a self replicator, but like a virus it uses the host organism, a human or a LLM to create more of itself. Ideas have their own life-cycle, they don't get born or die with us, they like to travel and evolve fast. My point was the capabilities we attribute to LLMs actually come from the training data, not the model. Any model would do, from 10B parameters to 500B, with cosine embeds, rotary or AliBi, even RNNs can do it today. But the data - that is precious, the source of all these "emergent" abilities. We are wondering at the mysteries of the transformer, but the mystery was hiding the training set. Let's look at the problem from the point of view of language as an evolutionary system of ideas. Like scientific papers and citations, that kind of stuff, leading to AI and human progress. That's where our progress will come from - the evolution of ideas.
- oskarkk 3y ago
  > Of course language relates to things, but I am thinking of language in the sense of a constantly growing corpus of text. It is a self replicator, but like a virus it uses the host organism, a human or a LLM to create more of itself. Ideas have their own life-cycle, they don't get born or die with us, they like to travel and evolve fast. You're talking about "memes", in the original meaning: https://en.wikipedia.org/wiki/Meme https://en.wikipedia.org/wiki/Meme
  - 082349872349872 3y ago
    cf https://i.kym-cdn.com/photos/images/original/000/573/606/d0a.jpg https://i.kym-cdn.com/photos/images/original/000/573/606/d0a...

by mxkopy 3y ago

visarga 3y ago

Of course language relates to things, but I am thinking of language in the sense of a constantly growing corpus of text. It is a self replicator, but like a virus it uses the host organism, a human or a LLM to create more of itself. Ideas have their own life-cycle, they don't get born or die with us, they like to travel and evolve fast. My point was the capabilities we attribute to LLMs actually come from the training data, not the model. Any model would do, from 10B parameters to 500B, with cosine embeds, rotary or AliBi, even RNNs can do it today. But the data - that is precious, the source of all these "emergent" abilities. We are wondering at the mysteries of the transformer, but the mystery was hiding the training set. Let's look at the problem from the point of view of language as an evolutionary system of ideas. Like scientific papers and citations, that kind of stuff, leading to AI and human progress. That's where our progress will come from - the evolution of ideas.

oskarkk 3y ago
> Of course language relates to things, but I am thinking of language in the sense of a constantly growing corpus of text. It is a self replicator, but like a virus it uses the host organism, a human or a LLM to create more of itself. Ideas have their own life-cycle, they don't get born or die with us, they like to travel and evolve fast. You're talking about "memes", in the original meaning: https://en.wikipedia.org/wiki/Meme https://en.wikipedia.org/wiki/Meme
- 082349872349872 3y ago
  cf https://i.kym-cdn.com/photos/images/original/000/573/606/d0a.jpg https://i.kym-cdn.com/photos/images/original/000/573/606/d0a...