4 ms·
I can point at code that sorts words by use pretty easily. And I can show you how the embeddings correlate words with similar meanings. Surely that answers th
by Dylan16807 4y ago
I can point at code that sorts words by use pretty easily. And I can show you how the embeddings correlate words with similar meanings. Surely that answers the question?
Remember that the real claim is that embeddings represent meaning. In a flawed but working way. You could say they are a map, not the territory, so arguing about the true meaning of meaning isn't relevant.
> I'm going to wonder whether, if I ask for the code for "quicksort" you'll point me to the code for "bubblesort"
Embeddings are meaning (noun) like a dictionary, not meaning (verb) like a qualia.
The code sorts. It doesn't matter what algorithm because the claim was just that it gives you sorted data. Just like a claim that embeddings represent things is not a claim about methods or qualia.
- travisjungroth 4y agoI was going to say something similar. The meanings aren’t code, they’re data. So you can definitely pull up the vector for “meaning” or anything else The claim (that I’m making at least) is not that these systems can understand meaning. It’s that they can store a representation of meaning and do something with it. I think there’s a way lower bar for this. (I also think the bar was cleared quite a while ago and we should consider it settled.) I could write down the meaning of a few words on some pieces of paper. (I guess if someone disagrees with that then talking about vectors is pointless). I could stuff each of those papers into a different dog toy and train a dog to get them on command. Then I think you could truthfully say something like “this dog can retrieve the meaning of the word ‘independence’, ‘hammer’ or ‘sing’ for you”. It wouldn’t be true to say the dog understands any of those words.
- YeGoblynQueenne 4y agoSo in other words, you're saying what you said above: >> Maybe the thing that will land is to say that embeddings are an attempt at representing meaning. And I don't disagree with that, neither with what you say in this comment. Or at least, I think I agree. What I would have said is that you can write down words on a paper, and the words are a representation of meaning, but the paper doesn't understand meaning, nor do the words. You need a human, with a human understanding of words, and language, and what those things mean, in order to decode the meaning from the words. In other ah words, the words on the paper are a representation of meaning, but only for a human. For a cat, say, they don't represent anything. Wat's more, just because a language model is trained on word embeddings, doesn't mean it encodes meaning. It's certainly an attempt to do that, but just because someone made an attempt doesn't mean they've done it.
- travisjungroth 4y agoI split the idea into “attempt” when they didn’t seem to understand what was being talked about at all. After that, you could discuss if the attempt was successful. I think that because these systems can solve problems that involve the meanings of words, we can say there must be some meaning information encoded in the data. > You need a human, with a human understanding of words, and language, and what those things mean, in order to decode the meaning from the words. By this definition I think you’ll always have doubt about these systems.
- YeGoblynQueenne 4y agoNot necessarily. I'm describing the current situation, as I understand it. There's nothing stopping a future system from having the same abilities as a human to decode meaning from words, or to do anything else. But I for one would like to know exactly how such a system would work, and why it would have the capabilities it would have. I don't think that's a particularly high bar to clear, for technology and science. The whole point of both is to figure out how things work, and how to make them work like we want.
- alexahn 4y ago>Or at least, I think I agree. What I would have said is that you can write down words on a paper, and the words are a representation of meaning, but the paper doesn't understand meaning, nor do the words. You need a human, with a human understanding of words, and language, and what those things mean, in order to decode the meaning from the words. In other ah words, the words on the paper are a representation of meaning, but only for a human. For a cat, say, they don't represent anything. If you change your perspective on language to mean any arbitrary capture of useful information (such that it can be used in the future), then you can see that the boundary between words and the world is not the heart of the issue. For example, your perception of the world works in a similar manner in that your sensory organs cannot comprehend the world, almost like your sensory organs interpret the world using their own language. Or maybe if that example is not very intuitive, then how about imagining an alien species that has sensory organs that act on linguistic structures. In some way, the aliens will figure out a coherent structure of their own, even though they cannot "experience" the world through senses like ours. What "intelligence" is doesn't seem to be bound by how "close" someone is to reality, and "close" might not even be the right word, since different perceptions can have different capabilities. I think the tricky part about "intelligence" is that there is always some "meaning" captured, it is just alien to those who do not share the same interpretive capacity. A cat could extract meaningful information from words on a paper, but certainly not in the same way we do. Now if we want to make an AI that acts and thinks like us, then understanding our own machinery (the relationship between the world and language) is certainly important. But I think the bitter lesson rears its head here, and I believe that something that is truly worthy of being called AGI will be able to thrive given any set of arbitrary senses, even linguistic ones. In other words, I do not think embodiment will naturally lead to AGI, rather that embodiment is a necessity if we want make AI in our image. And making an AI in our image is the fastest way to get an AI to do useful work for us.