3 ms·

> The bottom line is, just because somebody says their algorithm does a thing, doesn't mean that everyone has to accept it immediately, and without critical dis

> The bottom line is, just because somebody says their algorithm does a thing, doesn't mean that everyone has to accept it immediately, and without critical discussion. That critical discussion was 10 years ago. Embeddings are vector representations of meaning. They’re not perfect, but they can be used for enough meaning-dependent tasks that this statement shouldn’t be controversial. All this AI/ML stuff is so exhausting because the philosophers come out of the woodwork. They don’t prove anything or disprove anything, they just kinda whine. When I was a flight instructor and said the airplane wanted to float down the runway, no one tried to point out that airplanes don’t have desires. I can say a server knows another server is offline and that’s fine too. But say “reason”, “think”, or “meaning” with something AI related and it’s like all work has to stop until we stop using that precious word. I’ve tried to engage with that stuff less, but got tricked cause I thought this person just literally didn’t understand.

YeGoblynQueenne 4y ago
I hope you're not calling me a "philosopher". I identify as a coder. And what I want to know is where is the code for "meaning". I don't care why it's the code for "meaning", but if you point me to the code for "word embeddings" and you say "that's the code for 'meaning", then I'm going to wonder whether, if I ask for the code for "quicksort" you'll point me to the code for "bubblesort". So, no, what we call things that do things is important, otherwise we don't know what things do what, and what things are being done. And lest we forget: However, in AI, our programs to a great degree are problems rather than solutions. If a researcher tries to write an "understanding" program, it isn't because he has thought of a better way of implementing this well-understood task, but because he thinks he can come closer to writing the implementation. If he calls the main loop of his program "UNDERSTAND', he i s (until proven innocent) merely begging the question. He may mislead a lot of people, most prominently himself, and enrage a lot of others. What he should do instead is refer to this main loop as "G0034", and see if he can convince himself or anyone else that G0034 implements some part of understanding. Or he could give i t a name that reveals its intrinsic properties, like NODE-NET- INTERSECTION-FINDER, it being the substance of his theory that finding intersections in networks of nodes constitutes understanding. If Quillian <1969> had called his program the "Teachable Language Node Net Intersection Finder", he would have saved us some reading. (Except for those of us fanatic about finding the part on teachability.) https://cs.fit.edu/~kgallagher/Schtick/Serious/McDermott.AI.MeetsNaturalStupidity.pdf https://cs.fit.edu/~kgallagher/Schtick/Serious/McDermott.AI.... Edit: And this is not right at all: >> That critical discussion was 10 years ago. In AI, there are lots of people who say lots of things. They keep saying them, even when people point out that they are just things that they say. They continue to say them even long after the other people get tired and give up trying to make sense of what is being said. That doesn't mean that the "discussion" is over, it's just that there is no real way to stop people saying things if that's what they really want to do.
- Dylan16807 4y ago
  I can point at code that sorts words by use pretty easily. And I can show you how the embeddings correlate words with similar meanings. Surely that answers the question? Remember that the real claim is that embeddings represent meaning. In a flawed but working way. You could say they are a map, not the territory, so arguing about the true meaning of meaning isn't relevant. > I'm going to wonder whether, if I ask for the code for "quicksort" you'll point me to the code for "bubblesort" Embeddings are meaning (noun) like a dictionary, not meaning (verb) like a qualia. The code sorts. It doesn't matter what algorithm because the claim was just that it gives you sorted data. Just like a claim that embeddings represent things is not a claim about methods or qualia.
  - travisjungroth 4y ago
    I was going to say something similar. The meanings aren’t code, they’re data. So you can definitely pull up the vector for “meaning” or anything else The claim (that I’m making at least) is not that these systems can understand meaning. It’s that they can store a representation of meaning and do something with it. I think there’s a way lower bar for this. (I also think the bar was cleared quite a while ago and we should consider it settled.) I could write down the meaning of a few words on some pieces of paper. (I guess if someone disagrees with that then talking about vectors is pointless). I could stuff each of those papers into a different dog toy and train a dog to get them on command. Then I think you could truthfully say something like “this dog can retrieve the meaning of the word ‘independence’, ‘hammer’ or ‘sing’ for you”. It wouldn’t be true to say the dog understands any of those words.
    - YeGoblynQueenne 4y ago
      So in other words, you're saying what you said above: >> Maybe the thing that will land is to say that embeddings are an attempt at representing meaning. And I don't disagree with that, neither with what you say in this comment. Or at least, I think I agree. What I would have said is that you can write down words on a paper, and the words are a representation of meaning, but the paper doesn't understand meaning, nor do the words. You need a human, with a human understanding of words, and language, and what those things mean, in order to decode the meaning from the words. In other ah words, the words on the paper are a representation of meaning, but only for a human. For a cat, say, they don't represent anything. Wat's more, just because a language model is trained on word embeddings, doesn't mean it encodes meaning. It's certainly an attempt to do that, but just because someone made an attempt doesn't mean they've done it.
knome 4y ago
>But say “reason”, “think”, or “meaning” with something AI related and it’s like all work has to stop until we stop using that precious word. The unwillingness to admit to AI capabilities because they are expressed in terms of human capabilities is what I deem an argument that "boats can't swim". True, in that what it does is different, but irrelevant in almost every facet that most people discussing it would care about. The boat, if anything, "swims" better than humans in basically every way. The argument against its use is only that swimming doesn't refer to boats. (I admit I might have picked this argument and phrase somewhere, but I did recently a cursory glance back through old comments, and think I might have coined it based on having then recently learned that Russian uses the same term for swimming and sailing, which I had found interesting at the time. But perhaps not, my first use appears almost a decade ago, and my mind is ever forgetful)
- travisjungroth 4y ago
  Dijkstra. https://www.goodreads.com/quotes/32629-the-question-of-whether-a-computer-can-think-is-no https://www.goodreads.com/quotes/32629-the-question-of-wheth...
  - knome 4y ago
    Ah, fantastic and thank you. I had suspected I picked it up somewhere, I just couldn't find any proof of it. My previous failure to find the origin was likely because I was searching for "boats" rather than "submarines".

YeGoblynQueenne 4y ago

I hope you're not calling me a "philosopher". I identify as a coder. And what I want to know is where is the code for "meaning". I don't care why it's the code for "meaning", but if you point me to the code for "word embeddings" and you say "that's the code for 'meaning", then I'm going to wonder whether, if I ask for the code for "quicksort" you'll point me to the code for "bubblesort". So, no, what we call things that do things is important, otherwise we don't know what things do what, and what things are being done. And lest we forget: However, in AI, our programs to a great degree are problems rather than solutions. If a researcher tries to write an "understanding" program, it isn't because he has thought of a better way of implementing this well-understood task, but because he thinks he can come closer to writing the implementation. If he calls the main loop of his program "UNDERSTAND', he i s (until proven innocent) merely begging the question. He may mislead a lot of people, most prominently himself, and enrage a lot of others. What he should do instead is refer to this main loop as "G0034", and see if he can convince himself or anyone else that G0034 implements some part of understanding. Or he could give i t a name that reveals its intrinsic properties, like NODE-NET- INTERSECTION-FINDER, it being the substance of his theory that finding intersections in networks of nodes constitutes understanding. If Quillian <1969> had called his program the "Teachable Language Node Net Intersection Finder", he would have saved us some reading. (Except for those of us fanatic about finding the part on teachability.) https://cs.fit.edu/~kgallagher/Schtick/Serious/McDermott.AI.MeetsNaturalStupidity.pdf https://cs.fit.edu/~kgallagher/Schtick/Serious/McDermott.AI.... Edit: And this is not right at all: >> That critical discussion was 10 years ago. In AI, there are lots of people who say lots of things. They keep saying them, even when people point out that they are just things that they say. They continue to say them even long after the other people get tired and give up trying to make sense of what is being said. That doesn't mean that the "discussion" is over, it's just that there is no real way to stop people saying things if that's what they really want to do.

Dylan16807 4y ago
I can point at code that sorts words by use pretty easily. And I can show you how the embeddings correlate words with similar meanings. Surely that answers the question? Remember that the real claim is that embeddings represent meaning. In a flawed but working way. You could say they are a map, not the territory, so arguing about the true meaning of meaning isn't relevant. > I'm going to wonder whether, if I ask for the code for "quicksort" you'll point me to the code for "bubblesort" Embeddings are meaning (noun) like a dictionary, not meaning (verb) like a qualia. The code sorts. It doesn't matter what algorithm because the claim was just that it gives you sorted data. Just like a claim that embeddings represent things is not a claim about methods or qualia.
- travisjungroth 4y ago
  I was going to say something similar. The meanings aren’t code, they’re data. So you can definitely pull up the vector for “meaning” or anything else The claim (that I’m making at least) is not that these systems can understand meaning. It’s that they can store a representation of meaning and do something with it. I think there’s a way lower bar for this. (I also think the bar was cleared quite a while ago and we should consider it settled.) I could write down the meaning of a few words on some pieces of paper. (I guess if someone disagrees with that then talking about vectors is pointless). I could stuff each of those papers into a different dog toy and train a dog to get them on command. Then I think you could truthfully say something like “this dog can retrieve the meaning of the word ‘independence’, ‘hammer’ or ‘sing’ for you”. It wouldn’t be true to say the dog understands any of those words.
  - YeGoblynQueenne 4y ago
    So in other words, you're saying what you said above: >> Maybe the thing that will land is to say that embeddings are an attempt at representing meaning. And I don't disagree with that, neither with what you say in this comment. Or at least, I think I agree. What I would have said is that you can write down words on a paper, and the words are a representation of meaning, but the paper doesn't understand meaning, nor do the words. You need a human, with a human understanding of words, and language, and what those things mean, in order to decode the meaning from the words. In other ah words, the words on the paper are a representation of meaning, but only for a human. For a cat, say, they don't represent anything. Wat's more, just because a language model is trained on word embeddings, doesn't mean it encodes meaning. It's certainly an attempt to do that, but just because someone made an attempt doesn't mean they've done it.

knome 4y ago

>But say “reason”, “think”, or “meaning” with something AI related and it’s like all work has to stop until we stop using that precious word. The unwillingness to admit to AI capabilities because they are expressed in terms of human capabilities is what I deem an argument that "boats can't swim". True, in that what it does is different, but irrelevant in almost every facet that most people discussing it would care about. The boat, if anything, "swims" better than humans in basically every way. The argument against its use is only that swimming doesn't refer to boats. (I admit I might have picked this argument and phrase somewhere, but I did recently a cursory glance back through old comments, and think I might have coined it based on having then recently learned that Russian uses the same term for swimming and sailing, which I had found interesting at the time. But perhaps not, my first use appears almost a decade ago, and my mind is ever forgetful)

travisjungroth 4y ago
Dijkstra. https://www.goodreads.com/quotes/32629-the-question-of-whether-a-computer-can-think-is-no https://www.goodreads.com/quotes/32629-the-question-of-wheth...
- knome 4y ago
  Ah, fantastic and thank you. I had suspected I picked it up somewhere, I just couldn't find any proof of it. My previous failure to find the origin was likely because I was searching for "boats" rather than "submarines".