5 ms·

> In a 1991 interview, Winograd said about SHRDLU:[3] > [...] the famous dialogue with SHRDLU where you could pick up a block, and so on, I very carefully work

by slibhb 4y ago

> In a 1991 interview, Winograd said about SHRDLU:[3] > [...] the famous dialogue with SHRDLU where you could pick up a block, and so on, I very carefully worked through, line by line. If you sat down in front of it, and asked it a question that wasn't in the dialogue, there was some probability it would answer it. I mean, if it was reasonably close to one of the questions that was there in form and in content, it would probably get it. But there was no attempt to get it to the point where you could actually hand it to somebody and they could use it to move blocks around. And there was no pressure for that whatsoever. Pressure was for something you could demo. Take a recent example, Negroponte's Media Lab, where instead of "perish or publish" it's "demo or die." I think that's a problem. I think AI suffered from that a lot, because it led to "Potemkin villages", things which - for the things they actually did in the demo looked good, but when you looked behind that there wasn't enough structure to make it really work more generally. Unlike "Chomskyian efforts," statisical NLP actually allows the machine to interact with natural language.

YeGoblynQueenne 4y ago
The whole conversation with Winograd is here: https://conservancy.umn.edu/bitstream/handle/11299/107717/oh237taw.pdf https://conservancy.umn.edu/bitstream/handle/11299/107717/oh... In that passage you quoted, Winograd is saying that SHRDLU was essentially an academic demo, something created only for the purpose of passing a thesis examination, even as a tech demo is something created only to show off at a showroom. That's well known. But SHRDLU, academic demo that it was, was a program hand-crafted by a single PhD student in a few months. It didn't have behind it a team of 30+ people, or a budget of hundreds of thousands of dollars, like GPT-3 -and who knows about ChatGPT! And yet it still worked better than ChatGPT has ever worked, even though just for the purpose of moving virtual blocks around a virtual world. >> Unlike "Chomskyian efforts," statisical NLP actually allows the machine to interact with natural language. Sure. And do what? Go off the rails like a crazy train, most of the time.
- slibhb 4y ago
  You originally wrote: > SHRDLU, confined as it was in its virtual, toy world, could still follow the instructions of its user with absolute precision. It seems clear that, in light of the quote from Winograd, that simply isn't true. > But SHRDLU, academic demo that it was, was a program hand-crafted by a single PhD student in a few months. It didn't have behind it a team of 30+ people, or a budget of hundreds of thousands of dollars, like GPT-3 -and who knows about ChatGPT! Do you have some idea in mind how we might scale up from SHRDLU's approach? Does Winograd? It seems pretty clear to me that no one does; it was a one-off toy that didn't solve any fundamental problems. > And yet it still worked better than ChatGPT has ever worked, even though just for the purpose of moving virtual blocks around a virtual world. There are plenty of questions remaining concerning LLMs. We don't know what they will be used for, where they will succeed and fail. But they are absolutely a huge step forward from earlier attempts and it's utterly bizarre to deny that.
  - YeGoblynQueenne 4y ago
    A huge step forward- to what? To building an uncontrollable firehose of bullshit? What, exactly, is that going "forward" towards? In any case, I'm no clairvoyant and I can't see the future. What I see right now is a huge mess and a gigantic marketing bubble. So I'll just sit tightly and wait to see what will happen in a few years. I do have to note though that, so far, all the large hype models that have come out of OpenAI's research -GPT-2, GPT-3, now ChatGPT- have not really been useful in anything but parlor tricks: generating silly poems and malfunctioning code, and the like. They're great entertainment, but not very useful all. As indeed have been earlier overhyped results. I wonder if anyone remembers the Neural Translation Machine in Google Translate and how it invented its own "interlingua"? >> It seems clear that, in light of the quote from Winograd, that simply isn't true. You're right, my comment exaggerates the achievement in SHRDLU. So let's say, not "absolute precision" but "unprecedented precision" and precision not since achieved again. Certainly not by any statistical language model. Or, I mean, I don't know. People have controlled robots with code created by GPT-3 etc (I've been in a couple of conferences where people from MIT were showing off some research to that point). Can you show me an example of an interaction like the one in Winograd's thesis? Personally, I haven't seen any so far. Edit: I'm re-reading my comments to make sure the tone is civil. Please let me know if you don't think it's OK.

by slibhb 4y ago

YeGoblynQueenne 4y ago

The whole conversation with Winograd is here: https://conservancy.umn.edu/bitstream/handle/11299/107717/oh237taw.pdf https://conservancy.umn.edu/bitstream/handle/11299/107717/oh... In that passage you quoted, Winograd is saying that SHRDLU was essentially an academic demo, something created only for the purpose of passing a thesis examination, even as a tech demo is something created only to show off at a showroom. That's well known. But SHRDLU, academic demo that it was, was a program hand-crafted by a single PhD student in a few months. It didn't have behind it a team of 30+ people, or a budget of hundreds of thousands of dollars, like GPT-3 -and who knows about ChatGPT! And yet it still worked better than ChatGPT has ever worked, even though just for the purpose of moving virtual blocks around a virtual world. >> Unlike "Chomskyian efforts," statisical NLP actually allows the machine to interact with natural language. Sure. And do what? Go off the rails like a crazy train, most of the time.

slibhb 4y ago
You originally wrote: > SHRDLU, confined as it was in its virtual, toy world, could still follow the instructions of its user with absolute precision. It seems clear that, in light of the quote from Winograd, that simply isn't true. > But SHRDLU, academic demo that it was, was a program hand-crafted by a single PhD student in a few months. It didn't have behind it a team of 30+ people, or a budget of hundreds of thousands of dollars, like GPT-3 -and who knows about ChatGPT! Do you have some idea in mind how we might scale up from SHRDLU's approach? Does Winograd? It seems pretty clear to me that no one does; it was a one-off toy that didn't solve any fundamental problems. > And yet it still worked better than ChatGPT has ever worked, even though just for the purpose of moving virtual blocks around a virtual world. There are plenty of questions remaining concerning LLMs. We don't know what they will be used for, where they will succeed and fail. But they are absolutely a huge step forward from earlier attempts and it's utterly bizarre to deny that.
- YeGoblynQueenne 4y ago
  A huge step forward- to what? To building an uncontrollable firehose of bullshit? What, exactly, is that going "forward" towards? In any case, I'm no clairvoyant and I can't see the future. What I see right now is a huge mess and a gigantic marketing bubble. So I'll just sit tightly and wait to see what will happen in a few years. I do have to note though that, so far, all the large hype models that have come out of OpenAI's research -GPT-2, GPT-3, now ChatGPT- have not really been useful in anything but parlor tricks: generating silly poems and malfunctioning code, and the like. They're great entertainment, but not very useful all. As indeed have been earlier overhyped results. I wonder if anyone remembers the Neural Translation Machine in Google Translate and how it invented its own "interlingua"? >> It seems clear that, in light of the quote from Winograd, that simply isn't true. You're right, my comment exaggerates the achievement in SHRDLU. So let's say, not "absolute precision" but "unprecedented precision" and precision not since achieved again. Certainly not by any statistical language model. Or, I mean, I don't know. People have controlled robots with code created by GPT-3 etc (I've been in a couple of conferences where people from MIT were showing off some research to that point). Can you show me an example of an interaction like the one in Winograd's thesis? Personally, I haven't seen any so far. Edit: I'm re-reading my comments to make sure the tone is civil. Please let me know if you don't think it's OK.