5 ms·

Agreed, right down to their conclusion resonating as way overstated. Actually, meaningless would be more accurate. The thing about LLMs is exactly that they do

Agreed, right down to their conclusion resonating as way overstated. Actually, meaningless would be more accurate. The thing about LLMs is exactly that they don't understand by design. It often feels very distinctly like it's just engaging in sophisticated wordplay. A parlor trick. When ChatGPT 4 first came out I spent a couple of hours putting together a chess game using ChatGPT as the engine. It was shockingly bad, as in even attempting to make invalid moves. I get it: it's not tuned for that purpose, and its chess training corpus could probably be expanded to improve it as well. But, it actually served as a near-perfect demonstration of its lack of understanding, as well as the confidence with which it asserts things that are simply wrong. On a recent integration project with a good bit of nuanced functionality, it led me astray multiple times. I've gotten to a point where I can feel when its answers are not quite right, particularly if I know just a little about the topic. And, when challenged, it does that strange thing of responding with something along the lines of, "My apologies you're completely right that I was completely wrong". Over time, there becomes a sense that there is no there there. Even it's writing capabilities, lauded by so many, are of a style that is superficial and perfunctory or rote. That makes sense when you know what it is, but that's the thing: we get articles like these, lauding its wisdom.

bongodongobob 3y ago
Idk. One of my first tests for GPT4 was writing a website "for snakes." It was a flask app, and it did all the obvious things you'd expect. There was a title that said "Snake.com - A website for snakes" and a bunch of silly marketing stuff. What impressed me is when I asked to make it more snake-like (what does that even mean right?). It changed the colors to shades of green, used italic fonts, added some hisssssing sssstuff to wordssss, and added a diamond pattern through the background. It was a dumb and not very fancy site, but I'm not sure you can say it doesn't understand anything at all when you ask it to make a website more snakelike and actually made a pretty good attempt at doing it.
- unclebucknasty 3y ago
  Yeah, that's kind of a different conception of understanding though. The lines do get a little blurry at a certain point, and a lot of what it does "feels" like understanding, especially given how it "communicates". But I think it comes down to whether it can reason about things and whether it can draw new conclusions or create new information as a result. Your snake site is probably a good example. ChatGPT has a bunch of words that it knows are associated with snakes. It's pretty straightforward pattern matching. It doesn't really "understand" what those words mean, except that they have relationships to other words. But, if you were to ask it to reason and draw new conclusions about these things beyond its training corpus, it would be unable to reliably do so. Similarly, it had no idea about the quality (and sometimes legality) of the chess moves it generated.
  - bongodongobob 3y ago
    I mostly agree. I really think chess is just a terrible example though. You're really asking a lot of it but I'm honestly shocked it can do what it can. It seems to know some opening books, but falls down immediately. Which really makes sense because you'll find a lot of reading material on specific openings, but the problem space of the game is just too big to find texts about any given game state. Maybe if you have it reason about the board state and "think" about tactics you could push it farther. But we've already solved this. We have Stockfish et al and they've literally changed the game. Asking an LLM to play chess, while cool, is like trying to train a fish to dance. I think once we have an AI that's built with a bunch of different models that specialize in different things the idea of understanding is going to get even blurrier to the point that we might even say "yes, it doesn't 'understand' things, but it's better at humans at literally everything" so the difference becomes meaningless. I'm also of the opinion that humans are fancy automatons, so I tend to argue both sides. I'll say yeah it's thinking and so do we or ok it's not, but either do we.
    - unclebucknasty 3y ago
      >Asking an LLM to play chess, while cool, is like trying to train a fish to dance. 100%. I didn't expect much from it going in. Was more curious to know how good/bad it might be and, really, whether it could play at all. But, I mention it here because it illustrates the disparity between understanding/reasoning (LLMs bad) and semantics (LLMs good), per the subject of the thread. >get even blurrier to the point that we might even say "yes, it doesn't 'understand' things, but it's better at humans at literally everything" so the difference becomes meaningless. I get exactly what you're saying here and I've wondered the same. But, that's the thing I'm not sure about, given its inability to reason. I mean, there's the idea that it doesn't matter whether it "understands" if it correctly answers. But, then I wonder how it can "truly" be better than humans at "literally everything", if only humans truly understand.
- Solvency 3y ago
  You're not refuting and are only proving points above. The LLM understands relative patterns and/or associations. Statistically. Snakes tend to be green. People have written countless books with hisssssssing from snakes and snake-like characters. Diamond patterns are well documented with snakes. Again, it has no idea what these "are" and if we collectively decided snakes were called "Murder ropes" then it'd make the exact same associations.
  - bongodongobob 3y ago
    It does, because it did what I asked it to. I didn't ask for green or snake talk. I asked for snake-like and it obliged. If I did the exact same thing you'd be like "well, yeah, you're not an idiot, you understand what a snake is and what snake like things are."

bongodongobob 3y ago

Idk. One of my first tests for GPT4 was writing a website "for snakes." It was a flask app, and it did all the obvious things you'd expect. There was a title that said "Snake.com - A website for snakes" and a bunch of silly marketing stuff. What impressed me is when I asked to make it more snake-like (what does that even mean right?). It changed the colors to shades of green, used italic fonts, added some hisssssing sssstuff to wordssss, and added a diamond pattern through the background. It was a dumb and not very fancy site, but I'm not sure you can say it doesn't understand anything at all when you ask it to make a website more snakelike and actually made a pretty good attempt at doing it.

unclebucknasty 3y ago
Yeah, that's kind of a different conception of understanding though. The lines do get a little blurry at a certain point, and a lot of what it does "feels" like understanding, especially given how it "communicates". But I think it comes down to whether it can reason about things and whether it can draw new conclusions or create new information as a result. Your snake site is probably a good example. ChatGPT has a bunch of words that it knows are associated with snakes. It's pretty straightforward pattern matching. It doesn't really "understand" what those words mean, except that they have relationships to other words. But, if you were to ask it to reason and draw new conclusions about these things beyond its training corpus, it would be unable to reliably do so. Similarly, it had no idea about the quality (and sometimes legality) of the chess moves it generated.
- bongodongobob 3y ago
  I mostly agree. I really think chess is just a terrible example though. You're really asking a lot of it but I'm honestly shocked it can do what it can. It seems to know some opening books, but falls down immediately. Which really makes sense because you'll find a lot of reading material on specific openings, but the problem space of the game is just too big to find texts about any given game state. Maybe if you have it reason about the board state and "think" about tactics you could push it farther. But we've already solved this. We have Stockfish et al and they've literally changed the game. Asking an LLM to play chess, while cool, is like trying to train a fish to dance. I think once we have an AI that's built with a bunch of different models that specialize in different things the idea of understanding is going to get even blurrier to the point that we might even say "yes, it doesn't 'understand' things, but it's better at humans at literally everything" so the difference becomes meaningless. I'm also of the opinion that humans are fancy automatons, so I tend to argue both sides. I'll say yeah it's thinking and so do we or ok it's not, but either do we.
  - unclebucknasty 3y ago
    >Asking an LLM to play chess, while cool, is like trying to train a fish to dance. 100%. I didn't expect much from it going in. Was more curious to know how good/bad it might be and, really, whether it could play at all. But, I mention it here because it illustrates the disparity between understanding/reasoning (LLMs bad) and semantics (LLMs good), per the subject of the thread. >get even blurrier to the point that we might even say "yes, it doesn't 'understand' things, but it's better at humans at literally everything" so the difference becomes meaningless. I get exactly what you're saying here and I've wondered the same. But, that's the thing I'm not sure about, given its inability to reason. I mean, there's the idea that it doesn't matter whether it "understands" if it correctly answers. But, then I wonder how it can "truly" be better than humans at "literally everything", if only humans truly understand.
Solvency 3y ago
You're not refuting and are only proving points above. The LLM understands relative patterns and/or associations. Statistically. Snakes tend to be green. People have written countless books with hisssssssing from snakes and snake-like characters. Diamond patterns are well documented with snakes. Again, it has no idea what these "are" and if we collectively decided snakes were called "Murder ropes" then it'd make the exact same associations.
- bongodongobob 3y ago
  It does, because it did what I asked it to. I didn't ask for green or snake talk. I asked for snake-like and it obliged. If I did the exact same thing you'd be like "well, yeah, you're not an idiot, you understand what a snake is and what snake like things are."