4 ms·

Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that com

Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logic. I have an extremely basic question, that anyone in a (possibly mandatory) high school science class would answer correctly(although to be fair, it could be a 100 level college question). It still cannot answer it correctly because there is too many stay-at-home-parent blogs giving the wrong answer. Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said. So what can we make of this interview? That we have someone spouting opinions, and everyone else laughs at their opinion since they are famous, old, and out of touch. EDIT: I think LLMs are incredibly useful. I use it more than Google. It doesnt mean its smarter than humans, it means google is worse than LLMs. I can't even provide a list of all the uses, but it doesnt mean they are taking advantage of an old man out of their element.

colechristensen 3y ago
People overly impressed by LLMs haven't spent a lot of time trying to make them actually useful. When you do, you learn that they're talented mimics but still quite limited.
- nielsbot 3y ago
  Part of me wonders, though, could we "just" connect up an inference engine and voila? We could really be on a cusp of general AI. (Or it could be a ways off) That's a bit frightening in several ways.
  - hackeraccount 3y ago
    I kind of expected AI to be AI - and not a mirror.
    - nielsbot 3y ago
      Meaning it wouldn't necessarily be human-like?
      - hackeraccount 3y ago
        That'd be a start I think. Or at the very least if it was in a room alone there would be something there.
- hackeraccount 3y ago
  This. LLM's have a surface that suggests they're an incredibly useful UI. That usability is like the proverbial hand full of water though - when you start to really squeeze it, it just slips away. I'm still not convinced that the problem isn't me though.
- jurgenaut23 3y ago
  Yes, I actually think this is true. See my above comment, which supports your claim.
- wilg 3y ago
  I think the reason to be impressed is that they do things that were previously not possible. And they are absolutely directly useful! Just not for everything. But it seems like a very fruitful line of research, and it’s easy to believe that future iterations will have significant improvements and those improvements will happen quickly. There’s no sense worrying about whether GPT4 is smarter than a human, the interesting part is that it demonstrates that we have techniques that may be able to get you to a machine that is smarter than a human.
- TheOtherHobbes 3y ago
  This also applies to humans.
  - criddell 3y ago
    Most humans can mimic but can also describe their first-hand experience of being scared or happy or heartbroken or feeling desire etc…
gwright 3y ago
> Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said. One of my take aways from LLMs is that humans very often just repeating what other humans have said with only superficial understanding of what they are repeating. I think there is more to general intelligence than pattern matching and mimicking but a disconcerting amount of our day to day human interactions might just be that.
taeric 3y ago
I mean, this someone "who is past their prime" is a very respected someone that almost certainly inspired a fair number of the folks working in these fields. So, yes, this is largely mining for quotes. But those are great quotes to ponder and to echo back through the machines that are research and progress to see where they lead. It would be one thing if these were being taken as a "stop all current work to make sure we are aligned with those that came before us." I don't see it in that way, though. Rather, there is value in listening to those that went before.
fipar 3y ago
> Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said. I agree with this completely. That said, I think this part is a bit unfair: > Lets take someone, who is past their prime and interview them on a topic they have never worked on. AI has been a part of DH's work for decades. For most of that time, he's dismissed the mainstream approach as being intelligent in the Strange Loop sense, and was involved in alternative approaches to AI. Also, if we remove "faster", "repeating what other humans verbally said" is something a lot of humans do, especially little children. I think that may be the part that scares DH: at this point, what these models are doing is not that different (from a superficial POV) from what little kids or even some adults do. I still agree with you though, and I think what DH misses here is the fact that IMHO there is no introspection at all in these models. Somehow, in the case of humans, we go from parroting like an LLM to introspection and actually original productions over the course of some years (n = 2 for me, but I think any parent can confirm this; watching this happen in front of my eyes has been one of greatest experiences in my life), but I can still understand how someone like DH would be confused by the current state of affairs.
FrustratedMonky 3y ago
>"Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes!" So. Don't interview anybody over 40? Who judges who is past their prime? Mining for quotes is most interviews. Isn't this why interviews happen? He's been in the AI field for what? 30 Years? Has multiple books. I think he has earned enough respect to have an opinion, and it is probably worth more than most people tossing out ad-hoc opinions on AI in the last few months. Better than some 'programmers', who are so in-the-weeds they have lost track of what they are building.

colechristensen 3y ago

People overly impressed by LLMs haven't spent a lot of time trying to make them actually useful. When you do, you learn that they're talented mimics but still quite limited.

nielsbot 3y ago
Part of me wonders, though, could we "just" connect up an inference engine and voila? We could really be on a cusp of general AI. (Or it could be a ways off) That's a bit frightening in several ways.
- hackeraccount 3y ago
  I kind of expected AI to be AI - and not a mirror.
  - nielsbot 3y ago
    Meaning it wouldn't necessarily be human-like?
    - hackeraccount 3y ago
      That'd be a start I think. Or at the very least if it was in a room alone there would be something there.
hackeraccount 3y ago
This. LLM's have a surface that suggests they're an incredibly useful UI. That usability is like the proverbial hand full of water though - when you start to really squeeze it, it just slips away. I'm still not convinced that the problem isn't me though.
jurgenaut23 3y ago
Yes, I actually think this is true. See my above comment, which supports your claim.
wilg 3y ago
I think the reason to be impressed is that they do things that were previously not possible. And they are absolutely directly useful! Just not for everything. But it seems like a very fruitful line of research, and it’s easy to believe that future iterations will have significant improvements and those improvements will happen quickly. There’s no sense worrying about whether GPT4 is smarter than a human, the interesting part is that it demonstrates that we have techniques that may be able to get you to a machine that is smarter than a human.
TheOtherHobbes 3y ago
This also applies to humans.
- criddell 3y ago
  Most humans can mimic but can also describe their first-hand experience of being scared or happy or heartbroken or feeling desire etc…

gwright 3y ago

> Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said. One of my take aways from LLMs is that humans very often just repeating what other humans have said with only superficial understanding of what they are repeating. I think there is more to general intelligence than pattern matching and mimicking but a disconcerting amount of our day to day human interactions might just be that.

taeric 3y ago

I mean, this someone "who is past their prime" is a very respected someone that almost certainly inspired a fair number of the folks working in these fields. So, yes, this is largely mining for quotes. But those are great quotes to ponder and to echo back through the machines that are research and progress to see where they lead. It would be one thing if these were being taken as a "stop all current work to make sure we are aligned with those that came before us." I don't see it in that way, though. Rather, there is value in listening to those that went before.

fipar 3y ago

> Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said. I agree with this completely. That said, I think this part is a bit unfair: > Lets take someone, who is past their prime and interview them on a topic they have never worked on. AI has been a part of DH's work for decades. For most of that time, he's dismissed the mainstream approach as being intelligent in the Strange Loop sense, and was involved in alternative approaches to AI. Also, if we remove "faster", "repeating what other humans verbally said" is something a lot of humans do, especially little children. I think that may be the part that scares DH: at this point, what these models are doing is not that different (from a superficial POV) from what little kids or even some adults do. I still agree with you though, and I think what DH misses here is the fact that IMHO there is no introspection at all in these models. Somehow, in the case of humans, we go from parroting like an LLM to introspection and actually original productions over the course of some years (n = 2 for me, but I think any parent can confirm this; watching this happen in front of my eyes has been one of greatest experiences in my life), but I can still understand how someone like DH would be confused by the current state of affairs.

FrustratedMonky 3y ago

>"Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes!" So. Don't interview anybody over 40? Who judges who is past their prime? Mining for quotes is most interviews. Isn't this why interviews happen? He's been in the AI field for what? 30 Years? Has multiple books. I think he has earned enough respect to have an opinion, and it is probably worth more than most people tossing out ad-hoc opinions on AI in the last few months. Better than some 'programmers', who are so in-the-weeds they have lost track of what they are building.