8 ms·

Yann LeCun's "Hoisted by their own GPTards" is fantastic.

frays 1y ago
I might be missing context here, but I'm surprised to see Yann using language that plays on 'retard.' That seems out of character for him - more like something I'd expect from Elon Musk. What's the context I'm missing?
- deleted 1y ago
  [deleted]
- nova22033 1y ago
  It's a play on the word petard
  - microtonal 1y ago
    I found this background useful as a non-native speaker: https://en.wikipedia.org/wiki/Hoist_with_his_own_petard https://en.wikipedia.org/wiki/Hoist_with_his_own_petard
- grey-area 1y ago
  Hoist (thrown in the air) by your own petard (bomb) is a common phrase.
- znkr 1y ago
  I don’t think it’s a wordplay with the r-word, but rather a reference to the famous Shakespeare quote: “Hoist with his own petard”. It’s become an English proverb. (A petard is a smallish bomb)
  - card_zero 1y ago
    From péter, to fart. Possibly entered the language as a saying due to Shakespeare being scurrilous.
- blitzar 1y ago
  You have been Hoisted with your own retard
NitpickLawyer 1y ago
While Yann is clearly brilliant, and has a deeper understanding of the roots of the filed than many of us mortals, I think he's been on a debbie downer trend lately, and more importantly, some of his public stances have been proven wrong in mere months / years after he made them. I remember a public talk, where he was on the stage with some young researcher from MS. (I think it was one of the authors of the "sparks of brilliance in gpt4" paper, but not sure). Anyway, throughout that talk he kept talking above the guy, and didn't seem to listen, even though he obviously didn't try the "raw", "unaligned" model that the folks at MS were talking about. And he made 2 big claims: 1) LLMs can't do math. He went on to "argue" that LLMs trick you with poetry that sounds good, but is highly subjective, and when tested on hard verifiable problems like math, they fail. 2) LLMs can't plan. Well, merely one year later, here we are. AIME is saturated (with tool use), gold at IMO, and current agentic uses clearly can plan (and follow up with the plan, re-write parts, finish tasks, etc etc). So, yeah, I'd take everything any one singular person says with a huge grain of salt. No matter how brilliant said individual is. Edit: oh, and I forgot another important argument that Yann made at that time: 3) because of the nature of LLMs, errors compound. So the longer you go in a session, the more errors accumulate so they devolve in nonsense. Again, mere months later the o series of models came out, and basically proved this point moot. Turns out RL + long context mitigate this fairly well. And a year later, we have all SotA models being able to "solve" problems 100k+ tokens deep.
- mrbungie 1y ago
  Pretty sure you can fill a room with serious researchers that at the very least will doubt about 2) being solved with LLMs, especially when talking about formal planning with pure LLMs and without a planning framwork. PS: So just we're clear: formal planning in AI </> making a coding plan in Cursor.
  - NitpickLawyer 1y ago
    > with pure LLMs and without a planning framwork. Sure, but isn't that moving the goalposts? Why shouldn't we use LLMs + tools if it works? If anything it shows that the early detractors weren't even considering this could work. Yann in particular was skeptical that long-context things can happen in LLMs at all. We now have "agents" that can work a problem for hours, with self context trimming, planning to md files, editing those plans and so on. All of this just works, today. We used to dream about it a year ago.

I might be missing context here, but I'm surprised to see Yann using language that plays on 'retard.' That seems out of character for him - more like something I'd expect from Elon Musk. What's the context I'm missing?

deleted 1y ago
[deleted]
nova22033 1y ago
It's a play on the word petard
- microtonal 1y ago
  I found this background useful as a non-native speaker: https://en.wikipedia.org/wiki/Hoist_with_his_own_petard https://en.wikipedia.org/wiki/Hoist_with_his_own_petard
grey-area 1y ago
Hoist (thrown in the air) by your own petard (bomb) is a common phrase.
znkr 1y ago
I don’t think it’s a wordplay with the r-word, but rather a reference to the famous Shakespeare quote: “Hoist with his own petard”. It’s become an English proverb. (A petard is a smallish bomb)
- card_zero 1y ago
  From péter, to fart. Possibly entered the language as a saying due to Shakespeare being scurrilous.
blitzar 1y ago
You have been Hoisted with your own retard

NitpickLawyer 1y ago

While Yann is clearly brilliant, and has a deeper understanding of the roots of the filed than many of us mortals, I think he's been on a debbie downer trend lately, and more importantly, some of his public stances have been proven wrong in mere months / years after he made them. I remember a public talk, where he was on the stage with some young researcher from MS. (I think it was one of the authors of the "sparks of brilliance in gpt4" paper, but not sure). Anyway, throughout that talk he kept talking above the guy, and didn't seem to listen, even though he obviously didn't try the "raw", "unaligned" model that the folks at MS were talking about. And he made 2 big claims: 1) LLMs can't do math. He went on to "argue" that LLMs trick you with poetry that sounds good, but is highly subjective, and when tested on hard verifiable problems like math, they fail. 2) LLMs can't plan. Well, merely one year later, here we are. AIME is saturated (with tool use), gold at IMO, and current agentic uses clearly can plan (and follow up with the plan, re-write parts, finish tasks, etc etc). So, yeah, I'd take everything any one singular person says with a huge grain of salt. No matter how brilliant said individual is. Edit: oh, and I forgot another important argument that Yann made at that time: 3) because of the nature of LLMs, errors compound. So the longer you go in a session, the more errors accumulate so they devolve in nonsense. Again, mere months later the o series of models came out, and basically proved this point moot. Turns out RL + long context mitigate this fairly well. And a year later, we have all SotA models being able to "solve" problems 100k+ tokens deep.

mrbungie 1y ago
Pretty sure you can fill a room with serious researchers that at the very least will doubt about 2) being solved with LLMs, especially when talking about formal planning with pure LLMs and without a planning framwork. PS: So just we're clear: formal planning in AI </> making a coding plan in Cursor.
- NitpickLawyer 1y ago
  > with pure LLMs and without a planning framwork. Sure, but isn't that moving the goalposts? Why shouldn't we use LLMs + tools if it works? If anything it shows that the early detractors weren't even considering this could work. Yann in particular was skeptical that long-context things can happen in LLMs at all. We now have "agents" that can work a problem for hours, with self context trimming, planning to md files, editing those plans and so on. All of this just works, today. We used to dream about it a year ago.