5 ms·

I don't think it's "regardless", your opinion on LeCun being right should be highly correlated to your opinion on whether this is good for Europe. If you think

by jsnell 7mo ago

I don't think it's "regardless", your opinion on LeCun being right should be highly correlated to your opinion on whether this is good for Europe. If you think that LLMs are sufficient and RSI is imminent (<1 year), this is horrible for Europe. It is a distracting boondoggle exactly at the wrong time.

andrepd 7mo ago
It's been 6 months away for 5 years now. In that time we've seen relatively mild incremental changes, not any qualitative ones. It's probably not 6 months away.
- basket_horse 7mo ago
  But I swear this time is different! Just give me another 6 months!
  - andrepd 7mo ago
    And another 6 trillion dollars :^)
- AStrangeMorrow 7mo ago
  Yeah. I feel like that like many projects the last 20% take 80% of time, and imho we are not in the last 20% Sure LLMs are getting better and better, and at least for me more and more useful, and more and more correct. Arguably better than humans at many tasks yet terribly lacking behind in some others. Coding wise, one of the things it does “best”, it still has many issues: For me still some of the biggest issues are still lack of initiative and lack of reliable memory. When I do use it to write code the first manifests for me by often sticking to a suboptimal yet overly complex approach quite often. And lack of memory in that I have to keep reminding it of edge cases (else it often breaks functionality), or to stop reinventing the wheel instead of using functions/classes already implemented in the project. All that can be mitigated by careful prompting, but no matter the claim about information recall accuracy I still find that even with that information in the prompt it is quite unreliable. And more generally the simple fact that when you talk to one the only way to “store” these memories is externally (ie not by updating the weights), is kinda like dealing with someone that can’t retain memories and has to keep writing things down to even get a small chance to cope. I get that updating the weights is possible in theory but just not practical, still.
- lordmathis 7mo ago
  It's 6 months away the same way coding is apparently "solved" now.
  - HarHarVeryFunny 7mo ago
    I think we - in last few months - are very close to, if not already at, the point where "coding" is solved. That doesn't mean that software design or software engineering is solved, but it does mean that a SOTA model like GPT 5.4 or Opus 4.6 has a good chance of being able to code up a working version of whatever you specify, with reason. What's still missing is the general reasoning ability to plan what to build or how to attack novel problems - how to assess the consequences of deciding to build something a given way, and I doubt that auto-regressively trained LLMs is the way to get there, but there is a huge swathe of apps that are so boilerplate in nature that this isn't the limitation. I think that LeCun is on the right track to AGI with JEPA - hardly a unique insight, but significant to now have a well funded lab pursuing this approach. Whether they are successful, or timely, will depend if this startup executes as a blue skies research lab, or in more of an urgent engineering mode. I think at this point most of the things needed for AGI are more engineering challenges rather than what I'd consider as research problems.
    - lordmathis 7mo ago
      Sure, Claude and other SOTA LLMs do generate about 90% of my code but I feel like we are not closer to solving the last 10% than we were a year ago in the days of Claude 3.7. It can pretty reliably get 90% there and then I can either keep prompting it to get the rest done or just do it manually which is quite often faster.
      - j4k0o 7mo ago
        It's interesting that people don't seem to think the likely outcome might be... capital and labour. Not capital alone. You see this in construction - the capital is used for certain things and is operated by labour.
        7mo ago
- mfru 7mo ago
  Reminds me of how cold fusion reactors are only 5 years away for decades now
  - vidarh 7mo ago
    Cold fusion reactors haven't produced usable intermediate results. LLMs have.
    - leptons 7mo ago
      LLMs produce slop far to often to say they are in any way better than cold fusion in terms of usable results. "AI" kind of is the cold fusion of tech. We've always been 5 or 10 years away from "AGI" and likely always will be.
      - vidarh 7mo ago
        That's just nonsense. That they produce slop does not negate that I and many others get plenty of value out of them in their current form, while we get zero value out of fusion so far - cold or otherwise.
next_xibalba 7mo ago
> RSI Wait, we have another acronym to track. Is this the same/different than AGI and/or ASI?
- mietek 7mo ago
  Some people should definitely be getting Repetitive Strain Injury from all the hyping up of LLMs.
- notnullorvoid 7mo ago
  Recursive Self Improvement
- robrenaud 7mo ago
  Recursive self improvement. It's when AI speeds up the development of the next AI.
- deleted 7mo ago
  [deleted]
Insanity 7mo ago
Whenever I see claims about AGI being reachable through large language models, it reminds me of the miasma theory of disease. Many respectable medical professionals were convinced this was true, and they viewed the entire world through this lens. They interpreted data in ways that aligned with a miasmatic view. Of course now we know this was delusional and it seems almost funny in retrospect. I feel the same way when I hear that 'just scale language models' suddenly created something that's true AGI, indistinguishable from human intelligence.
- visarga 7mo ago
  > Whenever I see claims about AGI being reachable through large language models, it reminds me of the miasma theory of disease. Whenever I see people think the model architecture matters much, I think they have a magical view of AI. Progress comes from high quality data, the models are good as they are now. Of course you can still improve the models, but you get much more upside from data, or even better - from interactive environments. The path to AGI is not based on pure thinking, it's based on scaling interaction. To remain in the same miasma theory of disease analogy, if you think architecture is the key, then look at how humans dealt with pandemics... Black Death in the 14th century killed half of Europe, and none could think of the germ theory of disease. Think about it - it was as desperate a situation as it gets, and none had the simple spark to keep hygiene. The fact is we are also not smart from the brain alone, we are smart from our experience. Interaction and environment are the scaffolds of intelligence, not the model. For example 1B users do more for an AI company than a better model, they act like human in the loop curators of LLM work.
  - 0x3f 7mo ago
    If model arch doesn't matter much how come transformers changed everything?
    - visarga 7mo ago
      Luck. RNNs can do it just as good, Mamba, S4, etc - for a given budget of compute and data. The larger the model the less architecture makes a difference. It will learn in any of the 10,000 variations that have been tried, and come about 10-15% close to the best. What you need is a data loop, or a data source of exceptional quality and size, data has more leverage. Architecture games reflect more on efficiency, some method can be 10x more efficient than another.
dheera 7mo ago
Just because you raise 1 billion dollars to do X doesn't mean you can't pivot and do Y if it is in the best interest of your mission. I won't comment on Yann LeCun or his current technical strategy, but if you can avoid sunk cost fallacy and pivot nimbly I don't think it is bad for Europe at all. It is "1 billion dollars for an AI research lab", not "1 billion dollars to do X".
devonkelley 7mo ago
[flagged]
- sebmellen 7mo ago
  [flagged]
vidarh 7mo ago
It's sufficient to think that there is a chance that they will not be, however, for there to be a non-zero value to fund other approaches. And even if you think the chance is zero, unless you also think there is a zero chance they will be capable of pivoting quickly, it might still be beneficial. I think his views are largely flawed, but chances are there will still be lots of useful science coming out of it as well. Even if current architectures can achieve AGI, it does not mean there can't also be better, cheaper, more effective ways of doing the same things, and so exploring the space more broadly can still be of significant value.
Tenoke 7mo ago
I think LeCun has been so consistently wrong and boneheaded for basically all of the AI boom, that this is much, much more likely to be bad than good for Europe. Probably one of the worst people to give that much money to that can even raise it in the field.
- gozucito 7mo ago
  Could you please elaborate on what he was wrong about?
  - conradkay 7mo ago
    He said that LLMs wouldn't have common sense about how the real world physically works, because it's so obvious to humans that we don't bother putting it into text. This seems pretty foolish honestly given the scale of internet data, and even at the time LLMs could handle the example he said they couldn't I believe he didn't think that reasoning/CoT would work well or scale like it has
- ainch 7mo ago
  LeCun was stubbornly 'wrong and boneheaded' in the 80s, but turned out to be right. His contention now is that LLMs don't truly understand the physical world - I don't think we know enough yet to say whether he is wrong.
deleted 7mo ago
[deleted]

by jsnell 7mo ago

andrepd 7mo ago

It's been 6 months away for 5 years now. In that time we've seen relatively mild incremental changes, not any qualitative ones. It's probably not 6 months away.

basket_horse 7mo ago
But I swear this time is different! Just give me another 6 months!
- andrepd 7mo ago
  And another 6 trillion dollars :^)
AStrangeMorrow 7mo ago
Yeah. I feel like that like many projects the last 20% take 80% of time, and imho we are not in the last 20% Sure LLMs are getting better and better, and at least for me more and more useful, and more and more correct. Arguably better than humans at many tasks yet terribly lacking behind in some others. Coding wise, one of the things it does “best”, it still has many issues: For me still some of the biggest issues are still lack of initiative and lack of reliable memory. When I do use it to write code the first manifests for me by often sticking to a suboptimal yet overly complex approach quite often. And lack of memory in that I have to keep reminding it of edge cases (else it often breaks functionality), or to stop reinventing the wheel instead of using functions/classes already implemented in the project. All that can be mitigated by careful prompting, but no matter the claim about information recall accuracy I still find that even with that information in the prompt it is quite unreliable. And more generally the simple fact that when you talk to one the only way to “store” these memories is externally (ie not by updating the weights), is kinda like dealing with someone that can’t retain memories and has to keep writing things down to even get a small chance to cope. I get that updating the weights is possible in theory but just not practical, still.
lordmathis 7mo ago
It's 6 months away the same way coding is apparently "solved" now.
- HarHarVeryFunny 7mo ago
  I think we - in last few months - are very close to, if not already at, the point where "coding" is solved. That doesn't mean that software design or software engineering is solved, but it does mean that a SOTA model like GPT 5.4 or Opus 4.6 has a good chance of being able to code up a working version of whatever you specify, with reason. What's still missing is the general reasoning ability to plan what to build or how to attack novel problems - how to assess the consequences of deciding to build something a given way, and I doubt that auto-regressively trained LLMs is the way to get there, but there is a huge swathe of apps that are so boilerplate in nature that this isn't the limitation. I think that LeCun is on the right track to AGI with JEPA - hardly a unique insight, but significant to now have a well funded lab pursuing this approach. Whether they are successful, or timely, will depend if this startup executes as a blue skies research lab, or in more of an urgent engineering mode. I think at this point most of the things needed for AGI are more engineering challenges rather than what I'd consider as research problems.
  - lordmathis 7mo ago
    Sure, Claude and other SOTA LLMs do generate about 90% of my code but I feel like we are not closer to solving the last 10% than we were a year ago in the days of Claude 3.7. It can pretty reliably get 90% there and then I can either keep prompting it to get the rest done or just do it manually which is quite often faster.
    - j4k0o 7mo ago
      It's interesting that people don't seem to think the likely outcome might be... capital and labour. Not capital alone. You see this in construction - the capital is used for certain things and is operated by labour.
      - 7mo ago
mfru 7mo ago
Reminds me of how cold fusion reactors are only 5 years away for decades now
- vidarh 7mo ago
  Cold fusion reactors haven't produced usable intermediate results. LLMs have.
  - leptons 7mo ago
    LLMs produce slop far to often to say they are in any way better than cold fusion in terms of usable results. "AI" kind of is the cold fusion of tech. We've always been 5 or 10 years away from "AGI" and likely always will be.
    - vidarh 7mo ago
      That's just nonsense. That they produce slop does not negate that I and many others get plenty of value out of them in their current form, while we get zero value out of fusion so far - cold or otherwise.

next_xibalba 7mo ago

> RSI Wait, we have another acronym to track. Is this the same/different than AGI and/or ASI?

mietek 7mo ago
Some people should definitely be getting Repetitive Strain Injury from all the hyping up of LLMs.
notnullorvoid 7mo ago
Recursive Self Improvement
robrenaud 7mo ago
Recursive self improvement. It's when AI speeds up the development of the next AI.
deleted 7mo ago
[deleted]

Insanity 7mo ago

Whenever I see claims about AGI being reachable through large language models, it reminds me of the miasma theory of disease. Many respectable medical professionals were convinced this was true, and they viewed the entire world through this lens. They interpreted data in ways that aligned with a miasmatic view. Of course now we know this was delusional and it seems almost funny in retrospect. I feel the same way when I hear that 'just scale language models' suddenly created something that's true AGI, indistinguishable from human intelligence.

visarga 7mo ago
> Whenever I see claims about AGI being reachable through large language models, it reminds me of the miasma theory of disease. Whenever I see people think the model architecture matters much, I think they have a magical view of AI. Progress comes from high quality data, the models are good as they are now. Of course you can still improve the models, but you get much more upside from data, or even better - from interactive environments. The path to AGI is not based on pure thinking, it's based on scaling interaction. To remain in the same miasma theory of disease analogy, if you think architecture is the key, then look at how humans dealt with pandemics... Black Death in the 14th century killed half of Europe, and none could think of the germ theory of disease. Think about it - it was as desperate a situation as it gets, and none had the simple spark to keep hygiene. The fact is we are also not smart from the brain alone, we are smart from our experience. Interaction and environment are the scaffolds of intelligence, not the model. For example 1B users do more for an AI company than a better model, they act like human in the loop curators of LLM work.
- 0x3f 7mo ago
  If model arch doesn't matter much how come transformers changed everything?
  - visarga 7mo ago
    Luck. RNNs can do it just as good, Mamba, S4, etc - for a given budget of compute and data. The larger the model the less architecture makes a difference. It will learn in any of the 10,000 variations that have been tried, and come about 10-15% close to the best. What you need is a data loop, or a data source of exceptional quality and size, data has more leverage. Architecture games reflect more on efficiency, some method can be 10x more efficient than another.

dheera 7mo ago

Just because you raise 1 billion dollars to do X doesn't mean you can't pivot and do Y if it is in the best interest of your mission. I won't comment on Yann LeCun or his current technical strategy, but if you can avoid sunk cost fallacy and pivot nimbly I don't think it is bad for Europe at all. It is "1 billion dollars for an AI research lab", not "1 billion dollars to do X".

devonkelley 7mo ago

[flagged]

sebmellen 7mo ago
[flagged]

vidarh 7mo ago

It's sufficient to think that there is a chance that they will not be, however, for there to be a non-zero value to fund other approaches. And even if you think the chance is zero, unless you also think there is a zero chance they will be capable of pivoting quickly, it might still be beneficial. I think his views are largely flawed, but chances are there will still be lots of useful science coming out of it as well. Even if current architectures can achieve AGI, it does not mean there can't also be better, cheaper, more effective ways of doing the same things, and so exploring the space more broadly can still be of significant value.

Tenoke 7mo ago

I think LeCun has been so consistently wrong and boneheaded for basically all of the AI boom, that this is much, much more likely to be bad than good for Europe. Probably one of the worst people to give that much money to that can even raise it in the field.

gozucito 7mo ago
Could you please elaborate on what he was wrong about?
- conradkay 7mo ago
  He said that LLMs wouldn't have common sense about how the real world physically works, because it's so obvious to humans that we don't bother putting it into text. This seems pretty foolish honestly given the scale of internet data, and even at the time LLMs could handle the example he said they couldn't I believe he didn't think that reasoning/CoT would work well or scale like it has
ainch 7mo ago
LeCun was stubbornly 'wrong and boneheaded' in the 80s, but turned out to be right. His contention now is that LLMs don't truly understand the physical world - I don't think we know enough yet to say whether he is wrong.

deleted 7mo ago

[deleted]