5 ms·

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead th

by sy26 9mo ago

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead they let go Yann LeCun.

phailhaus 9mo ago
Most people don't like putting on VR headsets, no matter what the content is. It just never broke out of the tech enthusiast niche.
observationist 9mo ago
LeCun wasn't producing results. He was obstinate and insistent on his own theories and ideas which weren't, and possibly aren't, going anywhere. He refused to engage with LLMs and compete in the market that exists, and spent all his effort and energy on unproven ideas and research, which split the company's mission and competitiveness. They lost their place as one of the top 4 AI companies, and are now a full generation behind, in part due to the split efforts and lack of enthusiastic participation by all the Meta AI team. If you look at the chaos and churn at the highest levels across the industry, there's not a lot of room for mission creep by leadership, and LeCun thoroughly demonstrated he wasn't suited for the mission desired by Meta. I think he's lucky he got out with his reputation relatively intact.
- halfmatthalfcat 9mo ago
  Were you there or just an attentive outsider?
  - observationist 9mo ago
    Attentive outsider and acquaintance of a couple people who are or were employed there. Nothing I'm saying is particularly inside baseball, though, it's pretty well covered by all the blogs and podcasts.
    - richard___ 9mo ago
      What podcast?
      - observationist 9mo ago
        Machine Learning Street Talk and Dwarkesh are excellent. Various discord communities, forums, and blogs downstream of the big podcasts, and following researchers on X keeps you in the loop on a lot of these things, and then you can watch for random interviews and presentations on youtube when you know who the interesting people and subjects are.
  - qwertyi0k 9mo ago
    Most serious researchers want to work on interesting problems like reinforcement learning or robotics or RNN or dozen other avant-garde subjects. None want to work on "boring" LLM technology, requiring significant engineering effort and huge dataset wrangling effort.
    - observationist 9mo ago
      This is true - Ilya got an exit and is engaged in serious research, but research is by its nature unpredictable. Meta wanted a product and to compete in the AI market, and JEPA was incompatible with that. Now LeCun has a lab and resources to pursue his research, and Meta has refocused efforts on LLMs and the marketplace - it remains to be seen if they'll be able to regain their position. I hope they do - open models and relatively open research are important, and the more serious AI labs that do this, the more it incentivizes others to do the same, and keeps the ones that have committed to it honest.
- ezst 9mo ago
  Since a hot take is as good as the next one: LLMs are by the day more and more clearly understood as a "local maximum" with flawed capabilities, limited efficiency, a $trillion + a large chunk of the USA's GDP wasted, nobody even turning a profit from that nor able to build something that can't be reproduced for free within 6 months. When the right move (strategically, economically) is to not compete, the head of the AI division acknowledging the above and deciding to focus on the next breakthrough seems absolutely reasonable.
  - throw310822 9mo ago
    You really need to be obstinate in your convictions if you can dismiss LLMs at the time when everyone's job is being turned around by them. Everywhere I look, everyone I talk to, is using LLMs more and more to do their job and dramatically increase their productivity. It's one of the most successful technologies I've ever witnessed arriving on the market, and it's only just started- it's just three years old.
    - acedTrex 9mo ago
      What are you seeing people do with it? To my eyes everyone is in the same amount of meetings lol.
      - throw310822 9mo ago
        For one, since last month, AI is writing about 95% of my code and that of my colleagues. I just describe what I want and how it should be implemented and the AI takes care of all the details, solves bugs, configuration issues, etc. Also I use it to research libraries, dig into documentation (and then write implementations based on that), discuss architectural alternatives, etc. Non-developers I know use them to organise meetings, write emails, research companies, write down and summarise counselling sessions (not the clients, the counselor), write press reports, help with advertising campaigns management, review complex commercial insurance policies, fix translations... The list of uses is endless, really. And I'm only talking of work-related usage, personal usage goes of course well beyond this.
- qwertyi0k 9mo ago
  To be fair, this was his job description: Fundamental AI Research (FAIR) lab. Not AI products division. You can't expect marketable products from a fundamental AI research lab.
  - YetAnotherNick 9mo ago
    It's "Facebook Artificial Intelligence Research", not fundamental. So basically involves both fundamental and applied research. [1]: https://engineering.fb.com/category/ai-research/ https://engineering.fb.com/category/ai-research/
    - qwertyi0k 9mo ago
      Ref: Yann lecun post on linkedin, 3years ago: FAIR now stand for "Fundamental AI Research"
      - YetAnotherNick 9mo ago
        I literally linked the official site and it currently says Facebook. I have known FAIR for many years and I have always know it as Facebook. Can you link any official source of changing the name.
        stephenjayakar 9mo ago
        I gotcha. https://www.linkedin.com/posts/yann-lecun_big-changes-for-ai-rd-at-meta-fair-remains-activity-6938183698311766016-2Yh0 https://www.linkedin.com/posts/yann-lecun_big-changes-for-ai..., https://ai.meta.com/blog/fair-10-year-anniversary-open-science-meta/ https://ai.meta.com/blog/fair-10-year-anniversary-open-scien...
- mapmeld 9mo ago
  In an industry of big bets, especially considering the company has poured resources and renamed itself to secure a place in the VR world... staking your reputation on everyone's LLMs having peaked and shifting focus to finding a new path to AI is a pretty interesting bet, no?
- _zagj 9mo ago
  This sounds similar to the arc of Carpathy, who also managed to preserve his reputation despite sending Tesla down a FSD deadend and missing the initial LLM boat.
- Der_Einzige 9mo ago
  It's insane that you can argue this in a world where facebook continues to be state of the art (and it's not even close) on semantic segmentation. Those SAM models they produce deliver more value than a hypothetical competitive llama5 model coming out tomorrow. I'm banning my wife from ever buying any Alexander Wang clothing, because his leadership is so poor in comparison that he's going to also devalue the name-collision fashion brand that he shares a name with. That's how bad his leadership is going to be in comparison to Yann. Scale AI was only successful for the same reason Langchain was. Easy to be a big fish in a pond with no other fishes.
qwertox 9mo ago
Isn't it more like this: JEPA looks at the video, "a dog walks out of the door, the mailman comes, dog is happy" and the next frame would need to look like "mailman must move to mailbox, dog will run happily towards him", which then an image/video generator would need to render. Genie looks at the video, "when this group of pixels looks like this and the user presses 'jump', I will render the group different in this way in the next frame." Genie is an artist drawing a flipbook. To tell you what happens next, it must draw the page. If it doesn't draw it, the story doesn't exist. JEPA is a novelist writing a summary. To tell you what happens next, it just writes "The car crashes." It doesn't need to describe what the twisted metal looks like to know the crash happened.
- deleted 9mo ago
  [deleted]
general_reveal 9mo ago
You are beyond correct. World models is what saves their Reality Labs investment. I would say if Reality Labs cannot productize World Models, then that entire project needs to be scrapped.
slashdave 9mo ago
Failures are not publicly reported, in general. Do you we know what they have invested in?
energy123 9mo ago
Is Project Genie a "world model" as defined by Yann LeCun? Doesn't "world model" mean that the model generates things from a theory of the world, rather than the colloquial meaning of generating 3d navigable scenes (using a temporal ViT or whatever)?

by sy26 9mo ago

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead they let go Yann LeCun.

phailhaus 9mo ago

Most people don't like putting on VR headsets, no matter what the content is. It just never broke out of the tech enthusiast niche.

observationist 9mo ago

LeCun wasn't producing results. He was obstinate and insistent on his own theories and ideas which weren't, and possibly aren't, going anywhere. He refused to engage with LLMs and compete in the market that exists, and spent all his effort and energy on unproven ideas and research, which split the company's mission and competitiveness. They lost their place as one of the top 4 AI companies, and are now a full generation behind, in part due to the split efforts and lack of enthusiastic participation by all the Meta AI team. If you look at the chaos and churn at the highest levels across the industry, there's not a lot of room for mission creep by leadership, and LeCun thoroughly demonstrated he wasn't suited for the mission desired by Meta. I think he's lucky he got out with his reputation relatively intact.

halfmatthalfcat 9mo ago
Were you there or just an attentive outsider?
- observationist 9mo ago
  Attentive outsider and acquaintance of a couple people who are or were employed there. Nothing I'm saying is particularly inside baseball, though, it's pretty well covered by all the blogs and podcasts.
  - richard___ 9mo ago
    What podcast?
    - observationist 9mo ago
      Machine Learning Street Talk and Dwarkesh are excellent. Various discord communities, forums, and blogs downstream of the big podcasts, and following researchers on X keeps you in the loop on a lot of these things, and then you can watch for random interviews and presentations on youtube when you know who the interesting people and subjects are.
- qwertyi0k 9mo ago
  Most serious researchers want to work on interesting problems like reinforcement learning or robotics or RNN or dozen other avant-garde subjects. None want to work on "boring" LLM technology, requiring significant engineering effort and huge dataset wrangling effort.
  - observationist 9mo ago
    This is true - Ilya got an exit and is engaged in serious research, but research is by its nature unpredictable. Meta wanted a product and to compete in the AI market, and JEPA was incompatible with that. Now LeCun has a lab and resources to pursue his research, and Meta has refocused efforts on LLMs and the marketplace - it remains to be seen if they'll be able to regain their position. I hope they do - open models and relatively open research are important, and the more serious AI labs that do this, the more it incentivizes others to do the same, and keeps the ones that have committed to it honest.
ezst 9mo ago
Since a hot take is as good as the next one: LLMs are by the day more and more clearly understood as a "local maximum" with flawed capabilities, limited efficiency, a $trillion + a large chunk of the USA's GDP wasted, nobody even turning a profit from that nor able to build something that can't be reproduced for free within 6 months. When the right move (strategically, economically) is to not compete, the head of the AI division acknowledging the above and deciding to focus on the next breakthrough seems absolutely reasonable.
- throw310822 9mo ago
  You really need to be obstinate in your convictions if you can dismiss LLMs at the time when everyone's job is being turned around by them. Everywhere I look, everyone I talk to, is using LLMs more and more to do their job and dramatically increase their productivity. It's one of the most successful technologies I've ever witnessed arriving on the market, and it's only just started- it's just three years old.
  - acedTrex 9mo ago
    What are you seeing people do with it? To my eyes everyone is in the same amount of meetings lol.
    - throw310822 9mo ago
      For one, since last month, AI is writing about 95% of my code and that of my colleagues. I just describe what I want and how it should be implemented and the AI takes care of all the details, solves bugs, configuration issues, etc. Also I use it to research libraries, dig into documentation (and then write implementations based on that), discuss architectural alternatives, etc. Non-developers I know use them to organise meetings, write emails, research companies, write down and summarise counselling sessions (not the clients, the counselor), write press reports, help with advertising campaigns management, review complex commercial insurance policies, fix translations... The list of uses is endless, really. And I'm only talking of work-related usage, personal usage goes of course well beyond this.
qwertyi0k 9mo ago
To be fair, this was his job description: Fundamental AI Research (FAIR) lab. Not AI products division. You can't expect marketable products from a fundamental AI research lab.
- YetAnotherNick 9mo ago
  It's "Facebook Artificial Intelligence Research", not fundamental. So basically involves both fundamental and applied research. [1]: https://engineering.fb.com/category/ai-research/ https://engineering.fb.com/category/ai-research/
  - qwertyi0k 9mo ago
    Ref: Yann lecun post on linkedin, 3years ago: FAIR now stand for "Fundamental AI Research"
    - YetAnotherNick 9mo ago
      I literally linked the official site and it currently says Facebook. I have known FAIR for many years and I have always know it as Facebook. Can you link any official source of changing the name.
      - stephenjayakar 9mo ago
        I gotcha. https://www.linkedin.com/posts/yann-lecun_big-changes-for-ai-rd-at-meta-fair-remains-activity-6938183698311766016-2Yh0 https://www.linkedin.com/posts/yann-lecun_big-changes-for-ai..., https://ai.meta.com/blog/fair-10-year-anniversary-open-science-meta/ https://ai.meta.com/blog/fair-10-year-anniversary-open-scien...
mapmeld 9mo ago
In an industry of big bets, especially considering the company has poured resources and renamed itself to secure a place in the VR world... staking your reputation on everyone's LLMs having peaked and shifting focus to finding a new path to AI is a pretty interesting bet, no?
_zagj 9mo ago
This sounds similar to the arc of Carpathy, who also managed to preserve his reputation despite sending Tesla down a FSD deadend and missing the initial LLM boat.
Der_Einzige 9mo ago
It's insane that you can argue this in a world where facebook continues to be state of the art (and it's not even close) on semantic segmentation. Those SAM models they produce deliver more value than a hypothetical competitive llama5 model coming out tomorrow. I'm banning my wife from ever buying any Alexander Wang clothing, because his leadership is so poor in comparison that he's going to also devalue the name-collision fashion brand that he shares a name with. That's how bad his leadership is going to be in comparison to Yann. Scale AI was only successful for the same reason Langchain was. Easy to be a big fish in a pond with no other fishes.

qwertox 9mo ago

Isn't it more like this: JEPA looks at the video, "a dog walks out of the door, the mailman comes, dog is happy" and the next frame would need to look like "mailman must move to mailbox, dog will run happily towards him", which then an image/video generator would need to render. Genie looks at the video, "when this group of pixels looks like this and the user presses 'jump', I will render the group different in this way in the next frame." Genie is an artist drawing a flipbook. To tell you what happens next, it must draw the page. If it doesn't draw it, the story doesn't exist. JEPA is a novelist writing a summary. To tell you what happens next, it just writes "The car crashes." It doesn't need to describe what the twisted metal looks like to know the crash happened.

deleted 9mo ago
[deleted]

general_reveal 9mo ago

You are beyond correct. World models is what saves their Reality Labs investment. I would say if Reality Labs cannot productize World Models, then that entire project needs to be scrapped.

slashdave 9mo ago

Failures are not publicly reported, in general. Do you we know what they have invested in?

energy123 9mo ago

Is Project Genie a "world model" as defined by Yann LeCun? Doesn't "world model" mean that the model generates things from a theory of the world, rather than the colloquial meaning of generating 3d navigable scenes (using a temporal ViT or whatever)?