3 ms·

I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally perform

by elo02 3mo ago

I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally performant if not better open weight models

teruakohatu 3mo ago
At this point in time what does meta get from releasing open weight models? Why devote the resources to it.
- gmerc 3mo ago
  Why would there be any motivated person left in this place.
- chrsw 3mo ago
  You can make the same argument for closed models. Why spend hundreds of billions training larger and larger models when you can just use Chinese models? Spend that money somewhere else further up the stack where there’s more value. Let China do the training since they’re so efficient at it.
  - szge 3mo ago
    as a big tech company you have the resources to make many bets and do a lot of things at the same time. it's good to have some specialists with knowledge of model training "just in case".
    - chrsw 3mo ago
      Even Meta, with all their resources, makes all their hardware in China. Manufacturing anywhere else is just burning cash.
  - Levitz 3mo ago
    Because without bigger players, Chinese models don't get anywhere. Playing catch-up is a radically different game.
    - chrsw 3mo ago
      [dead]
walrus01 3mo ago
The cynic in me says maybe they would be further along if they hadn't spent $80 billion on trying to build the "Metaverse" VR world. I've never met anyone who actually uses it and to the best of my knowledge it has very low mass market uptake. https://finance.yahoo.com/sectors/technology/articles/mark-zuckerbergs-80-billion-metaverse-000000105.html?guccounter=1 https://finance.yahoo.com/sectors/technology/articles/mark-z... Not exactly the best use of dollars and the labor hours of some of the best minds of our generation.
- trollbridge 3mo ago
  Particularly when you could vibe-code Metaverse VR for a lot less than $80bn.
gizmodo59 3mo ago
Because lack of talent and organizational disfunction matters a lot more than you think. The reason why OAI and Ant are always at the top is because of this and I’d say compute is third on the list.
- Aboutplants 3mo ago
  I would argue that they actually don’t lack talent, they have an insane bench of really smart people. What they lack is any sort of direction and leadership. They are a ship lost in the ocean and up until now have been lucky to find a few treasures along the their way.
  - nicce 3mo ago
    > they have an insane bench of really smart people. Filtered heavily into those who care only about money. Many don’t want to work there. Smart people have other choices.
    - foobar_______ 3mo ago
      Thank you. I agree. I hope this is the new learned sentiment for Meta employees. Smart? Check. Money-obsessed and soulless? Check.
bearjaws 3mo ago
because imagine starting work every week and finding that your dumb ass CEO pivoted the company again and is ruining other peoples lives, and he then reorgs the management again so you now have your 5th leader this year. Morale and momentum are huge things in companies, Zuck has been murdering both of those in Meta since... well naming it Meta.
nl 3mo ago
The SemiAnalysis piece on this is long but very much worth reading: > The company appears burdened by far too many disparate groups that are over-optimizing for certain metrics as opposed to delivering usable technology for the company as a whole. > And because Meta has a reputation for throwing money at problems and executing at high speed, these U-turns end up becoming more costly versus other companies that take a more disciplined or conservative approach. Suppliers also lose faith when given design wins are later cancelled. This has lead to less supply chain prioritization on new designs. Some suppliers favor focusing on Amazon or Google designs due to Meta’s frequent reshuffling. > Few inside Meta’s chip division have a full understanding of why the company bought Rivos in the first place, and those who championed the deal internally have since gone quiet. etc etc It goes into a lot of depth. https://newsletter.semianalysis.com/p/metas-infrastructure-team-needs-a https://newsletter.semianalysis.com/p/metas-infrastructure-t...
- dominotw 3mo ago
  he seems to be bullish on meta though and considers muse on slope than an intercept https://newsletter.semianalysis.com/p/the-future-of-meta-superintelligence https://newsletter.semianalysis.com/p/the-future-of-meta-sup...
mmaunder 3mo ago
Turns out sitting quietly in a room and doing math is worth more than all the money and network in the world.
anabis 3mo ago
I heard Muse Spark 1.1 isn't too shabby. Since Meta compute >> Chinese labs, we'll see how it goes from here.

by elo02 3mo ago

I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally performant if not better open weight models

teruakohatu 3mo ago

At this point in time what does meta get from releasing open weight models? Why devote the resources to it.

gmerc 3mo ago
Why would there be any motivated person left in this place.
chrsw 3mo ago
You can make the same argument for closed models. Why spend hundreds of billions training larger and larger models when you can just use Chinese models? Spend that money somewhere else further up the stack where there’s more value. Let China do the training since they’re so efficient at it.
- szge 3mo ago
  as a big tech company you have the resources to make many bets and do a lot of things at the same time. it's good to have some specialists with knowledge of model training "just in case".
  - chrsw 3mo ago
    Even Meta, with all their resources, makes all their hardware in China. Manufacturing anywhere else is just burning cash.
- Levitz 3mo ago
  Because without bigger players, Chinese models don't get anywhere. Playing catch-up is a radically different game.
  - chrsw 3mo ago
    [dead]

walrus01 3mo ago

The cynic in me says maybe they would be further along if they hadn't spent $80 billion on trying to build the "Metaverse" VR world. I've never met anyone who actually uses it and to the best of my knowledge it has very low mass market uptake. https://finance.yahoo.com/sectors/technology/articles/mark-zuckerbergs-80-billion-metaverse-000000105.html?guccounter=1 https://finance.yahoo.com/sectors/technology/articles/mark-z... Not exactly the best use of dollars and the labor hours of some of the best minds of our generation.

trollbridge 3mo ago
Particularly when you could vibe-code Metaverse VR for a lot less than $80bn.

gizmodo59 3mo ago

Because lack of talent and organizational disfunction matters a lot more than you think. The reason why OAI and Ant are always at the top is because of this and I’d say compute is third on the list.

Aboutplants 3mo ago
I would argue that they actually don’t lack talent, they have an insane bench of really smart people. What they lack is any sort of direction and leadership. They are a ship lost in the ocean and up until now have been lucky to find a few treasures along the their way.
- nicce 3mo ago
  > they have an insane bench of really smart people. Filtered heavily into those who care only about money. Many don’t want to work there. Smart people have other choices.
  - foobar_______ 3mo ago
    Thank you. I agree. I hope this is the new learned sentiment for Meta employees. Smart? Check. Money-obsessed and soulless? Check.

bearjaws 3mo ago

because imagine starting work every week and finding that your dumb ass CEO pivoted the company again and is ruining other peoples lives, and he then reorgs the management again so you now have your 5th leader this year. Morale and momentum are huge things in companies, Zuck has been murdering both of those in Meta since... well naming it Meta.

nl 3mo ago

The SemiAnalysis piece on this is long but very much worth reading: > The company appears burdened by far too many disparate groups that are over-optimizing for certain metrics as opposed to delivering usable technology for the company as a whole. > And because Meta has a reputation for throwing money at problems and executing at high speed, these U-turns end up becoming more costly versus other companies that take a more disciplined or conservative approach. Suppliers also lose faith when given design wins are later cancelled. This has lead to less supply chain prioritization on new designs. Some suppliers favor focusing on Amazon or Google designs due to Meta’s frequent reshuffling. > Few inside Meta’s chip division have a full understanding of why the company bought Rivos in the first place, and those who championed the deal internally have since gone quiet. etc etc It goes into a lot of depth. https://newsletter.semianalysis.com/p/metas-infrastructure-team-needs-a https://newsletter.semianalysis.com/p/metas-infrastructure-t...

dominotw 3mo ago
he seems to be bullish on meta though and considers muse on slope than an intercept https://newsletter.semianalysis.com/p/the-future-of-meta-superintelligence https://newsletter.semianalysis.com/p/the-future-of-meta-sup...

mmaunder 3mo ago

Turns out sitting quietly in a room and doing math is worth more than all the money and network in the world.

anabis 3mo ago

I heard Muse Spark 1.1 isn't too shabby. Since Meta compute >> Chinese labs, we'll see how it goes from here.