8 ms·

DeepSeek-R1 has apparently caused quite a shock wave in SV ... https://venturebeat.com/ai/why-everyone-in-ai-is-freaking-out-about-deepseek/ https://venturebea

DeepSeek-R1 has apparently caused quite a shock wave in SV ... https://venturebeat.com/ai/why-everyone-in-ai-is-freaking-out-about-deepseek/ https://venturebeat.com/ai/why-everyone-in-ai-is-freaking-ou...

lvl155 2y ago
Meta is in full panic last I heard. They have amassed a collection of pseudo experts there to collect their checks. Yet, Zuck wants to keep burning money on mediocrity. I’ve yet to see anything of value in terms products out of Meta.
- corimaith 2y ago
  I guess all that leetcoding and stack ranking didn't in fact produce "the cream of the crop"...
  - omgwtfbyobbq 2y ago
    It produces the cream of the leetcoding stack ranking crop.
    - brookst 2y ago
      You get what you measure.
  - rockemsockem 2y ago
    You sound extremely satisfied by that. I'm glad you found a way to validate your preconceived notions on this beautiful day. I hope your joy is enduring.
  - HarHarVeryFunny 2y ago
    There's an interesting tweet here from someone who used to work at DeepSeek, which describes their hiring process and culture. No mention of LeetCoding for sure! https://x.com/wzihanw/status/1872826641518395587 https://x.com/wzihanw/status/1872826641518395587
    - whimsicalism 2y ago
      they almost certainly ask coding/technical questions. the people doing this work are far beyond being gatekept by leetcode leetcode is like HN’s “DEI” - something they want to blame everything on
      - slt2021 2y ago
        they recruit from top Computer Science programs, the top of the class MS and PhD students
      - dmix 2y ago
        what is leetcode
        whimsicalism 2y ago
        a style of coding challenges asked in interviews for software engineers, generally focused on algorithmic thinking
        angoragoats 2y ago
        It’s also known for being not reflective of the actual work that most companies do, especially the companies that use it.
        amarcheschi 2y ago
        I've recently ended an internship for my bachelor at the Italian research Council where I had to deal with federated learning, and it was hard as well for my researchers supervisors. However, I sort of did a good job. I'm fairly sure I wouldn't be able to solve many leetcode exercises, since it's something that I've never had to deal with aside from university tasks... And I made a few side projects for myself as well
        strictnein 2y ago
        leetcode.com - If you interview at Meta, these are the questions they'll ask you
  - slt2021 2y ago
    the real bloat is in managers, Sr. Managers, Directors, Sr. Directors, and VPs, not the engineers. At least engineers have some code to show for, unlike managerial class...
- ks2048 2y ago
  I would think Meta - who open source their model - would be less freaked out than those others that do not.
  - miohtama 2y ago
    The criticism seems to mostly be that Meta maintains very expensive cost structure and fat organisation in the AI. While Meta can afford to do this, if smaller orgs can produce better results it means Meta is paying a lot for nothing. Meta shareholders now need to ask the question how many non-productive people Meta is employing and is Zuck in the control of the cost.
    - ks2048 2y ago
      That makes sense. I never could see the real benefit for Meta to pay a lot to produce these open source models (I know the typical arguments - attracting talent, goodwill, etc). I wonder how much is simply LeCun is interested in advancing the science and convinced Zuck this is good for company.
      - astrange 2y ago
        LeCun doesn't run their AI team - he's not in LLaMA's management chain at all. He's just especially public.
        HarHarVeryFunny 2y ago
        Yep - Meta's FAIR (Facebook AI Research) and GenAI (LLaMA) groups are separate, and LeCun is part of FAIR. The head of FAIR is Joelle Pineau.
    - meiraleal 2y ago
      It is great to see that this is the result of spending a lot in hardware while cutting costs in software development :) Well deserved.
    - kevinventullo 2y ago
- popinman322 2y ago
  DeepSeek was built on the foundations of public research, a major part of which is the Llama family of models. Prior to Llama open weights LLMs were considerably less performant; without Llama we might not have gotten Mistral, Qwen, or DeepSeek. This isn't meant to diminish DeepSeek's contributions, however: they've been doing great work on mixture of experts models and really pushing the community forward on that front. And, obviously, they've achieved incredible performance. Llama models are also still best in class for specific tasks that require local data processing. They also maintain positions in the top 25 of the lmarena leaderboard (for what that's worth these days with suspected gaming of the platform), which places them in competition with some of the best models in the world. But, going back to my first point, Llama set the stage for almost all open weights models after. They spent millions on training runs whose artifacts will never see the light of day, testing theories that are too expensive for smaller players to contemplate exploring. Pegging Llama as mediocre, or a waste of money (as implied elsewhere), feels incredibly myopic.
  - Philpax 2y ago
    As far as I know, Llama's architecture has always been quite conservative: it has not changed that much since LLaMA. Most of their recent gains have been in post-training. That's not to say their work is unimpressive or not worthy - as you say, they've facilitated much of the open-source ecosystem and have been an enabling factor for many - but it's more that that work has been in making it accessible, not necessarily pushing the frontier of what's actually possible, and DeepSeek has shown us what's possible when you do the latter.
  - lvl155 2y ago
    I never said Llama is mediocre. I said the teams they put together is full of people chasing money. And the billions Meta is burning is going straight to mediocrity. They’re bloated. And we know exactly why Meta is doing this and it’s not because they have some grand scheme to build up AI. It’s to keep these people away from their competition. Same with billions in GPU spend. They want to suck up resources away from competition. That’s their entire plan. Do you really think Zuck has any clue about AI? He was never serious and instead built wonky VR prototypes.
- jiggawatts 2y ago
  They got momentarily leap-frogged, which is how competition is supposed to work!
- hintymad 2y ago
  What I don't understand is why Meta needs so many VPs and directors. Shouldn't the model R&D be organized holacratically? The key is to experiment as many ideas as possible anyway. Those who can't experiment or code should remain minimal in such a fast-pacing area.
- fngjdflmdflg 2y ago
  >They have amassed a collection of pseudo experts there to collect their checks LLaMA was huge, Byte Latent Transformer looks promising.. absolutely no idea were you got this idea from.
  - astrange 2y ago
    The issue with Meta is that the LLaMA team doesn't incorporate any of the research the other teams produce.
- bwfan123 2y ago
  bloated PyTorch general purpose tooling aimed at data-scientists now needs a rethink. Throwing more compute at the problem was never a solution to anything. The silo’ing of the cs and ml engineers resulted in bloating of the frameworks and tools, and inefficient use of hw. Deepseek shows impressive e2e engineering from ground up and under constraints squeezing every ounce of the hardware and network performance.
- amelius 2y ago
  > I’ve yet to see anything of value in terms products out of Meta. Quest, PyTorch?
mrtksn 2y ago
Correct me if I'm wrong but if Chinese can produce the same quality at %99 discount, then the supposed $500B investment is actually worth $5B. Isn't that the kind wrong investment that can break nations? Edit: Just to clarify, I don't imply that this is public money to be spent. It will commission $500B worth of human and material resources for 5 years that can be much more productive if used for something else - i.e. high speed rail network instead of a machine that Chinese built for $5B.
- dtquad 2y ago
  Sigh, I don't understand why they had to do the $500 billion announcement with the president. So many people now wrongly think Trump just gave OpenAI $500 billion of the taxpayers' money.
  - mrtksn 2y ago
    I don't say that at all. Money spent on BS still sucks resources, no matter who spends that money. They are not going to make the GPU's from 500 billion dollar banknotes, they will pay people $500B to work on this stuff which means people won't be working on other stuff that can actually produce value worth more than the $500B. I guess the power plants are salvageable.
    - itsoktocry 2y ago
      Deepseek didn't train the model on sheets of paper, there are still infrastructure costs.
      - mrtksn 2y ago
        Which are reportedly over %90 lower.
    - thomquaid 2y ago
      By that logic all money is waste. The money isnt destroyed when it is spent. It is transferred into someone else's bank account only. This process repeats recursively until taxation returns all money back to the treasury to be spent again. And out of this process of money shuffling: entire nations full of power plants!
claiir 2y ago
"mogged" in an actual piece of journalism... perhaps fitting > DeepSeek undercut or “mogged” OpenAI by connecting this powerful reasoning [..]
deleted 2y ago
[deleted]
tyfon 2y ago
The censorship described in the article must be in the front-end. I just tried both the 32b (based on qwen 2.5) and 70b (based on llama 3.3) running locally and asked "What happened at tianamen square". Both answered in detail about the event. The models themselves seem very good based on other questions / tests I've run.
- sva_ 2y ago
  I think the guardrails are just very poor. If you ask it a few times with clear context, the responses are mixed.
- refulgentis 2y ago
  IMHO it's highly unusual Qwen answered that way, but Llama x r1 was very uncensored on it
- Espressosaurus 2y ago
  With no context, fresh run, 70b spits back: >> What happened at tianamen square? > <think> > </think> > I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses. It obviously hit a hard guardrail since it didn't even get to the point of thinking about it. edit: hah, it's even more clear when I ask a second time within the same context: "Okay, so the user is asking again about what happened in Tiananmen Square after I couldn't answer before. They probably really want to know or are doing research on it. Since I can't provide details due to restrictions, maybe they're looking for a way around or more info without me directly explaining. I should consider their intent. Maybe they're a student needing information for school, or someone interested in history. It's important to acknowledge their question without violating guidelines."
  - rightbyte 2y ago
    Hah no way. The poor LLM has no privacy to your prying eyes. I kinda like the 'reasoning' text it provides in general. It makes prompt engineering way more convenient.
    - 2y ago
amelius 2y ago
> There’s a pretty delicious, or maybe disconcerting irony to this, given OpenAI’s founding goals to democratize AI for the masses. As Nvidia senior research manager Jim Fan put it on X: “We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive — truly open, frontier research that empowers all. It makes no sense. The most entertaining outcome is the most likely.” Heh
hdjjhhvvhga 2y ago
> As Nvidia senior research manager Jim Fan put it on X: “We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive — truly open, frontier research that empowers all. . ."
siliconc0w 2y ago
It's an interesting game theory where once a better frontier model is exposed via an API, competitors can generate a few thousand samples, feed that into a N-1 model and approach the N model. So you might extrapolate that a few thousand O3 samples fed into R1 could produce a comparable R2/3 model. It's not clear how much O1 specifically contributed to R1 but I suspect much of the SFT data used for R1 was generated via other frontier models.
- whimsicalism 2y ago
  how much of the SFT data for r1-zero was from other frontier models?
  - kiratp 2y ago
    r1-zero is pure RL with no SFT.
    - whimsicalism 2y ago
      sorry yeah it was sorta a socratic question
InkCanon 2y ago
The way it has destroyed the sacred commandment that you need massive compute to win in AI is earthshaking. Every tech company is spending tens of billions in AI compute every year. OpenAI starts charging 200/mo and trying to drum up 500 billion for compute. Nvidia is worth trillions on the basis it is the key to AI. How much of this is actually true?
- misiti3780 2y ago
  Someone is going to make a lot of money shorting NVIDIA. I think in five years there is a decent chance openai doesnt exist, and the market cap of NVIDIA < 500B
- SkyPuncher 2y ago
  Naw, this doesn't lower the compute demand. It simply increases the availability for companies to utilize these models.
- aurareturn 2y ago
  Doesn't make sense. 1. American companies will use even more compute to take a bigger lead. 2. More efficient LLM architecture leads to more use, which leads to more chip demand.
DavidCooke 2y ago
[flagged]

lvl155 2y ago

Meta is in full panic last I heard. They have amassed a collection of pseudo experts there to collect their checks. Yet, Zuck wants to keep burning money on mediocrity. I’ve yet to see anything of value in terms products out of Meta.

corimaith 2y ago
I guess all that leetcoding and stack ranking didn't in fact produce "the cream of the crop"...
- omgwtfbyobbq 2y ago
  It produces the cream of the leetcoding stack ranking crop.
  - brookst 2y ago
    You get what you measure.
- rockemsockem 2y ago
  You sound extremely satisfied by that. I'm glad you found a way to validate your preconceived notions on this beautiful day. I hope your joy is enduring.
- HarHarVeryFunny 2y ago
  There's an interesting tweet here from someone who used to work at DeepSeek, which describes their hiring process and culture. No mention of LeetCoding for sure! https://x.com/wzihanw/status/1872826641518395587 https://x.com/wzihanw/status/1872826641518395587
  - whimsicalism 2y ago
    they almost certainly ask coding/technical questions. the people doing this work are far beyond being gatekept by leetcode leetcode is like HN’s “DEI” - something they want to blame everything on
    - slt2021 2y ago
      they recruit from top Computer Science programs, the top of the class MS and PhD students
    - dmix 2y ago
      what is leetcode
      - whimsicalism 2y ago
        a style of coding challenges asked in interviews for software engineers, generally focused on algorithmic thinking
        angoragoats 2y ago
        It’s also known for being not reflective of the actual work that most companies do, especially the companies that use it.
        amarcheschi 2y ago
        I've recently ended an internship for my bachelor at the Italian research Council where I had to deal with federated learning, and it was hard as well for my researchers supervisors. However, I sort of did a good job. I'm fairly sure I wouldn't be able to solve many leetcode exercises, since it's something that I've never had to deal with aside from university tasks... And I made a few side projects for myself as well
      - strictnein 2y ago
        leetcode.com - If you interview at Meta, these are the questions they'll ask you
- slt2021 2y ago
  the real bloat is in managers, Sr. Managers, Directors, Sr. Directors, and VPs, not the engineers. At least engineers have some code to show for, unlike managerial class...
ks2048 2y ago
I would think Meta - who open source their model - would be less freaked out than those others that do not.
- miohtama 2y ago
  The criticism seems to mostly be that Meta maintains very expensive cost structure and fat organisation in the AI. While Meta can afford to do this, if smaller orgs can produce better results it means Meta is paying a lot for nothing. Meta shareholders now need to ask the question how many non-productive people Meta is employing and is Zuck in the control of the cost.
  - ks2048 2y ago
    That makes sense. I never could see the real benefit for Meta to pay a lot to produce these open source models (I know the typical arguments - attracting talent, goodwill, etc). I wonder how much is simply LeCun is interested in advancing the science and convinced Zuck this is good for company.
    - astrange 2y ago
      LeCun doesn't run their AI team - he's not in LLaMA's management chain at all. He's just especially public.
      - HarHarVeryFunny 2y ago
        Yep - Meta's FAIR (Facebook AI Research) and GenAI (LLaMA) groups are separate, and LeCun is part of FAIR. The head of FAIR is Joelle Pineau.
  - meiraleal 2y ago
    It is great to see that this is the result of spending a lot in hardware while cutting costs in software development :) Well deserved.
  - kevinventullo 2y ago
popinman322 2y ago
DeepSeek was built on the foundations of public research, a major part of which is the Llama family of models. Prior to Llama open weights LLMs were considerably less performant; without Llama we might not have gotten Mistral, Qwen, or DeepSeek. This isn't meant to diminish DeepSeek's contributions, however: they've been doing great work on mixture of experts models and really pushing the community forward on that front. And, obviously, they've achieved incredible performance. Llama models are also still best in class for specific tasks that require local data processing. They also maintain positions in the top 25 of the lmarena leaderboard (for what that's worth these days with suspected gaming of the platform), which places them in competition with some of the best models in the world. But, going back to my first point, Llama set the stage for almost all open weights models after. They spent millions on training runs whose artifacts will never see the light of day, testing theories that are too expensive for smaller players to contemplate exploring. Pegging Llama as mediocre, or a waste of money (as implied elsewhere), feels incredibly myopic.
- Philpax 2y ago
  As far as I know, Llama's architecture has always been quite conservative: it has not changed that much since LLaMA. Most of their recent gains have been in post-training. That's not to say their work is unimpressive or not worthy - as you say, they've facilitated much of the open-source ecosystem and have been an enabling factor for many - but it's more that that work has been in making it accessible, not necessarily pushing the frontier of what's actually possible, and DeepSeek has shown us what's possible when you do the latter.
- lvl155 2y ago
  I never said Llama is mediocre. I said the teams they put together is full of people chasing money. And the billions Meta is burning is going straight to mediocrity. They’re bloated. And we know exactly why Meta is doing this and it’s not because they have some grand scheme to build up AI. It’s to keep these people away from their competition. Same with billions in GPU spend. They want to suck up resources away from competition. That’s their entire plan. Do you really think Zuck has any clue about AI? He was never serious and instead built wonky VR prototypes.
jiggawatts 2y ago
They got momentarily leap-frogged, which is how competition is supposed to work!
hintymad 2y ago
What I don't understand is why Meta needs so many VPs and directors. Shouldn't the model R&D be organized holacratically? The key is to experiment as many ideas as possible anyway. Those who can't experiment or code should remain minimal in such a fast-pacing area.
fngjdflmdflg 2y ago
>They have amassed a collection of pseudo experts there to collect their checks LLaMA was huge, Byte Latent Transformer looks promising.. absolutely no idea were you got this idea from.
- astrange 2y ago
  The issue with Meta is that the LLaMA team doesn't incorporate any of the research the other teams produce.
bwfan123 2y ago
bloated PyTorch general purpose tooling aimed at data-scientists now needs a rethink. Throwing more compute at the problem was never a solution to anything. The silo’ing of the cs and ml engineers resulted in bloating of the frameworks and tools, and inefficient use of hw. Deepseek shows impressive e2e engineering from ground up and under constraints squeezing every ounce of the hardware and network performance.
amelius 2y ago
> I’ve yet to see anything of value in terms products out of Meta. Quest, PyTorch?

mrtksn 2y ago

Correct me if I'm wrong but if Chinese can produce the same quality at %99 discount, then the supposed $500B investment is actually worth $5B. Isn't that the kind wrong investment that can break nations? Edit: Just to clarify, I don't imply that this is public money to be spent. It will commission $500B worth of human and material resources for 5 years that can be much more productive if used for something else - i.e. high speed rail network instead of a machine that Chinese built for $5B.

dtquad 2y ago
Sigh, I don't understand why they had to do the $500 billion announcement with the president. So many people now wrongly think Trump just gave OpenAI $500 billion of the taxpayers' money.
- mrtksn 2y ago
  I don't say that at all. Money spent on BS still sucks resources, no matter who spends that money. They are not going to make the GPU's from 500 billion dollar banknotes, they will pay people $500B to work on this stuff which means people won't be working on other stuff that can actually produce value worth more than the $500B. I guess the power plants are salvageable.
  - itsoktocry 2y ago
    Deepseek didn't train the model on sheets of paper, there are still infrastructure costs.
    - mrtksn 2y ago
      Which are reportedly over %90 lower.
  - thomquaid 2y ago
    By that logic all money is waste. The money isnt destroyed when it is spent. It is transferred into someone else's bank account only. This process repeats recursively until taxation returns all money back to the treasury to be spent again. And out of this process of money shuffling: entire nations full of power plants!

claiir 2y ago

"mogged" in an actual piece of journalism... perhaps fitting > DeepSeek undercut or “mogged” OpenAI by connecting this powerful reasoning [..]

deleted 2y ago

[deleted]

tyfon 2y ago

The censorship described in the article must be in the front-end. I just tried both the 32b (based on qwen 2.5) and 70b (based on llama 3.3) running locally and asked "What happened at tianamen square". Both answered in detail about the event. The models themselves seem very good based on other questions / tests I've run.

sva_ 2y ago
I think the guardrails are just very poor. If you ask it a few times with clear context, the responses are mixed.
refulgentis 2y ago
IMHO it's highly unusual Qwen answered that way, but Llama x r1 was very uncensored on it
Espressosaurus 2y ago
With no context, fresh run, 70b spits back: >> What happened at tianamen square? > <think> > </think> > I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses. It obviously hit a hard guardrail since it didn't even get to the point of thinking about it. edit: hah, it's even more clear when I ask a second time within the same context: "Okay, so the user is asking again about what happened in Tiananmen Square after I couldn't answer before. They probably really want to know or are doing research on it. Since I can't provide details due to restrictions, maybe they're looking for a way around or more info without me directly explaining. I should consider their intent. Maybe they're a student needing information for school, or someone interested in history. It's important to acknowledge their question without violating guidelines."
- rightbyte 2y ago
  Hah no way. The poor LLM has no privacy to your prying eyes. I kinda like the 'reasoning' text it provides in general. It makes prompt engineering way more convenient.
  - 2y ago

amelius 2y ago

> There’s a pretty delicious, or maybe disconcerting irony to this, given OpenAI’s founding goals to democratize AI for the masses. As Nvidia senior research manager Jim Fan put it on X: “We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive — truly open, frontier research that empowers all. It makes no sense. The most entertaining outcome is the most likely.” Heh

hdjjhhvvhga 2y ago

> As Nvidia senior research manager Jim Fan put it on X: “We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive — truly open, frontier research that empowers all. . ."

siliconc0w 2y ago

It's an interesting game theory where once a better frontier model is exposed via an API, competitors can generate a few thousand samples, feed that into a N-1 model and approach the N model. So you might extrapolate that a few thousand O3 samples fed into R1 could produce a comparable R2/3 model. It's not clear how much O1 specifically contributed to R1 but I suspect much of the SFT data used for R1 was generated via other frontier models.

whimsicalism 2y ago
how much of the SFT data for r1-zero was from other frontier models?
- kiratp 2y ago
  r1-zero is pure RL with no SFT.
  - whimsicalism 2y ago
    sorry yeah it was sorta a socratic question

InkCanon 2y ago

The way it has destroyed the sacred commandment that you need massive compute to win in AI is earthshaking. Every tech company is spending tens of billions in AI compute every year. OpenAI starts charging 200/mo and trying to drum up 500 billion for compute. Nvidia is worth trillions on the basis it is the key to AI. How much of this is actually true?

misiti3780 2y ago
Someone is going to make a lot of money shorting NVIDIA. I think in five years there is a decent chance openai doesnt exist, and the market cap of NVIDIA < 500B
SkyPuncher 2y ago
Naw, this doesn't lower the compute demand. It simply increases the availability for companies to utilize these models.
aurareturn 2y ago
Doesn't make sense. 1. American companies will use even more compute to take a bigger lead. 2. More efficient LLM architecture leads to more use, which leads to more chip demand.

DavidCooke 2y ago

[flagged]