6 ms·

Given that AI is a national security matter now, I'd expect the U.S.A to step in and rescue certain companies in the event of a crash. However, I'd give higher

Given that AI is a national security matter now, I'd expect the U.S.A to step in and rescue certain companies in the event of a crash. However, I'd give higher chances to NVIDIA than OpenAI. Weights are easily transferrable and the expertise is in the engineers, but ability to continue making advanced chips is not as easily transferred.

embedding-shape 11mo ago
Why is ML knowledge "in the engineers" while chip manufacturing apparently sits in the company/hardware/something else than the engineers/humans?
- NBJack 11mo ago
  Read up a bit on the effort needed to get a fab going, and the yield rates. While engineers are crucial in the setup, the fab itself is not as 'fungible' as the employees involved. I can spin up a strong ML team through hiring in probably 6-12 months with the right funding. Building a chip fab and getting it to a sensible yield would take 3-5 years, significantly more funding, strong supply lines, etc.
  - singron 11mo ago
    Nvidia isn't a fab.
  - wongarsu 11mo ago
    But the fabs don't belong to NVIDIA, they belong to TSMC. I have no doubt that Taiwan and maybe even the US government would step in to save TSMC if for some reason it got existential problems, but that doesn't provide an argument for saving NVIDIA
  - trollbridge 11mo ago
    Right. I could spin up a strong ML team, an AI startup, build a foundational model, etc give a reasonable amount of seed capital. Build a chip fab? I’ve got no idea where to start, where to even find people to hire, and i know the equipment we’d need to acquire would be also quite difficult to get at any price.
  - OfficialTurkey 11mo ago
    > I can spin up a strong ML team through hiring in probably 6-12 months with the right funding. Mark Zuckerberg would like a word with you
  - embedding-shape 11mo ago
    > I can spin up a strong ML team through hiring in probably 6-12 months with the right funding Not sure what to call this except "HN hubris" or something. There are hundreds of companies who thought (and still think) the exact same thing, and even after 24 months or more of "the right funding" they still haven't delivered the results. I think you're misunderstanding how difficult all of this is, if you think it's merely a money problem. Otherwise we'd see SOTA models from new groups every month, which we obviously aren't, we have a few big labs iteratively progressing SOTA, with some upstarts appearing sometimes (DeepSeek, Kimi et al) but it isn't as easy as you're trying to make it out to be.
    - whimsicalism 11mo ago
      There’s a lot in LLM training that is pretty commodity at this point. The difficulty is in data - and a large part of why it has gotten more challenging is simply that some of the best sources of data have locked down against scraping post-2022 and it is less permissible to use copyrighted data than the “move fast and break things” pre-2023 era. As you mentioned, multiple no name chinese companies have done it and published many of their results. There is a commodity recipe for dense transformer training. The difference between Chinese and US is that they have less data restrictions. I think people overindex on the Meta example. It’s hard to fully understand why Meta/llama have failed as hard as they have - but they are an outlier case. Microsoft AI only just started their efforts in earnest and are already beating Meta shockingly.
    - marcyb5st 11mo ago
      Fully agree. I also think we are deep into the diminishing returns territory. If I have to guess OAI and others pay top dollars for talent that has a higher probability of discovering the next "attention" mechanism and investors are betting this is coming soon (hence the hige capitalizations and willing to loive with 11B losses/quarter). If they lose patience in throwing money at the problem I see only few players remaining in the race because they have other revenue streams
    - noosphr 11mo ago
      >Otherwise we'd see SOTA models from new groups every month We do. It's just that startups don't go after the frontier models but niche spaces which are under served and can be explored with a few million in hardware. Just like how open AI made gpt2 before they made gpt3.
- tonyarkles 11mo ago
  First-order: because of the capex and lead times. If you grab a bunch of world-class ML folks and put them in a room together, they're going to be able to start producing world-class work together. If you grab a bunch of world-class chip designers in the same scenario but don't have world-class fabs for them to use, they're not going to be able to ship competitive designs.
  - embedding-shape 11mo ago
    > If you grab a bunch of world-class chip designers in the same scenario but don't have world-class fabs for them to use, they're not going to be able to ship competitive designs. But why such an unfair comparison? Instead of comparing "skilled people with hardware VS skilled people without hardware", why not compare it to "a bunch of world-class ML folks" without any computers to do the work, how could they produce world-class work then?
    - jimbokun 11mo ago
      Much easier and cheaper to source computers than a fab.
      - embedding-shape 11mo ago
        Right, but to source a fab you need experience as well, nothing you can just hire a random person to do exactly.
        tonyarkles 11mo ago
        To simplify it down even more: - For the ML team, you need money. Money to pay them and money to get access to GPUs. You might buy the GPUs and make your own server farm (which also takes time) or you might just burn all that money with AWS and use their GPUs. You can trade off money vs. time. - For the chip design team, you need money and time. There's no workaround for the time aspect of it. You can't spend more money and get a fab quicker.
- bob1029 11mo ago
  One person can implement a transformer model from scratch in a weekend. Hardware is not the valuable part of machine learning. Data and how it is used are. The "magic of AI" doesn't live inside an Nvidia GPU. There are billions of dollars of marketing being deployed to convince you it does. As soon as the market realizes that nvidia != magic AI box, the music should stop pretty quickly.
  - tehjoker 11mo ago
    That's true, but without the kind of horsepower provided by modern hardware, even though I'm skeptical that it's all needed, especially given DeepSeek's amazing results, AI would be nearly impossible. There are some important innovations on the algorithm / network structure side, but all these ideas are only able to be tried because the hardware supports it. This stuff has been around for decades.
    - chermi 11mo ago
      Deepseek required existing models that required the horsepower.
      - tehjoker 11mo ago
        That was claimed but never proven. I personally don't believe the American companies making this claim. I suspect they made this up to protect their valuations when they were hideously embarassed and lost a trillion dollars in equity.
  - chermi 11mo ago
    Umm, part of it does. It necessary but not sufficient, at least to achieve it on the timescales we've seen. Scale is part of the "magic".
- jeffwask 11mo ago
  The start-up costs of creating a new chip manufacture are significantly higher (you can't just SAAS your way into factories) and the chips themselves more subject to IP and patents owned by that company.
- thesz 11mo ago
  Chip manufacturing is extremely time consuming, especially when we are talking about masks for lithography. The rights on masks for chips and their parts (IPs) belong to companies. And one definitely does not want these masks to be sold during bankruptcy process to (arbitrary) higher bidders.
- jimbokun 11mo ago
  Chip designs have strong IP protections. AI models do not. Sure you can't just copy the exact floating point values without permission. But with enough capital you can train a model just as good, as the training and inference techniques are well known.
  - embedding-shape 11mo ago
    > But with enough capital you can train a model just as good, as the training and inference techniques are well known You're not alone in believing just money can train a good model, and I've already answered elsewhere why things aren't so easy as you believe, but besides this, where are y'all getting that from? Is there some popular social media influencer that keeps parroting this or where it comes from? Clearly you're not involved in those processes/workflows yourself, then you wouldn't claim it's just a money problem, so where are you all getting this from?
philipwhiuk 11mo ago
If they're too-important-to-fail they're too important not to be broken up or nationalised.
- jimbokun 11mo ago
  While that is a sensible opinion the 2008 crash showed that it is not the opinion of decision makers in the US.
- whimsicalism 11mo ago
  I’m curious if those of you calling for nationalization have worked for the government or a state-owned enterprise like Amtrak. People should witness the effects of long-term public sector ownership on productivity and effectiveness in a workplace.
  - saulpw 11mo ago
    Yeah, like IBM and Intel and GE and GM are shining examples of how effectively the private sector runs companies. Maybe large enterprises are by their nature inefficient. Maybe productivity isn't the best metric for a utility. We could, for instance, prioritize resiliency, longevity, accessibility, and environmental concerns.
    - whimsicalism 11mo ago
      Even those problematic companies exemplify the difference: when enterprises are mismanaged and fail, capital is reallocated away from them.
      - saulpw 11mo ago
        The US government just allocated $10b towards Intel, and bailed out GM in the past. So what you said is clearly not the case. Now we have publicly-funded private management that is failing. At least if they were publicly owned and managed outright, they wouldn't be gutted by executives prioritizing quarterly profits.
deleted 11mo ago
[deleted]
lz400 11mo ago
Even if/when the bubble pops, I don't think NVIDIA is even close to need rescuing or being in trouble. They might end being worth 2 trillion instead of 5 but they're still selling GPUs nobody else knows how to make that power one of the most important technologies in the world. Also, all their other divisions. The .com bubble didn't stop the internet or e-commerce, they still won, revolutioned everything, etc. etc. Just because there's a bubble it doesn't mean AI won't be successful. It will be, almost for sure. We've all used it, it's truly useful and transformative. Let's not miss the forest for the trees.

embedding-shape 11mo ago

Why is ML knowledge "in the engineers" while chip manufacturing apparently sits in the company/hardware/something else than the engineers/humans?

NBJack 11mo ago
Read up a bit on the effort needed to get a fab going, and the yield rates. While engineers are crucial in the setup, the fab itself is not as 'fungible' as the employees involved. I can spin up a strong ML team through hiring in probably 6-12 months with the right funding. Building a chip fab and getting it to a sensible yield would take 3-5 years, significantly more funding, strong supply lines, etc.
- singron 11mo ago
  Nvidia isn't a fab.
- wongarsu 11mo ago
  But the fabs don't belong to NVIDIA, they belong to TSMC. I have no doubt that Taiwan and maybe even the US government would step in to save TSMC if for some reason it got existential problems, but that doesn't provide an argument for saving NVIDIA
- trollbridge 11mo ago
  Right. I could spin up a strong ML team, an AI startup, build a foundational model, etc give a reasonable amount of seed capital. Build a chip fab? I’ve got no idea where to start, where to even find people to hire, and i know the equipment we’d need to acquire would be also quite difficult to get at any price.
- OfficialTurkey 11mo ago
  > I can spin up a strong ML team through hiring in probably 6-12 months with the right funding. Mark Zuckerberg would like a word with you
- embedding-shape 11mo ago
  > I can spin up a strong ML team through hiring in probably 6-12 months with the right funding Not sure what to call this except "HN hubris" or something. There are hundreds of companies who thought (and still think) the exact same thing, and even after 24 months or more of "the right funding" they still haven't delivered the results. I think you're misunderstanding how difficult all of this is, if you think it's merely a money problem. Otherwise we'd see SOTA models from new groups every month, which we obviously aren't, we have a few big labs iteratively progressing SOTA, with some upstarts appearing sometimes (DeepSeek, Kimi et al) but it isn't as easy as you're trying to make it out to be.
  - whimsicalism 11mo ago
    There’s a lot in LLM training that is pretty commodity at this point. The difficulty is in data - and a large part of why it has gotten more challenging is simply that some of the best sources of data have locked down against scraping post-2022 and it is less permissible to use copyrighted data than the “move fast and break things” pre-2023 era. As you mentioned, multiple no name chinese companies have done it and published many of their results. There is a commodity recipe for dense transformer training. The difference between Chinese and US is that they have less data restrictions. I think people overindex on the Meta example. It’s hard to fully understand why Meta/llama have failed as hard as they have - but they are an outlier case. Microsoft AI only just started their efforts in earnest and are already beating Meta shockingly.
  - marcyb5st 11mo ago
    Fully agree. I also think we are deep into the diminishing returns territory. If I have to guess OAI and others pay top dollars for talent that has a higher probability of discovering the next "attention" mechanism and investors are betting this is coming soon (hence the hige capitalizations and willing to loive with 11B losses/quarter). If they lose patience in throwing money at the problem I see only few players remaining in the race because they have other revenue streams
  - noosphr 11mo ago
    >Otherwise we'd see SOTA models from new groups every month We do. It's just that startups don't go after the frontier models but niche spaces which are under served and can be explored with a few million in hardware. Just like how open AI made gpt2 before they made gpt3.
tonyarkles 11mo ago
First-order: because of the capex and lead times. If you grab a bunch of world-class ML folks and put them in a room together, they're going to be able to start producing world-class work together. If you grab a bunch of world-class chip designers in the same scenario but don't have world-class fabs for them to use, they're not going to be able to ship competitive designs.
- embedding-shape 11mo ago
  > If you grab a bunch of world-class chip designers in the same scenario but don't have world-class fabs for them to use, they're not going to be able to ship competitive designs. But why such an unfair comparison? Instead of comparing "skilled people with hardware VS skilled people without hardware", why not compare it to "a bunch of world-class ML folks" without any computers to do the work, how could they produce world-class work then?
  - jimbokun 11mo ago
    Much easier and cheaper to source computers than a fab.
    - embedding-shape 11mo ago
      Right, but to source a fab you need experience as well, nothing you can just hire a random person to do exactly.
      - tonyarkles 11mo ago
        To simplify it down even more: - For the ML team, you need money. Money to pay them and money to get access to GPUs. You might buy the GPUs and make your own server farm (which also takes time) or you might just burn all that money with AWS and use their GPUs. You can trade off money vs. time. - For the chip design team, you need money and time. There's no workaround for the time aspect of it. You can't spend more money and get a fab quicker.
bob1029 11mo ago
One person can implement a transformer model from scratch in a weekend. Hardware is not the valuable part of machine learning. Data and how it is used are. The "magic of AI" doesn't live inside an Nvidia GPU. There are billions of dollars of marketing being deployed to convince you it does. As soon as the market realizes that nvidia != magic AI box, the music should stop pretty quickly.
- tehjoker 11mo ago
  That's true, but without the kind of horsepower provided by modern hardware, even though I'm skeptical that it's all needed, especially given DeepSeek's amazing results, AI would be nearly impossible. There are some important innovations on the algorithm / network structure side, but all these ideas are only able to be tried because the hardware supports it. This stuff has been around for decades.
  - chermi 11mo ago
    Deepseek required existing models that required the horsepower.
    - tehjoker 11mo ago
      That was claimed but never proven. I personally don't believe the American companies making this claim. I suspect they made this up to protect their valuations when they were hideously embarassed and lost a trillion dollars in equity.
- chermi 11mo ago
  Umm, part of it does. It necessary but not sufficient, at least to achieve it on the timescales we've seen. Scale is part of the "magic".
jeffwask 11mo ago
The start-up costs of creating a new chip manufacture are significantly higher (you can't just SAAS your way into factories) and the chips themselves more subject to IP and patents owned by that company.
thesz 11mo ago
Chip manufacturing is extremely time consuming, especially when we are talking about masks for lithography. The rights on masks for chips and their parts (IPs) belong to companies. And one definitely does not want these masks to be sold during bankruptcy process to (arbitrary) higher bidders.
jimbokun 11mo ago
Chip designs have strong IP protections. AI models do not. Sure you can't just copy the exact floating point values without permission. But with enough capital you can train a model just as good, as the training and inference techniques are well known.
- embedding-shape 11mo ago
  > But with enough capital you can train a model just as good, as the training and inference techniques are well known You're not alone in believing just money can train a good model, and I've already answered elsewhere why things aren't so easy as you believe, but besides this, where are y'all getting that from? Is there some popular social media influencer that keeps parroting this or where it comes from? Clearly you're not involved in those processes/workflows yourself, then you wouldn't claim it's just a money problem, so where are you all getting this from?

philipwhiuk 11mo ago

If they're too-important-to-fail they're too important not to be broken up or nationalised.

jimbokun 11mo ago
While that is a sensible opinion the 2008 crash showed that it is not the opinion of decision makers in the US.
whimsicalism 11mo ago
I’m curious if those of you calling for nationalization have worked for the government or a state-owned enterprise like Amtrak. People should witness the effects of long-term public sector ownership on productivity and effectiveness in a workplace.
- saulpw 11mo ago
  Yeah, like IBM and Intel and GE and GM are shining examples of how effectively the private sector runs companies. Maybe large enterprises are by their nature inefficient. Maybe productivity isn't the best metric for a utility. We could, for instance, prioritize resiliency, longevity, accessibility, and environmental concerns.
  - whimsicalism 11mo ago
    Even those problematic companies exemplify the difference: when enterprises are mismanaged and fail, capital is reallocated away from them.
    - saulpw 11mo ago
      The US government just allocated $10b towards Intel, and bailed out GM in the past. So what you said is clearly not the case. Now we have publicly-funded private management that is failing. At least if they were publicly owned and managed outright, they wouldn't be gutted by executives prioritizing quarterly profits.

deleted 11mo ago

[deleted]

lz400 11mo ago

Even if/when the bubble pops, I don't think NVIDIA is even close to need rescuing or being in trouble. They might end being worth 2 trillion instead of 5 but they're still selling GPUs nobody else knows how to make that power one of the most important technologies in the world. Also, all their other divisions. The .com bubble didn't stop the internet or e-commerce, they still won, revolutioned everything, etc. etc. Just because there's a bubble it doesn't mean AI won't be successful. It will be, almost for sure. We've all used it, it's truly useful and transformative. Let's not miss the forest for the trees.