6 ms·

I'm sure there are LOTS of issues that need to be addressed, but the demand for the chips are so high that the incentives are overwhelmingly in favor of this co

• smokefoot 1y ago
  I mean, I don’t know how long the NVIDIA moats can hold. With this much money at stake, others will challenge their dominance especially in a market as diverse and fragmented as advanced semiconductors. That’s not to say I’m brave enough to short NVDA.
  • xbmcuser 1y ago
    google has already started offering its TPUs to other neocloud providers
    • xnx 1y ago
      I hadn't heard that. Source?
      • xbmcuser 1y ago
        https://www.datacenterdynamics.com/en/news/google-offers-its-tpus-to-ai-cloud-providers-report/ https://www.datacenterdynamics.com/en/news/google-offers-its...
        • xnx 1y ago
          Interesting. I read that as Google is using colocation to host its TPUs. I don't think Google is selling its TPUs like Nvidia sells H100s.
  • dworks 1y ago
    "Your margin is my opportunity" as someone said. Certainly Google must have plans to sell its chips externally with this much up for grabs?
    • mark_l_watson 1y ago
      I was also wondering if Google would try to make profit from selling TPUs, but they probably won’t because: At least for me, Google has some real cachet and deserves kudos for not losing money selling Gemini services, at least I think it is plausible that they are already profitable, or soon will be. In the US, I get the impression that everyone else is burning money to get market share, but if I am wrong I would enjoy seeing evidence to the contrary. I suspect that Microsoft might be doing OK because of selling access to their infrastructure (just like Google).
      • alephnerd 1y ago
        There's no point selling TPUs when you can bundle TPU access as part of much more profitable training services. The margins are much higher providing a service as part of GCP versus selling.
        • mark_l_watson 1y ago
          I agree. Amazon and I think Microsoft are also working on their own NVIDIA replacement chips - it will be interesting to see if any companies start selling chips, or stick with services.
          • alephnerd 1y ago
            From what I'm hearing in my network, the name of the game is custom chips hyperoptimized for your own workloads. A major reason Deepseek was so successful margins wise was because the team heavily understood Nvidia, CUDA, and Linux internals. If you have an understanding of the intricacies of your custom ASIC's architecture, it's easier for you to solve perf issues, parallelize, and debug problems. And then you can make up the cost by selling inference as a service. > Amazon and I think Microsoft are also working on their own NVIDIA replacement chips Not just them. I know of at least 4-5 other similar initiatives (some public like OpenAI's, another which is being contracted by a large nation, and a couple others which haven't been announced yet so I can't divulge). Contract ASIC and GPU design is booming, and Broadcom, Marvell, HPE, Nvidia, and others are cashing in on it.
  • mrktf 1y ago
    As long as only TMSC is only top performance chip producer and it is possible to reserve all it manufacturing capacity for one two clients the NVIDIA will hold without problem... My opinion, the problems for NVIDIA will start when China ramp up internal chip manufacturing performance enough to be in same order of magnitude as TMSC.
    • TSiege 1y ago
      They are currently doing this. It’s part of their Made in China 2025 plan
    • user34283 1y ago
      I'm not knowledgeable about this, but I wonder how important performance really is here. Wont it be enough to just solder on a large amount of high bandwidth memory and produce these cards relatively cheaply?
      • TylerE 1y ago
        Isn’t memory production relatively limited also?
      • alephnerd 1y ago
        > but I wonder how important performance really is here. Perf is important, but ime American MLEs are less likely to investigate GPU and OS internals to get maximum perf, and just throw money at the problem. > solder on a large amount of high bandwidth memory and produce these cards relatively cheaply HBM is somewhat limited in China as well. CXMT is around 3-4 years behind other HBM vendors. That said, you don't need the latest and most performant GPUs if you can tune older GPUs and parallelize training at a large scale. ----------- IMO, Model training is an embarrassingly parallel problem, and a large enough cluster leveraging 1-2 generation older architectures that is heavily tuned should be able to provide similar performance to train models. This is why I bemoan America's failures at OS internals and systems education. You have entire generations of "ML Engineers" and researchers in the US who don't know their way around CUDA or Infiniband optimization or the ins-and-outs of the Linux kernel. They're just boffins who like math and using wrappers. That said, I'd be cautious to trust a press release or secondhand report from CCTV, especially after the Kirin 9000 saga and SMIC. But arguably, it doesn't matter - even if Alibaba's system isn't comparably performant to an H20, if it can be manufactured at scale without eating Nvidia's margins, it's good enough.
  • mark_l_watson 1y ago
    I think that NVIDIA’s moat is the US government. Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world? I am a long time fan of Dave Sacks and the All In podcast ‘besties’ but now that he is ‘AI czar’ for our government it is interesting what he does not talk about. For example on a recent podcast he was pumping up AI as a long term solution to US economic woes, but a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails. Another thing that he swept under the rug was the recent Stanford study that 80% of US startups are saving money using less expensive Chinese (and Mistral, and Google Gemma??) models. When the Stanford study was released, I watched All In material for a few weeks, expecting David Sack’s take on the study. Not a word from him. Apologies for this off-topic rant but I am really concerned how my country is spending resources on AI infrastructure. I think this is a massive bubble, but I am not sure how catastrophic the bubble will be.
    • heavyset_go 1y ago
      > Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world? The US is burning good will at an alarming rate, how long will countries keep paying a premium to be spied on by the US instead of China?
      • mark_l_watson 1y ago
        I think the answer to your question is ‘not for very long.’ I frequently have breakfast with a friend who is a retired math professor and he is an avid investor in the stock market. We talk a lot about how long the US stock market will keep increasing in value. We don’t know the answer about the stock market, but it is fun to talk about. We both want to start easing out of the stock market.
      • rsynnott 1y ago
        The main competitors to Huawei in cell network stuff are mostly European (Nokia and friends), not American.
    • anonymousDan 1y ago
  • StopDisinfo910 1y ago
    > That’s not to say I’m brave enough to short NVDA. Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky.
    • re-thc 1y ago
      > Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky. They've been trying really hard to pivot and find new growth areas. They've taken their "inflated" stock price as capital to invest in many other companies. If at least some of these bets pay off it's not so bad.
• johndhi 1y ago
  >America has slowed the speed of Chinese AI development by a tiny number of years, if that, in return for losing total domination of the GPU market. I'm open to considering the argument that banning exports of a thing creates a market incentive for the people impacted by the ban to build aa better and cheaper thing themselves, but I don't think it's as black and white as you say. If the only ingredient needed to support massive innovation and cost cutting is banning exports, wouldn't we have tons of examples of that happening already - like in Russia or Korea or Cuba? Additionally, even if the sale of NVIDIA H100s weren't banned in China, doesn't China already have a massive incentive to throw resources behind creating competitive chips? I actually don't really like export bans, generally, and certainly not long-term ones. But I think you (and many other people in the public) are overstating the direct connection between banning exports of a thing and the affected country generating a competing or better product quickly.
  • brazukadev 1y ago
    The catch-up would happen one way or another but with the exports ban it definitely accelerated
  • lukevp 1y ago
    Russia and Korea and Cuba don’t have the economy, manufacturing and competent research scientists that China has
  • teyc 1y ago
    Head of SMIC was ex TSMC IIRC. They were able to poach TSMC engineers because Taiwan didn’t pay as well.
    • robotnikman 1y ago
      >They were able to poach TSMC engineers because Taiwan didn’t pay as well. Apparently that was an issue for them when it came to hiring people to work at their US fabs as well.
  • filoleg 1y ago
    > If the only ingredient needed to support massive innovation and cost cutting is banning exports, wouldn't we have tons of examples of that happening already - like in Russia or Korea or Cuba? That's just one of the ingredients that could help with chance of it happening, far from being "the only ingredient". The other (imo even more crucial) ingredients are the actual engineering/research+economical+industrial production capabilities. And it just so happens that none of the countries you listed (Russia, DPRK, and Cuba) have that. That's not a dig at you, it is just really rare in general for a country to have all of those things available in place, and especially for an authoritarian country. Ironically, it feels like being an authoritarian country makes it more difficult to have all those pieces together, but if such a country already has those pieces, then being authoritarian imo only helps (as you can just employ the "shove it down everyone's throat until it reaches critical mass, improves, and succeeds" strategy). However, it is important to remember that even with all those ingredients available on hand, all it means is that you have a non-zero chance at succeeding, not a guarantee of that happening.
• catigula 1y ago
  Slowing AI development by even one month is essentially infinite slowness in terms of superintelligence development. It's a kill-shot, a massive policy success. Lost months are lost exponentially and it becomes impossible to catch up. If this policy worked at all, let alone if it worked as you describe, this was a masterstroke of foreign policy. This isn't merely my opinion, experts in this field feel superintelligence is at least possible, if not plausible. This is a massively successful policy is true, and, if it's not, little is lost. You've made a very strong case for it.
  • jyscao 1y ago
    >in terms of superintelligence development doing a lot of heavy lifting in your conjecture
    • catigula 1y ago
      This is not merely my opinion, but that of knowledgable AI researchers, many of whom place ASI at not a simple remote possibility, but something they see as almost inevitable given our current understanding of the science. I don't see myself there, but, given that even the faint possibility of superintelligence would be an instant national security priority #1, grinding China into the dust on that permanently seems like a high reward, low risk endeavor. I'm not recruitable via any levers myself into a competitive ethnostate so I'm an American and believe in American primacy.