6 ms·

Here's an interesting question: why are there lots of startups claiming they can create better bespoke DL hardware accelerators than Nvidia's offerings? I unde

Here's an interesting question: why are there lots of startups claiming they can create better bespoke DL hardware accelerators than Nvidia's offerings? I understand how a bespoke architecture could in theory accommodate larger models and offer better throughput despite being based on older generations of MOFSET nodes. But if that was the case, wouldn't China simply create their own bespoke hardware accelerators? So what's stopping them?

russli1993 3y ago
hint: they are
alephnerd 3y ago
> wouldn't China simply create their own bespoke hardware accelerators? So what's stopping them? Fabrication. SMEE is supposedly able to mass produce 28nm lithography machines, but most modern (2016/Pascal onwards) GPUs are fabricated using 16nm lithography or lower (eg. Ampere uses a 7nm process, and there are multiple newer architectures in the pipeline that leverage 3nm fabrication processes at Samsung, TSMC, and Intel). Chinese companies like SMEE are trying, but it will take 3-5 years to reach 16nm lithography at scale. Also, GPUs are being limited for simulation (aka Nuclear weapons testing) reasons - not "AI" - as just about every country except North Korea honors the Comprehensive Nuclear Test Ban Treaty, forcing countries to test using HPC (edit: also used for Jet Turbine simulation a la Autodesk Federal). A 28nm process is more than enough to make EWS, Avionics, and Precision weapons, which is what Russia uses to manufacture the Elbus-8s chipset domestically. This is partially related to why companies like Nvidia has begun moving fabrication to Samsung fabs over TSMC in the short term, as SK has a formal defense agreement unlike Taiwan.
- mistrial9 3y ago
  > GPUs are being limited for simulation (aka Nuclear weapons testing) reasons - not "AI" that information seems dated in 2024
  - alephnerd 3y ago
    Not really. You don't need bespoke cutting edge hardware or models for most defense applications (aka to kill people) today. For example, C-RAMs are using Maxwell level hardware at most. The biggest driver for GPU, FPGA, and CPU development has been nuclear research, and is a major reason why the top supercomputers and HPC programs globally are usually linked with Nuclear Weapons Labs (eg. LLNL, LBL, Argonne, Oak Ridge, NSC Guangzhou). It just so happens that you use the same math for nuclear simulations as you would for "ML", bioinformatics, and computer graphics. It's all Numerical Analysis and Optimization Theory at the end of the day.
    - rfoo 3y ago
      I don't know. I thought Gina specifically said the recent ban is for AI. [0] > "What we cannot allow them to ship is the most sophisticated, highest-processing power AI chips, which would enable China to train their frontier models," she added. [0] https://www.reuters.com/technology/us-talks-with-nvidia-about-ai-chip-sales-china-raimondo-2023-12-11/ https://www.reuters.com/technology/us-talks-with-nvidia-abou...
      - alephnerd 3y ago
        The origin of these bans started in 2016-17, with Intel Xeon processors being restricted after the Chinese NSC program was found to be using some of their HPC infra for nuclear weapons simulations. No one wants to say the "Ne" word as it causes some pretty severe domestic political blowback. Already China has been looking at countering American second strike capabilities with their nuclear weapons buildup over the past decade. It also doesn't help that unlike most previous Premiers post-Mao, Xi Jinping started his administration career in the PLA.
        rfoo 3y ago
        Xeon Phi bans were targeted and successful. It shows that you don't need to issue a country-wide ban to prevent China from using a chip to build their supercomputers. And then, Sunway happened, it's built for military-use supercomputers. Do we even know whether it's 40nm, 28nm or 14nm or who on the earth fabricated them? And nothing changed on how export control works after that, that's certainly not the trigger. GPU bans before ChatGPT happened were also targeted, similar to how BIS Entity List works. Let's face it: the recent ban-entire-China movement was just about "AI", instead of HPC/simulations, the only purpose of it is to deny China NVIDIA GPU access and ensure they can't compete on SOTA language models.
- kkarakk 3y ago
  >as just about every country except North Korea honors the Comprehensive Nuclear Test Ban Treaty i don't think this is correct - https://www.nti.org/education-center/treaties-and-regimes/comprehensive-nuclear-test-ban-treaty-ctbt/ https://www.nti.org/education-center/treaties-and-regimes/co... there are 8 other countries that have not signed so why would they honor the treaty? india in particular.
  - alephnerd 3y ago
    > india in particular Last time India (and Pakistan) tested a live nuclear weapon was 1998, and both faced SEVERE sanctions at the time - and was a major reason both countries stopped buying American weapons systems and switched to Israeli and Chinese vendors respectively. The only country left that tests nuclear weapons live is North Korea
    - kkarakk 3y ago
      yes but couldn't the countries that want to do simulations just buy through india/saudi arabia etc?
      - alephnerd 3y ago
        It doesn't scale out logistically. While some lossage is expected, you can't build a Tianhe-2 type supercomputer by smuggling Nvidia (or AMD) GPUs - and a nuclear program needs dozens. China, Russia, NK, and Iran have very severe hardware import restrictions circa 2024. This is why Russia, China, and Iran have been building our domestic fabrication capabilities (Russia in the early 2010s, China in the early 2020s, and Iran presently)
        kkarakk 3y ago
        Interesting, thank you for answering!
    - rokkitmensch 3y ago
      Pretty funny that US sanctions (according to this thread) won't let you buy hardware from el US, but you totally absolutely can from our client states. Man, wouldn't it be great if we operated a real empire, instead of this optical delusion?
cherioo 3y ago
Huawei has created the “ascend” series ai chip. Reportedly it has 80% of A100 in theoretical performance. Rumor is that they had trouble selling it against A100 due to worse price to performance and worse software integration. But the US sanction has now created a huge market for it.
- alephnerd 3y ago
  It's still fabricated by TSMC though. Domestic fabrication is still at the 28nm phase at most. That said, the Ascend series a fairly massive leap, as it means domestic EDA capabilities have grown massively in China. That said, I remember they had a massive Design labs in both SV and Delhi+Bangalore that poached from the Samsung, Nvidia, and Intel labs in town before they were kicked out of US+India so idk how much design was done domestically in China.
  - DinaCoder99 3y ago
    > Domestic fabrication is still at the 28nm phase at most. Allegedly SMIC has both 14nm and 7nm fabs as of last year.
    - alephnerd 3y ago
      Allegedly but with low yield rates and leveraging ASML+Nikon DUV products. Not to say that Chinese vendors won't eventually crack that nut, but it'll take several (3-7) years. Also, domestic design capacity seems to be somewhat lacking (though rapidly changing) The most cutting edge domestic lithography tool China has is SMEE's 28nm one - that's still a MASSIVE accomplishment, but still significantly far from 20/16/14/7nm processes. Even Russia had domestic capabilities for 28nm fabrication at scale in the 2010s. That said, for most military use cases, this is more than enough.
      - alephnerd 3y ago
        > Even Russia had domestic capabilities for 28nm fabrication at scale in the 2010s. Edit: this is wrong. They have design capabilities but domestic fabrication is limited to 90nm.
imtringued 3y ago
Because most of them only implement inference, duh.

russli1993 3y ago

hint: they are

alephnerd 3y ago

> wouldn't China simply create their own bespoke hardware accelerators? So what's stopping them? Fabrication. SMEE is supposedly able to mass produce 28nm lithography machines, but most modern (2016/Pascal onwards) GPUs are fabricated using 16nm lithography or lower (eg. Ampere uses a 7nm process, and there are multiple newer architectures in the pipeline that leverage 3nm fabrication processes at Samsung, TSMC, and Intel). Chinese companies like SMEE are trying, but it will take 3-5 years to reach 16nm lithography at scale. Also, GPUs are being limited for simulation (aka Nuclear weapons testing) reasons - not "AI" - as just about every country except North Korea honors the Comprehensive Nuclear Test Ban Treaty, forcing countries to test using HPC (edit: also used for Jet Turbine simulation a la Autodesk Federal). A 28nm process is more than enough to make EWS, Avionics, and Precision weapons, which is what Russia uses to manufacture the Elbus-8s chipset domestically. This is partially related to why companies like Nvidia has begun moving fabrication to Samsung fabs over TSMC in the short term, as SK has a formal defense agreement unlike Taiwan.

mistrial9 3y ago
> GPUs are being limited for simulation (aka Nuclear weapons testing) reasons - not "AI" that information seems dated in 2024
- alephnerd 3y ago
  Not really. You don't need bespoke cutting edge hardware or models for most defense applications (aka to kill people) today. For example, C-RAMs are using Maxwell level hardware at most. The biggest driver for GPU, FPGA, and CPU development has been nuclear research, and is a major reason why the top supercomputers and HPC programs globally are usually linked with Nuclear Weapons Labs (eg. LLNL, LBL, Argonne, Oak Ridge, NSC Guangzhou). It just so happens that you use the same math for nuclear simulations as you would for "ML", bioinformatics, and computer graphics. It's all Numerical Analysis and Optimization Theory at the end of the day.
  - rfoo 3y ago
    I don't know. I thought Gina specifically said the recent ban is for AI. [0] > "What we cannot allow them to ship is the most sophisticated, highest-processing power AI chips, which would enable China to train their frontier models," she added. [0] https://www.reuters.com/technology/us-talks-with-nvidia-about-ai-chip-sales-china-raimondo-2023-12-11/ https://www.reuters.com/technology/us-talks-with-nvidia-abou...
    - alephnerd 3y ago
      The origin of these bans started in 2016-17, with Intel Xeon processors being restricted after the Chinese NSC program was found to be using some of their HPC infra for nuclear weapons simulations. No one wants to say the "Ne" word as it causes some pretty severe domestic political blowback. Already China has been looking at countering American second strike capabilities with their nuclear weapons buildup over the past decade. It also doesn't help that unlike most previous Premiers post-Mao, Xi Jinping started his administration career in the PLA.
      - rfoo 3y ago
        Xeon Phi bans were targeted and successful. It shows that you don't need to issue a country-wide ban to prevent China from using a chip to build their supercomputers. And then, Sunway happened, it's built for military-use supercomputers. Do we even know whether it's 40nm, 28nm or 14nm or who on the earth fabricated them? And nothing changed on how export control works after that, that's certainly not the trigger. GPU bans before ChatGPT happened were also targeted, similar to how BIS Entity List works. Let's face it: the recent ban-entire-China movement was just about "AI", instead of HPC/simulations, the only purpose of it is to deny China NVIDIA GPU access and ensure they can't compete on SOTA language models.
kkarakk 3y ago
>as just about every country except North Korea honors the Comprehensive Nuclear Test Ban Treaty i don't think this is correct - https://www.nti.org/education-center/treaties-and-regimes/comprehensive-nuclear-test-ban-treaty-ctbt/ https://www.nti.org/education-center/treaties-and-regimes/co... there are 8 other countries that have not signed so why would they honor the treaty? india in particular.
- alephnerd 3y ago
  > india in particular Last time India (and Pakistan) tested a live nuclear weapon was 1998, and both faced SEVERE sanctions at the time - and was a major reason both countries stopped buying American weapons systems and switched to Israeli and Chinese vendors respectively. The only country left that tests nuclear weapons live is North Korea
  - kkarakk 3y ago
    yes but couldn't the countries that want to do simulations just buy through india/saudi arabia etc?
    - alephnerd 3y ago
      It doesn't scale out logistically. While some lossage is expected, you can't build a Tianhe-2 type supercomputer by smuggling Nvidia (or AMD) GPUs - and a nuclear program needs dozens. China, Russia, NK, and Iran have very severe hardware import restrictions circa 2024. This is why Russia, China, and Iran have been building our domestic fabrication capabilities (Russia in the early 2010s, China in the early 2020s, and Iran presently)
      - kkarakk 3y ago
        Interesting, thank you for answering!
  - rokkitmensch 3y ago
    Pretty funny that US sanctions (according to this thread) won't let you buy hardware from el US, but you totally absolutely can from our client states. Man, wouldn't it be great if we operated a real empire, instead of this optical delusion?

cherioo 3y ago

Huawei has created the “ascend” series ai chip. Reportedly it has 80% of A100 in theoretical performance. Rumor is that they had trouble selling it against A100 due to worse price to performance and worse software integration. But the US sanction has now created a huge market for it.

alephnerd 3y ago
It's still fabricated by TSMC though. Domestic fabrication is still at the 28nm phase at most. That said, the Ascend series a fairly massive leap, as it means domestic EDA capabilities have grown massively in China. That said, I remember they had a massive Design labs in both SV and Delhi+Bangalore that poached from the Samsung, Nvidia, and Intel labs in town before they were kicked out of US+India so idk how much design was done domestically in China.
- DinaCoder99 3y ago
  > Domestic fabrication is still at the 28nm phase at most. Allegedly SMIC has both 14nm and 7nm fabs as of last year.
  - alephnerd 3y ago
    Allegedly but with low yield rates and leveraging ASML+Nikon DUV products. Not to say that Chinese vendors won't eventually crack that nut, but it'll take several (3-7) years. Also, domestic design capacity seems to be somewhat lacking (though rapidly changing) The most cutting edge domestic lithography tool China has is SMEE's 28nm one - that's still a MASSIVE accomplishment, but still significantly far from 20/16/14/7nm processes. Even Russia had domestic capabilities for 28nm fabrication at scale in the 2010s. That said, for most military use cases, this is more than enough.
    - alephnerd 3y ago
      > Even Russia had domestic capabilities for 28nm fabrication at scale in the 2010s. Edit: this is wrong. They have design capabilities but domestic fabrication is limited to 90nm.

imtringued 3y ago

Because most of them only implement inference, duh.