10 ms·

Nvidia R&D chief on how AI is improving chip design

225 points by chuckjchen 4y ago

rektide 4y ago
Did nvidia just promise us singularity? :) Hard to read a talk like this from a pulpit & not see shout outs to the incredibly super-fantastic open-source innovative projects like OpenROAD which have been shipping amazingly well-routed-by-AI chips for a while now. There's papers you can cite, galore, many open source designs[1]. It's not like Nvidia is promising anyone else will benefit from this work. This seems to be very high level coverage their R&D department is looking at, perhaps/perhaps not using. The article makes it hard to find out what is available, what has been published or otherwise deeply discussed (which is I think the best we can hope from Nvidia not real participation). There's only one paper linked, on NVCell[2], described as: > The first is a system we have called NVCell, which uses a combination of simulated annealing and reinforcement learning to basically design our standard cell library. This just feels like so much else going on in computing. WSL coming to windows, the recent Unity vs Unreal topic[3]. It's hard to imagine refusing to participate with others. It's hard to imagine not being part of the open source community working shoulder to shoulder to push for better. NVidia patently doesn't get it, patently isn't participating, patently isn't there. It's cool we can hear what they are up to, but it's also extremely NVidia that they're doing it all on their own. Anyhow, Looking forward to more AI based chip power system design starting to emerge; that sounds like a good idea NV. [1] https://theopenroadproject.org/ https://theopenroadproject.org/ [2] https://research.nvidia.com/publication/2021-12_nvcell-standard-cell-layout-advanced-technology-nodes-reinforcement-learning https://research.nvidia.com/publication/2021-12_nvcell-stand... [3] https://news.ycombinator.com/item?id=31064552 https://news.ycombinator.com/item?id=31064552 (412 points, 3 days ago, 311 comments)
- nomel 4y ago
  > but it's also extremely NVidia that they're doing it all on their own. Having a lead in chip design is their literal bread and butter. I think it's extremely "publicly traded company" more than "NVidia". Do you have an example of a company releasing an open source version of their secret sauce (foundation of their profits)?
  - bawolff 4y ago
    Netscape. Worked out great for them /s (albeit writing was already on the wall for them by that point.)
  - rektide 4y ago
    > Do you have an example of a company releasing an open source version of their secret sauce? The chip design itself should be the secret sauce. Not the tools you make the chip with. Nvidia is resolutely not-contributing. Many other companies are starting to get onboard with open chip design. This doesn't mean the chips have to be open, but the tooling needs to be something shared & co-developable. If this is a little pet research project that's one thing, but there really needs to be ongoing workforce development, a strong advance. The NSF's TILOS, a strong alliance/nexus of researchers within & around the OpenROAD community, get this[1]: > TILOS – The Institute for Learning-enabled Optimization at Scale – is an NSF National AI Research Institute for advances in optimization, partially supported by Intel Corporation. The institute began operations in November 2021 with a mission to "make impossible optimizations possible, at scale and in practice". > There are six universities in TILOS: UCSD, MIT, National University, Penn, UT-Austin, and Yale. The institute seeks a new nexus of AI and machine learning, optimization, and use in practice. Figure 4 shows four virtuous cycles envisioned for the institute: 1. mutual advances of AI and optimization provide the foundations; 2. challenges of scale, along with breakthroughs from scaling, bind together foundations and the use domains of chip design, networks and robotics; 3. the cycle of translation and impact brings research and the leading edge of practice closer together; and 4. the cycle of research, education, and broadening participation grows the field and its workforce. The virtues written here are self evident & obvious. Trying to just get good yourself without trying to help advance the field, not participating, not taking advantages of scale of many working together, not participating in open research, the risks of having isolated teams, and not participating in cycles of development: whatever the nvidia or "publicly traded company" worlds think they're doing, they're missing out, and hurting everyone and especially themselves for this oldschool zero-sum competitive thinking. There are plenty of company's releasing the chips too. Google's OpenTitan[2] security chip. WD's Swerv RISC-V core for their driver controller ARM R-series replacement[3]. Open standards if not chips like UCI for chiplets or CXL for interconnect are again examples of literally everyone but NVidia playing well together, trying for better, standardizing a future for participation & healthy competition & growth. Nvidia again and again is the company which simply will not play with others. I challenge you to answer your own question in reverse: are any companies other than Nvidia embarking up AI/ML chipmaking in a closed fashion? There probably are, let's follow & watch them. [1] https://theopenroadproject.org/news/leveling-up-a-trajectory-of-openroad-tilos-and-beyond/ https://theopenroadproject.org/news/leveling-up-a-trajectory... [2] https://opentitan.org/ https://opentitan.org/ [3] https://github.com/chipsalliance/Cores-SweRV https://github.com/chipsalliance/Cores-SweRV
- jwlake 4y ago
  That article doesn't actually have enough data to really know. The anecdotes are all just design assist, make things faster from 20 mins to 3 seconds. Not singularity. There are teasers but nothing clear singularity. I think the big problem is they are just using it for fuzzy algorithm optimization, which is clearly not self learning.
- rrss 4y ago
  which openroad tools use "AI" rather than normal optimization like traditional EDA tools have used for decades?
- areskay3 4y ago
  All of the work discussed is published to some extent. References are in the slides https://arxiv.org/pdf/2012.10597.pdf https://arxiv.org/pdf/2012.10597.pdf https://research.nvidia.com/publication/2020-07_grannite-graph-neural-network-inference-transferable-power-estimation https://research.nvidia.com/publication/2020-07_grannite-gra... https://research.nvidia.com/sites/default/files/pubs/2020-07_ParaGraph:-Layout-Parasitics/057_4_Paragraph.pdf https://research.nvidia.com/sites/default/files/pubs/2020-07... https://ieeexplore.ieee.org/document/8920342 https://ieeexplore.ieee.org/document/8920342
bsder 4y ago
What is extremely telling is what is missing ... Design Rule Checking (DRC) and Layout Vs Schematic (LVS). These require: 1) Longer bit length arithmetic 32-bit float simply isn't enough. 64-bit float is close, but limited. You really want 128-bit integer. And nVidia isn't delivering that. 2) Real algorithmic improvements We're still stuck with computational geometry algorithms that don't parallelize. It would be awfully useful if nVidia would actually research some new algorithms instead of just waving around the ML/AI marketing wand. But, then, this is the company that built itself on benchmarketing, so ...
- Dylan16807 4y ago
  > You really want 128-bit integer. And nVidia isn't delivering that. How much slower (per unit area) is that to do in software, compared to a full 128-bit hardware unit?
  - erwincoumans 4y ago
    "as of 11.5, CUDA and nvcc support __int128_t in device code when the host compiler supports it (e.g., clang/gcc, but not MSVC). 11.6 added support for debug tools with __int128_t." See: https://developer.nvidia.com/blog/cuda-11-6-toolkit-new-release-revealed/ https://developer.nvidia.com/blog/cuda-11-6-toolkit-new-rele... https://developer.nvidia.com/blog/implementing-high-precision-decimal-arithmetic-with-cuda-int128/ https://developer.nvidia.com/blog/implementing-high-precisio...
- FutureZeitgeist 4y ago
  Can you explain why you need greater precision/range?
- tboerstad 4y ago
  DRC and LVS are just logical checks right? “Is the minimal distance between all metal routing > 10 nm” etc. Can you explain why high precision is needed for that?
  - atq2119 4y ago
W-Stool 4y ago
I've got a whole "HAL9000" feeling going here right now. "Sorry Dave - I can't quite do that ..."
productceo 4y ago
Please keep up the processing power progress! Economics of the software industry (or at least the products that I work on) depend on the assumption that cost of computing (including storage) diminish exponentially over time! <3
- kevincox 4y ago
  You say this like it is a good thing. It seems to me that if a whole industry is dependant on exponential growth of another than the former is being quite reckless. Of course exponential growth will help, but relying on it seems like a bit too much risk.
  - hurflmurfl 4y ago
    I think that's the point GP is making with his sarcastic remark :)
b20000 4y ago
the last time I checked autorouters were still not capable of doing all the routing on a multi layer PCB properly, and manual work was still required to produce a decent design.
- TomVDB 4y ago
  How is that a relevant comment in a discussion about ASIC design? I hope you don't have the idea that chip routing is done manually.
  - tboerstad 4y ago
    Are the analog parts (current nitrist etc) autorouted now? I worked on MCU layout around 2011, and only the digital logic was autorouted/placed.
    - TomVDB 4y ago
      My comment is in the context of this topic: large digital ASICs with billions of wires, where the only analog parts are PLLs, DLLs, and digital IOs.
  - beambot 4y ago
    IIRC, Place & Route is a known NP complete problem. In this regard, autorouters (whether IC or PCB) can benefit from "better" heuristics -- i.e. it's an optimization problem where AI can help.
    - TomVDB 4y ago
      PCB routing is generally considered a much harder problem. There are a bunch of reasons that add up, but one of them is almost certainly that a PCB is supposed to look good too. The routing on an IC is total chaos (which actually reduces crosstalk issues), but nobody will ever notice.
      - raverbashing 4y ago
_blz2 4y ago
I remember him from the vlsi text dally and poulton.
maxwells-daemon 4y ago
I work on this team! (Specifically: applied deep learning research, chip design). It's a shame to see so many people dismissing this work as marketing. I see lots of clever people working hard on really novel and interesting stuff, and I really do think that ML has real potential to customize a design much more "deeply" than traditional automation tools.
- xbmcuser 4y ago
  It is the same with a lot of the machine learning stuff posted here the 2nd or 3rd comment is that how it could be achieved with normal algos etc. But slowly as more people start applying to different problems machine learning is solving many of them.
- selimthegrim 4y ago
  Ha, this does sound awesome. Are you guys hiring?
- marginalia_nu 4y ago
  This is directed at AI marketing in general: "AI" has been used to market so much nonsense it's probably becoming a problem communicating actual interesting uses of AI. I very much get a dot com vibe off it, like nobody on the team knows how it works but we're sure we're gonna be rich somehow! In my head, I've begun substituting AI with "wizards" when I read it. It's very much the sort of problems crypto is having. So many grifters actual interesting uses of the technology are very hard to identify and take seriously.
  - q-big 4y ago
    > "AI" has been used to market so much nonsense it's probably becoming a problem communicating actual interesting uses of AI. On the other hand: if the people who do serious work in this area don't call out this nonsense, they must accept that their (serious) work becomes devalued. > It's very much the sort of problems crypto is having. So many grifters actual interesting uses of the technology are very hard to identify and take seriously. Here, the same holds.
orangebeet 4y ago
I really hope that they can apply some of these AI approaches on the driver situation on Linux as well. I will never buy an Nvidia product after the nightmares they've put me through.
ImportOllie 4y ago
I don't understand the backlash here. The jist seemed to be traditional tools that are exact take a long time to process complex designs. Deep learning offers a statistical approach that can give a 'coarse' prediction and they're using this to reduce development time. That seems to make sense to me, especially in the earlier verification phases of the hardware design lifecycle. To me this sounds like a good use-case of AI and Neural Nets. It doesn't appear to be looking to replace the traditional tools, just augment.
- jonnycomputer 4y ago
  I seem to recall that the original title of the post was more sensationalist; something about replacing human designers.
Linda703 4y ago
[dead]

225 points by chuckjchen 4y ago

Did nvidia just promise us singularity? :) Hard to read a talk like this from a pulpit & not see shout outs to the incredibly super-fantastic open-source innovative projects like OpenROAD which have been shipping amazingly well-routed-by-AI chips for a while now. There's papers you can cite, galore, many open source designs[1]. It's not like Nvidia is promising anyone else will benefit from this work. This seems to be very high level coverage their R&D department is looking at, perhaps/perhaps not using. The article makes it hard to find out what is available, what has been published or otherwise deeply discussed (which is I think the best we can hope from Nvidia not real participation). There's only one paper linked, on NVCell[2], described as: > The first is a system we have called NVCell, which uses a combination of simulated annealing and reinforcement learning to basically design our standard cell library. This just feels like so much else going on in computing. WSL coming to windows, the recent Unity vs Unreal topic[3]. It's hard to imagine refusing to participate with others. It's hard to imagine not being part of the open source community working shoulder to shoulder to push for better. NVidia patently doesn't get it, patently isn't participating, patently isn't there. It's cool we can hear what they are up to, but it's also extremely NVidia that they're doing it all on their own. Anyhow, Looking forward to more AI based chip power system design starting to emerge; that sounds like a good idea NV. [1] https://theopenroadproject.org/ https://theopenroadproject.org/ [2] https://research.nvidia.com/publication/2021-12_nvcell-standard-cell-layout-advanced-technology-nodes-reinforcement-learning https://research.nvidia.com/publication/2021-12_nvcell-stand... [3] https://news.ycombinator.com/item?id=31064552 https://news.ycombinator.com/item?id=31064552 (412 points, 3 days ago, 311 comments)

nomel 4y ago
> but it's also extremely NVidia that they're doing it all on their own. Having a lead in chip design is their literal bread and butter. I think it's extremely "publicly traded company" more than "NVidia". Do you have an example of a company releasing an open source version of their secret sauce (foundation of their profits)?
- bawolff 4y ago
  Netscape. Worked out great for them /s (albeit writing was already on the wall for them by that point.)
- rektide 4y ago
  > Do you have an example of a company releasing an open source version of their secret sauce? The chip design itself should be the secret sauce. Not the tools you make the chip with. Nvidia is resolutely not-contributing. Many other companies are starting to get onboard with open chip design. This doesn't mean the chips have to be open, but the tooling needs to be something shared & co-developable. If this is a little pet research project that's one thing, but there really needs to be ongoing workforce development, a strong advance. The NSF's TILOS, a strong alliance/nexus of researchers within & around the OpenROAD community, get this[1]: > TILOS – The Institute for Learning-enabled Optimization at Scale – is an NSF National AI Research Institute for advances in optimization, partially supported by Intel Corporation. The institute began operations in November 2021 with a mission to "make impossible optimizations possible, at scale and in practice". > There are six universities in TILOS: UCSD, MIT, National University, Penn, UT-Austin, and Yale. The institute seeks a new nexus of AI and machine learning, optimization, and use in practice. Figure 4 shows four virtuous cycles envisioned for the institute: 1. mutual advances of AI and optimization provide the foundations; 2. challenges of scale, along with breakthroughs from scaling, bind together foundations and the use domains of chip design, networks and robotics; 3. the cycle of translation and impact brings research and the leading edge of practice closer together; and 4. the cycle of research, education, and broadening participation grows the field and its workforce. The virtues written here are self evident & obvious. Trying to just get good yourself without trying to help advance the field, not participating, not taking advantages of scale of many working together, not participating in open research, the risks of having isolated teams, and not participating in cycles of development: whatever the nvidia or "publicly traded company" worlds think they're doing, they're missing out, and hurting everyone and especially themselves for this oldschool zero-sum competitive thinking. There are plenty of company's releasing the chips too. Google's OpenTitan[2] security chip. WD's Swerv RISC-V core for their driver controller ARM R-series replacement[3]. Open standards if not chips like UCI for chiplets or CXL for interconnect are again examples of literally everyone but NVidia playing well together, trying for better, standardizing a future for participation & healthy competition & growth. Nvidia again and again is the company which simply will not play with others. I challenge you to answer your own question in reverse: are any companies other than Nvidia embarking up AI/ML chipmaking in a closed fashion? There probably are, let's follow & watch them. [1] https://theopenroadproject.org/news/leveling-up-a-trajectory-of-openroad-tilos-and-beyond/ https://theopenroadproject.org/news/leveling-up-a-trajectory... [2] https://opentitan.org/ https://opentitan.org/ [3] https://github.com/chipsalliance/Cores-SweRV https://github.com/chipsalliance/Cores-SweRV
jwlake 4y ago
That article doesn't actually have enough data to really know. The anecdotes are all just design assist, make things faster from 20 mins to 3 seconds. Not singularity. There are teasers but nothing clear singularity. I think the big problem is they are just using it for fuzzy algorithm optimization, which is clearly not self learning.
rrss 4y ago
which openroad tools use "AI" rather than normal optimization like traditional EDA tools have used for decades?
areskay3 4y ago
All of the work discussed is published to some extent. References are in the slides https://arxiv.org/pdf/2012.10597.pdf https://arxiv.org/pdf/2012.10597.pdf https://research.nvidia.com/publication/2020-07_grannite-graph-neural-network-inference-transferable-power-estimation https://research.nvidia.com/publication/2020-07_grannite-gra... https://research.nvidia.com/sites/default/files/pubs/2020-07_ParaGraph:-Layout-Parasitics/057_4_Paragraph.pdf https://research.nvidia.com/sites/default/files/pubs/2020-07... https://ieeexplore.ieee.org/document/8920342 https://ieeexplore.ieee.org/document/8920342

bsder 4y ago

What is extremely telling is what is missing ... Design Rule Checking (DRC) and Layout Vs Schematic (LVS). These require: 1) Longer bit length arithmetic 32-bit float simply isn't enough. 64-bit float is close, but limited. You really want 128-bit integer. And nVidia isn't delivering that. 2) Real algorithmic improvements We're still stuck with computational geometry algorithms that don't parallelize. It would be awfully useful if nVidia would actually research some new algorithms instead of just waving around the ML/AI marketing wand. But, then, this is the company that built itself on benchmarketing, so ...

Dylan16807 4y ago
> You really want 128-bit integer. And nVidia isn't delivering that. How much slower (per unit area) is that to do in software, compared to a full 128-bit hardware unit?
- erwincoumans 4y ago
  "as of 11.5, CUDA and nvcc support __int128_t in device code when the host compiler supports it (e.g., clang/gcc, but not MSVC). 11.6 added support for debug tools with __int128_t." See: https://developer.nvidia.com/blog/cuda-11-6-toolkit-new-release-revealed/ https://developer.nvidia.com/blog/cuda-11-6-toolkit-new-rele... https://developer.nvidia.com/blog/implementing-high-precision-decimal-arithmetic-with-cuda-int128/ https://developer.nvidia.com/blog/implementing-high-precisio...
FutureZeitgeist 4y ago
Can you explain why you need greater precision/range?
tboerstad 4y ago
DRC and LVS are just logical checks right? “Is the minimal distance between all metal routing > 10 nm” etc. Can you explain why high precision is needed for that?
- atq2119 4y ago

W-Stool 4y ago

I've got a whole "HAL9000" feeling going here right now. "Sorry Dave - I can't quite do that ..."

productceo 4y ago

Please keep up the processing power progress! Economics of the software industry (or at least the products that I work on) depend on the assumption that cost of computing (including storage) diminish exponentially over time! <3

kevincox 4y ago
You say this like it is a good thing. It seems to me that if a whole industry is dependant on exponential growth of another than the former is being quite reckless. Of course exponential growth will help, but relying on it seems like a bit too much risk.
- hurflmurfl 4y ago
  I think that's the point GP is making with his sarcastic remark :)

b20000 4y ago

the last time I checked autorouters were still not capable of doing all the routing on a multi layer PCB properly, and manual work was still required to produce a decent design.

TomVDB 4y ago
How is that a relevant comment in a discussion about ASIC design? I hope you don't have the idea that chip routing is done manually.
- tboerstad 4y ago
  Are the analog parts (current nitrist etc) autorouted now? I worked on MCU layout around 2011, and only the digital logic was autorouted/placed.
  - TomVDB 4y ago
    My comment is in the context of this topic: large digital ASICs with billions of wires, where the only analog parts are PLLs, DLLs, and digital IOs.
- beambot 4y ago
  IIRC, Place & Route is a known NP complete problem. In this regard, autorouters (whether IC or PCB) can benefit from "better" heuristics -- i.e. it's an optimization problem where AI can help.
  - TomVDB 4y ago
    PCB routing is generally considered a much harder problem. There are a bunch of reasons that add up, but one of them is almost certainly that a PCB is supposed to look good too. The routing on an IC is total chaos (which actually reduces crosstalk issues), but nobody will ever notice.
    - raverbashing 4y ago

_blz2 4y ago

I remember him from the vlsi text dally and poulton.

maxwells-daemon 4y ago

I work on this team! (Specifically: applied deep learning research, chip design). It's a shame to see so many people dismissing this work as marketing. I see lots of clever people working hard on really novel and interesting stuff, and I really do think that ML has real potential to customize a design much more "deeply" than traditional automation tools.

xbmcuser 4y ago
It is the same with a lot of the machine learning stuff posted here the 2nd or 3rd comment is that how it could be achieved with normal algos etc. But slowly as more people start applying to different problems machine learning is solving many of them.
selimthegrim 4y ago
Ha, this does sound awesome. Are you guys hiring?
marginalia_nu 4y ago
This is directed at AI marketing in general: "AI" has been used to market so much nonsense it's probably becoming a problem communicating actual interesting uses of AI. I very much get a dot com vibe off it, like nobody on the team knows how it works but we're sure we're gonna be rich somehow! In my head, I've begun substituting AI with "wizards" when I read it. It's very much the sort of problems crypto is having. So many grifters actual interesting uses of the technology are very hard to identify and take seriously.
- q-big 4y ago
  > "AI" has been used to market so much nonsense it's probably becoming a problem communicating actual interesting uses of AI. On the other hand: if the people who do serious work in this area don't call out this nonsense, they must accept that their (serious) work becomes devalued. > It's very much the sort of problems crypto is having. So many grifters actual interesting uses of the technology are very hard to identify and take seriously. Here, the same holds.

orangebeet 4y ago

I really hope that they can apply some of these AI approaches on the driver situation on Linux as well. I will never buy an Nvidia product after the nightmares they've put me through.

ImportOllie 4y ago

I don't understand the backlash here. The jist seemed to be traditional tools that are exact take a long time to process complex designs. Deep learning offers a statistical approach that can give a 'coarse' prediction and they're using this to reduce development time. That seems to make sense to me, especially in the earlier verification phases of the hardware design lifecycle. To me this sounds like a good use-case of AI and Neural Nets. It doesn't appear to be looking to replace the traditional tools, just augment.

jonnycomputer 4y ago
I seem to recall that the original title of the post was more sensationalist; something about replacing human designers.

Linda703 4y ago

[dead]