12 ms·
Nvidia Trains LLM on Chip Design
- ballenf 3y agoThe first chip they give it to design should be an ML chip that is optimized for ML chip design.
- I_Am_Nous 3y agoIt's turtles all the way down :P
- WesSouza 3y ago"Nvidia Trains LLM on Chip Design" + " documentation"
- hasbot 3y agoThe title is a bit misleading as the first sentence says "to help chip designers with tasks related to chip design, including answering general questions about chip design, summarizing bug documentation, and writing scripts for EDA tools." Still pretty cool though.
- gumby 3y agoIsn’t that what chip design is?
- hasbot 3y agoThe title suggested to me, and I see other commenters here, that the LLM was doing the chip design which isn't the case at all. So misleading title.
- ethanbond 3y agoI remember seeing a tweet from an AI guy at Nvidia saying they were using AI for chip layout. Presumably not LLMs and I’m not going back on X to find the tweet, but just to say I think they are doing this (at least experimentally).
- dragontamer 3y agoThe entire field of chip-layout is considered an NP-complete problem. Any computer program trying to solve NP-complete problems is in the realm of what I call "1980s AI". Traveling salesman, knapsack, automated reasoning, verification, binary decision diagrams, etc. etc. Its "AI", but its not machine learning or LLMs or whatever kids these days do with Stable Diffusion.
- WASDx 3y agoThat's called "Classical AI".
- brutusborn 3y agoOr GOFAI?
- brutusborn 3y agoThere are plenty of researchers using machine learning for NP-complete problems. Are you saying that this work is fruitless or just that the current state of the art is still in “1980s AI” territory?
- dragontamer 3y ago1980s style AI, such as Binary Decision Diagrams for automated reasoning, have continued to evolve over the last 40 years. The state of the art, today, remain BDD-based methodologies. I guess in theory machine learning could take a swing at the problem. And sure, some professor out there is probably trying to mix the fields and find new solutions or something. But the bulk of the work, and problem-solving, is BDDs for a reason. Or 3SAT-solvers, or... CSP solvers... etc. etc. Lots and lots of highly successful algorithms here. There's obviously open-questions for how to improve a CSP solver (faster, less RAM, more accurate estimations) and I've seen machine learning techniques applied before. But the bulk of the methodology remains in whatever solver model you're going for. Even today.
- 3y ago
- uoaei 3y agoIt would be very surprising if a Large Language Model trained to speak English could adequately specify chip architecture...
- DesiLurker 3y agowhy not? a netlist can be easily 'tokenized'. its already in parse-able format. you can just could just chop off the English input portion and it will consume. in fact I am sure you could write a 'read & speak' type program to read the RTL spec and feed it in but I'll suspect they'll a custom trained LLM on millions of generated RTL examples.
- uoaei 3y agoYou'd need to spend a long time training it on something other than English... at which point it's not an LLM. I definitely agree that the LLM architecture is useful for a lot more than just language per se assuming you can appropriately tokenize your inputs.
- IshKebab 3y agoWhy? Chip architecture specifications are written in English! Are you surprised that ChatGPT can write Python?
- meltyness 3y agoProbably excludes qualitative tasks like architecture and apportioning resources.
- HPsquared 3y agoIt's a bit like "assistant manager" vs "assistant to the manager".
- deleted 3y ago[deleted]
- xnx 3y agoGoogle has been using machine learning for chip design since at least 2021: https://www.nature.com/articles/s41586-021-03544-w https://www.nature.com/articles/s41586-021-03544-w Hasn't brought about the singularity yet.
- scrlk 3y agoDEC did it in the 1980s: https://en.wikipedia.org/wiki/VAX_9000#SID_Scalar_and_Vector_Processor_Synthesis https://en.wikipedia.org/wiki/VAX_9000#SID_Scalar_and_Vector...
- jdblair 3y agoSID was an "expert system with over 1000 hand-written rules" Wow.
- indeyets 3y agoso, something like eslint then (sorry)
- uxp100 3y agoThis sounds like an example of “AI is whatever we think computers can’t do”. Because rule based gate synthesis sounds like what has been done today.
- satvikpendem 3y ago> The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not "real" intelligence.[1] > Author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."[2] Researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'"[3] > "AI is whatever hasn't been done yet." > —Larry Tesler https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
- dddrh 3y agoInteresting concept that raised the question for me: What is the primary limiting factor right now that prevents LLM’s or any other AI model to go “end to end” on programming a full software solution or full design/engineering solution? Is it token limitations or accuracy the further you get into the solution?
- thechao 3y agoLLM's can't gut a fish in the cube when they get to their limits. On a more serious note: I think the high-level structuring of the architecture, and then the breakdown into tactical solutions — weaving the whole program together — is a fundamental limitation. It's akin to theorem-proving, which is just hard. Maybe it's just a scale issue; I'm bullish on AGI, so that's my preferred opinion.
- alasarmas 3y agoActually I think this is a good point: fundamentally an AI is forced to “color inside the lines”. It won’t tell you your business plan is stupid and walk away, which is a strong signal that is hard to ignore. So will this lead to people with more money than sense to do even more extravagantly stupid things than we’ve seen in the past, or is it basically just “Accenture-in-a-box”?
- RecycledEle 3y agoAI will absolutely rate your business plan if you ask it to. Try this prompt:"Please rate this business plan on a scale of 1-100 and provide buttle points on how it can be improved without rewriting any of it: <business plan>"
- alasarmas 3y agoI agree that AI is totally capable of rating a business plan. However, I think that the act of submitting a business plan to be rated requires some degree of humility on the part of the user, and I do doubt that an AI will “push back” when it comes to an obviously bad business plan unless specifically instructed to do so.
- carabiner 3y agoNo! LLM's must be destroyed!
- andy_ppp 3y agoWhen I see the headline "LLM trains Nvidia on Chip Design" I'll start to worry :-/
- falcor84 3y ago"""Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind... Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously. """ I. J. Good, in 1965 - https://en.wikipedia.org/wiki/I._J._Good https://en.wikipedia.org/wiki/I._J._Good
- ethanbond 3y agoDon’t worry, it’ll be good because it’s trained on human stories, in which usually the good guy wins.
- sho_hn 3y agoAfter a Road of Trials with potentially a lot of collateral damage, if the AI read Campbell too. Also, who says we're the protagonist in the script? :-)
- DesiLurker 3y agoI know its /s but we also say that the victor writes the history books. and its kinda hard not to portray yourself as the good guy when the other guy is not around to defend himself. AGI will learn that too. I mean one look at earth and its not like we have been model citizens exactly.
- omneity 3y agoKeeping in mind, this holds true in a runaway fashion if the only bottleneck to more intelligence is further intelligence. I suspect physical limitations similar to how many runaway processes in the universe are more logistical than exponential in nature.
- 3y ago
- 4b11b4 3y agoreminding me of flux.ai for PCBs
- antimatter15 3y agoThe code generation tool better be called "Tcl me NeMo"
- kumarski 3y agohttps://motivo.ai https://motivo.ai https://www.silimate.com/ https://www.silimate.com/ https://www.astrus.ai https://www.astrus.ai
- jstummbillig 3y agoI can't be alone in just assuming that everyone with a few millions to spare is training LLMs to help with their problem right now.
- lucubratory 3y agoLimiting factor actually isn't money right now, it's compute. Anyone with the compute to spare is doing it though, yes.
- einpoklum 3y agoWell, at least everyone with too much money on their hands and no useful notion of how to spend it. Hey, I have an idea! Let me train an LLM to make suggestions on how to use LLMs! That can't fail, can it?
- moistoreos 3y ago> “If we even got a couple percent improvement in productivity, this would be worth it. And our goals are actually to do quite a bit better than that.” So your engineers will get a productivity salary increase, right? RIGHT?!
- chii 3y agoThe engineers get paid to do the work, but won't partake in the profits of the company (except by prior agreement like a bonus target reached etc). I suppose equity grants straddle the fence regarding this, since such productivity gains would translate to higher prices for equity.
- einpoklum 3y agoNVIDIA trains press on clickbait and gimmicks. Result: Stellar success!