4 ms·

LLMs have a long way to go in the world of EDA. A few months ago I saw a post on LinkedIn where someone fed the leading LLMs a counter-intuitively drawn circui

LLMs have a long way to go in the world of EDA. A few months ago I saw a post on LinkedIn where someone fed the leading LLMs a counter-intuitively drawn circuit with 3 capacitors in parallel and asked what the total capacitance was. Not a single one got it correct - not only did they say the caps were in series (they were not) it even got the series capacitance calculations wrong. I couldn’t believe they whiffed it and had to check myself and sure enough I got the same results as the author and tried all types of prompt magic to get the right answer… no dice. I also saw an ad for an AI tool that’s designed to help you understand schematics. In its pitch to you, it’s showing what looks like a fairly generic guitar distortion pedal circuit and does manage to correctly identify a capacitor as blocking DC but failed to mention it also functions as a component in an RC high-pass filter. I chuckled when the voice over proudly claims “they didn’t even teach me this in 4 years of Electrical Engineering!” (Really? They don’t teach how capacitors block DC and how RC filters work????) If you’re in this space you probably need to compile your own carefully curated codex and train something more specialized. The general purpose ones struggle too much.

deleted 2y ago
[deleted]
MrsPeaches 2y ago
How many words in the art of electronics? Could you give that as context and see if might help?
hinkley 2y ago
I still have nightmares about the entry level EE class I was required to take for a CS degree. RC circuits man.
- cruffle_duffle 2y ago
  “Oh shit I better remember all that matrix algebra I forgot already!” …Then takes a class on anything with 3d graphics… “oh shit matrix algebra again!” …then takes a class on machine learning “urg more matrix math!”
  - seanmcdirmid 2y ago
    EEs actually had a head start on ML, especially those who took signal processing.
    - liontwist 2y ago
      I think anyone with an academic bone in their body had a head start on ML. Linear algebra, optimization, gradients, etc are 1st and 2nd year topics for STEM. The software industry is dominated by self taught hackers and EE/math/physics drop outs. CS prestige is less than 10 years old.
  - zahlman 2y ago
    From my experience, complex exponentials were a much more important fundamental than matrices.
  - hinkley 2y ago
    I remember vectors in 3D graphics but I don’t recall them in EE 101. maybe I blotted it out.
    - deleted 2y ago
      [deleted]
- crabmusket 2y ago
  I studied mechatronics and did reasonably well... but in any electrical class I would just scrape by. I loved it but was apparently not suited to it. I remember a whole unit basically about transistors. On the software/mtrx side we were so happy treating MOSFETs as digital. Having to analyse them in more depth did my head in.
  - mportela 2y ago
    I had a similar experience, except Mechanical Engineering being my weakest area. Computer Science felt like a children's game compared to fluid dynamics...
    - bsder 2y ago
      They call it Thermogoddamics for a reason ...
    - bitwize 2y ago
      Maybe this is why Sussman decided to approach understanding physics by way of programming.
- seattle_spring 2y ago
  I dropped EE entirely and switched from Computer Engineering to Computer Science because of my entry level EE course professor. I know I'm not the only person pushed away from EE due to Neil Cotter. Boggles my mind why he's still allowed to be the gateway to that discipline for so many people.
  - mafuyu 2y ago
    While I didn’t switch majors, I had a similar experience with my intro EE class. My theory was that it was intentionally a weeder class to push students towards the other engineering concentrations. Intro EE is kinda brutal in that there’s a lot of theory to cover, and you need to build the intuition on how it applies to real world circuit design on the fly. I had a bit of an epiphany when I was in a set theory/number theory class and some classmates were breezing through proofs that I struggled with. I was having to do algebraic manipulations in a way that was novel to me, but was intuitive to math nerds. I felt like that guy who didn’t “get” the intuition in an intro programming or circuits class. But yeah, students often get some context for math or programming in high school, but rarely for circuit design. E&M in physics at best. EE programs have solved this by weeding out anyone who can’t bash their way through the foundational theory… which isn’t great. If you’re still interested, I would recommend the Student Manual to the Art of Electronics. It’s a very practical, lab-based book that throws out a lot of the math in favor of rules of thumb and gaining intuition for circuit design.
    - hinkley 2y ago
      The thing I hated most about EE 101 though was that the diagrams predated the discovery of the electron so all the arrows point the wrong way. AND NOBODY BOTHERED TO FIX IT. It felt like taking a racketball class with my foot stuck in a bucket.
  - epolanski 2y ago
    Most entry level engineering classes (first 3/4 semesters) in most of Europe (all kinds) are designed to gate keep. I graduated in chemistry, and Chemistry 1 in engineering had tests much more difficult than any other Chemistry 1 in any other faculty. After noticing that the same pattern applied to Physics 1 or Calculus I started realizing it was an engineering thing, which was later confirmed to me by an associate professor that was the design. I asked him why, and he told me that it's a long established thing that you don't want people that struggle with science fundamentals to build bridges, ships or electrical circuits so the first semesters are very focused on this weeding.
nerdponx 2y ago
> I couldn’t believe they whiffed it Why should we expect a general-purpose instruction-tuned LLM to get this right in the first place? I am not at all surprised it didn't work, and I would be more than a little surprised if it did.
- sangnoir 2y ago
  > Why should we expect a general-purpose instruction-tuned LLM to get this right in the first place? The argument goes: Language encodes knowledge, so from the vast reams of training data, the model will have encoded the fundamentals of electromagnetism. This is based in the belief that LLMs being adept at manipulating language, are therefore inchoate general intelligences, and indeed, attaining AGI is a matter of scaling parameters and/or training data on the existing LLM foundations.
  - TheOtherHobbes 2y ago
    Which is like saying that if you read enough textbooks you'll become an engineer/physicist/ballerina/whatever.
    - sph 2y ago
      A huge number of people in academia believe so. The entire self-help literary genre is based upon this concept. In reality, and with my biases as self-taught person, experience is crucial. Learning on the field. 10,000 hours of practice. Something LLMs are not very good at. You train them a priori, then it's a relatively static product compared to how human brains operate and self-adjust.
  - garyfirestorm 2y ago
    This could be up for debate - https://www.scientificamerican.com/article/you-dont-need-words-to-think/ https://www.scientificamerican.com/article/you-dont-need-wor...
  - Vampiero 2y ago
    Yeah but language sucks at encoding the locality relations that represent a 2D picture such as a circuit diagram. Language is a fundamentally 1D concept. And I'm baffled that HN is not picking up on that and ACTUALLY BELIEVES that you can achieve AGI with a simple language model scaled to billions of parameters. It's as futile as trying to explain vision to a blind man using "only" a few billion words. There's simply no string of words that can create a meaningful representation in the mind of the blind man.
zahlman 2y ago
I would expect an LLM's internal modeling to be on approximately the level of "this is a diagram of a capacitor circuit for some student's homework; electrical component calculations for homework tend to use the adding-in-reciprocal rule, because simple addition would be too straightforward for homework". > “they didn’t even teach me this in 4 years of Electrical Engineering!” (Really? They don’t teach how capacitors block DC and how RC filters work????) My experience with being an adult, in general, is that many people who went to university don't believe that any given course taught them anything meaningful. I can absolutely believe that such people didn't learn and remember anything meaningful from those courses. Whether the course is to blame, is far more questionable.
- atoav 2y ago
  As an educationer at the academic level the number of times I have to explain absolute basic "everybody should have learned it in school"-physics is staggering.
- mrguyorama 2y ago
  >I can absolutely believe that such people didn't learn and remember anything meaningful from those courses. Whether the course is to blame, is far more questionable. It's the same as all the people who say "Why didn't high school teach me how to balance a check book or calculate a mortgage or blah blah?" In nearly every case, they literally did, but you weren't paying attention. You also had to cheat off me to pass biology, so I'm going to go ahead and press X to doubt that you "understand the immune system" We are surrounded by people who failed to invest in their own education, and instead of facing that awful reality, they INSIST that WE are the dumb ones. It's infuriating.
  - jiggawatts 2y ago
    I keep thinking of a science fiction scenario of being abducted by aliens and then being rescued by alien cops. “Where are you from?” “What’s the chemistry of your required sustenance?” “How long is your sleep cycle as measured with physical time constants?” And similar basic questions could not be answered by 99.9% of the human population. Fundamentally, almost none of us can give an accurate answer to what were made of, where we’re from, or what we need to survive.

deleted 2y ago

[deleted]

MrsPeaches 2y ago

How many words in the art of electronics? Could you give that as context and see if might help?

hinkley 2y ago

I still have nightmares about the entry level EE class I was required to take for a CS degree. RC circuits man.

cruffle_duffle 2y ago
“Oh shit I better remember all that matrix algebra I forgot already!” …Then takes a class on anything with 3d graphics… “oh shit matrix algebra again!” …then takes a class on machine learning “urg more matrix math!”
- seanmcdirmid 2y ago
  EEs actually had a head start on ML, especially those who took signal processing.
  - liontwist 2y ago
    I think anyone with an academic bone in their body had a head start on ML. Linear algebra, optimization, gradients, etc are 1st and 2nd year topics for STEM. The software industry is dominated by self taught hackers and EE/math/physics drop outs. CS prestige is less than 10 years old.
- zahlman 2y ago
  From my experience, complex exponentials were a much more important fundamental than matrices.
- hinkley 2y ago
  I remember vectors in 3D graphics but I don’t recall them in EE 101. maybe I blotted it out.
  - deleted 2y ago
    [deleted]
crabmusket 2y ago
I studied mechatronics and did reasonably well... but in any electrical class I would just scrape by. I loved it but was apparently not suited to it. I remember a whole unit basically about transistors. On the software/mtrx side we were so happy treating MOSFETs as digital. Having to analyse them in more depth did my head in.
- mportela 2y ago
  I had a similar experience, except Mechanical Engineering being my weakest area. Computer Science felt like a children's game compared to fluid dynamics...
  - bsder 2y ago
    They call it Thermogoddamics for a reason ...
  - bitwize 2y ago
    Maybe this is why Sussman decided to approach understanding physics by way of programming.
seattle_spring 2y ago
I dropped EE entirely and switched from Computer Engineering to Computer Science because of my entry level EE course professor. I know I'm not the only person pushed away from EE due to Neil Cotter. Boggles my mind why he's still allowed to be the gateway to that discipline for so many people.
- mafuyu 2y ago
  While I didn’t switch majors, I had a similar experience with my intro EE class. My theory was that it was intentionally a weeder class to push students towards the other engineering concentrations. Intro EE is kinda brutal in that there’s a lot of theory to cover, and you need to build the intuition on how it applies to real world circuit design on the fly. I had a bit of an epiphany when I was in a set theory/number theory class and some classmates were breezing through proofs that I struggled with. I was having to do algebraic manipulations in a way that was novel to me, but was intuitive to math nerds. I felt like that guy who didn’t “get” the intuition in an intro programming or circuits class. But yeah, students often get some context for math or programming in high school, but rarely for circuit design. E&M in physics at best. EE programs have solved this by weeding out anyone who can’t bash their way through the foundational theory… which isn’t great. If you’re still interested, I would recommend the Student Manual to the Art of Electronics. It’s a very practical, lab-based book that throws out a lot of the math in favor of rules of thumb and gaining intuition for circuit design.
  - hinkley 2y ago
    The thing I hated most about EE 101 though was that the diagrams predated the discovery of the electron so all the arrows point the wrong way. AND NOBODY BOTHERED TO FIX IT. It felt like taking a racketball class with my foot stuck in a bucket.
- epolanski 2y ago
  Most entry level engineering classes (first 3/4 semesters) in most of Europe (all kinds) are designed to gate keep. I graduated in chemistry, and Chemistry 1 in engineering had tests much more difficult than any other Chemistry 1 in any other faculty. After noticing that the same pattern applied to Physics 1 or Calculus I started realizing it was an engineering thing, which was later confirmed to me by an associate professor that was the design. I asked him why, and he told me that it's a long established thing that you don't want people that struggle with science fundamentals to build bridges, ships or electrical circuits so the first semesters are very focused on this weeding.

nerdponx 2y ago

> I couldn’t believe they whiffed it Why should we expect a general-purpose instruction-tuned LLM to get this right in the first place? I am not at all surprised it didn't work, and I would be more than a little surprised if it did.

sangnoir 2y ago
> Why should we expect a general-purpose instruction-tuned LLM to get this right in the first place? The argument goes: Language encodes knowledge, so from the vast reams of training data, the model will have encoded the fundamentals of electromagnetism. This is based in the belief that LLMs being adept at manipulating language, are therefore inchoate general intelligences, and indeed, attaining AGI is a matter of scaling parameters and/or training data on the existing LLM foundations.
- TheOtherHobbes 2y ago
  Which is like saying that if you read enough textbooks you'll become an engineer/physicist/ballerina/whatever.
  - sph 2y ago
    A huge number of people in academia believe so. The entire self-help literary genre is based upon this concept. In reality, and with my biases as self-taught person, experience is crucial. Learning on the field. 10,000 hours of practice. Something LLMs are not very good at. You train them a priori, then it's a relatively static product compared to how human brains operate and self-adjust.
- garyfirestorm 2y ago
  This could be up for debate - https://www.scientificamerican.com/article/you-dont-need-words-to-think/ https://www.scientificamerican.com/article/you-dont-need-wor...
- Vampiero 2y ago
  Yeah but language sucks at encoding the locality relations that represent a 2D picture such as a circuit diagram. Language is a fundamentally 1D concept. And I'm baffled that HN is not picking up on that and ACTUALLY BELIEVES that you can achieve AGI with a simple language model scaled to billions of parameters. It's as futile as trying to explain vision to a blind man using "only" a few billion words. There's simply no string of words that can create a meaningful representation in the mind of the blind man.

zahlman 2y ago

I would expect an LLM's internal modeling to be on approximately the level of "this is a diagram of a capacitor circuit for some student's homework; electrical component calculations for homework tend to use the adding-in-reciprocal rule, because simple addition would be too straightforward for homework". > “they didn’t even teach me this in 4 years of Electrical Engineering!” (Really? They don’t teach how capacitors block DC and how RC filters work????) My experience with being an adult, in general, is that many people who went to university don't believe that any given course taught them anything meaningful. I can absolutely believe that such people didn't learn and remember anything meaningful from those courses. Whether the course is to blame, is far more questionable.

atoav 2y ago
As an educationer at the academic level the number of times I have to explain absolute basic "everybody should have learned it in school"-physics is staggering.
mrguyorama 2y ago
>I can absolutely believe that such people didn't learn and remember anything meaningful from those courses. Whether the course is to blame, is far more questionable. It's the same as all the people who say "Why didn't high school teach me how to balance a check book or calculate a mortgage or blah blah?" In nearly every case, they literally did, but you weren't paying attention. You also had to cheat off me to pass biology, so I'm going to go ahead and press X to doubt that you "understand the immune system" We are surrounded by people who failed to invest in their own education, and instead of facing that awful reality, they INSIST that WE are the dumb ones. It's infuriating.
- jiggawatts 2y ago
  I keep thinking of a science fiction scenario of being abducted by aliens and then being rescued by alien cops. “Where are you from?” “What’s the chemistry of your required sustenance?” “How long is your sleep cycle as measured with physical time constants?” And similar basic questions could not be answered by 99.9% of the human population. Fundamentally, almost none of us can give an accurate answer to what were made of, where we’re from, or what we need to survive.