3 ms·
As someone who does both hardware design (mostly high precision analog electronics) and software engineering, the difference between both fields in AI capabilit
by XRG 12d ago
As someone who does both hardware design (mostly high precision analog electronics) and software engineering, the difference between both fields in AI capability is striking. Models like Fable and Astra can provide lots of interesting insights into the design of specialty electronics when prompted correctly. Yet they can hardly put that knowledge to work if instructed to generate a schematic or lay out a PCB.
Of course, for readers here with a solid grasp on how LLM's operate it's not exactly surprising. Still, I feel this is exactly the kind of thing that drives home that the way we reason about (general) intelligence does not map cleanly to LLM operation, no matter how advanced the current models are.
- crote 12d agoIf anything, it is surprising that the LLM is so bad at stealing a ready-made RP2350 design! There are plenty of easy-to-find tutorials like [0] and [1] that handhold you through the entire design process, and a simple search will give you dozens of design files for open-source boards. At this level of board complexity you almost have to try to make it completely non-functional, so although it is nice to see it in practice I am not exactly surprised that a LLM with a decent bunch of assistance managed to do it. But there's quite a gap between the equivalent of poorly copying a "Hello World" from StackOverflow and making actual production-ready boards. [0]: https://pip-assets.raspberrypi.com/categories/1214-rp2350/documents/RP-008280-DS-2-hardware-design-with-rp2350.pdf https://pip-assets.raspberrypi.com/categories/1214-rp2350/do... [1]: https://jlcpcb.com/blog/how-to-design-layout-with-rp2350 https://jlcpcb.com/blog/how-to-design-layout-with-rp2350
- genxy 9d agoYou have to be more specific about what you mean by design. The models are excellent and the netlist and the components along with tradeoffs.
- akiselev 9d agoDesign means PCB routing. A schematic is the PCB equivalent to a Mermaid diagram.
- poulpy123 9d agoThere is very little sources compared to software and with hardware you can't really afford too many errors
- K0balt 11d agoIt’s the training data. Finished work isn’t training data because parts are sooooo different, but maybe if the finished designs were properly labeled into the training pipeline it might work? Or the iterative work itself might need to be the training corpus.
- snvzz 9d agoMaybe it's not properly benchmarked. A standard benchmark using kicad to create or fix electronic designs would go a long way in motivating and measuring improvement.
- adrianvi 9d agoThere's eebench, but maybe it's not exactly what you were thinking
- perching_aix 9d agoIt was featured on Astra's model card and promotional materials, which the folks there (at EE-Bench) really appreciated: https://eebench.org/blog/can-ai-design-circuit-boards-yet/ https://eebench.org/blog/can-ai-design-circuit-boards-yet/ I'd expect that upcoming models will be increasingly better tested and developed for this workload from now on.
- eluru 9d agoi noticed the same treying to just generate 3D models using AI, i wonder if a well trained very specific world model would be better for the task
- Schlagbohrer 9d agoMe too, LLMs are bad at generating 3D models still but that and EE/ME CAD are the next frontiers.
- trollbridge 9d agoMaybe they could try conquering GUI layouts and CSS first
- IshKebab 9d agoYeah enjoy it while it lasts. This is basically where LLMs were with coding a year or two ago. Give it a couple more years...
- Neywiny 9d agoI just started using opus 4.8 and it's still trash at very domain specific stuff and debugging. It went down a rabbit hole trying to coerce the algorithm to work for 1 data point, which dramatically overfit the code. Later it did 11 experiments to try and figure out how to do the math, each one yielding an accuracy no better than a coin flip and each one it asserted beforehand that it had it this time, 100% accuracy. We've got some time.
- IshKebab 9d agoThat's last gen though. I agree it's mediocre. Try Astra. It's significantly better than Opus and even Fable. I'm not sure about earlier OpenAI models. I also thought we had time.
- Oreb 9d agoIt seems to depend on the domain, and perhaps the tech stack? I’ve been disappointed by Astra, it’s worse than both Fable and Opus for me.
- IshKebab 9d agoYeah maybe. I dunno though, I'm using it in obscure SystemVerilog stuff. AI used to absolutely suck at that very recently but now it seems to have no problem. I think the key is that you give it all of the info - put all the docs, the LRM, tool manuals, etc. in a directory so it can read it if it wants (it really really wants).
- Neywiny 9d agoThey say that every generation. https://www.youtube.com/watch?v=akNcZOoLKVk https://www.youtube.com/watch?v=akNcZOoLKVk
- baq 9d agoNo worries, Astra already shows signs of improvement which should make you happy for the short time before you realize why they’re making it better at this and CAD in general
- PunchyHamster 9d agoI used them a bunch just for the component shopping, I can route PCBs just fine, that's not a throttling factor in any project, but ability to "send it shopping" and not only get the price/availablity for the options but look thru datasheet for quirks and parameters is very useful.