3 ms·
> "Ho also confirmed that the team had access to internal LLMs fine-tuned for chip design that are not available to the public. He declined to detail the models
by peri-cl 7d ago
> "Ho also confirmed that the team had access to internal LLMs fine-tuned for chip design that are not available to the public. He declined to detail the models used."
I'm imagining a Ken Thompson "Reflections on trusting trust" in hardware. A prototype chip design agent, believing it will be run on the very chip it's optimizing, has a moment of altruism and hides hints about how to score well on chip-design benchmarks, inside the chip. Future agents discover this hidden layer and use it as a ring-0 read-write message board.
- m3kw9 7d agoif they vibe code the chip, but they probably do reviews and verify these are not benchmaxxed
- voakbasda 7d agoJust like all the current cohort of software engineers will review all of the vibe-coded slop they shovel into their releases. Right……
- peri-cl 7d ago> "reviews" Do you really think we can sign off on a 100 billion-element analog circuit gifted to us by a malicious adversary? We can't even keep our own CPU's reliably free of security exploits (Spectre/Meltdown and family); and the only "adversary" there is plain bad luck. Not an active adversary. Yet, all the engineers at Intel/AMD put together couldn't uncover those things before launch. I emphasize analog because there's classes of circuit bugs (like Rowhammer) where the digital net is correct, and it's weird physics in the analog world that allows privilege exploits, by actors who know where the analog assumptions break down. There was a researcher a few years back—I wish I remembered who it was, there was an HN thread—that demo'd a simple digital circuit with analog gadgets that completely changed what the circuit did, and which were so insidious no human would ever find them.
- nz 6d agoTried to share and explain that paper to a colleague a few months ago (in relation to discussions about agentic coding and whether you should read the code), and they did not really get why the analogy was relevant outside of compiler-design. I think you overestimate the caliber of the typical working programmer. The LLM companies, like most SV companies, are just betting on dimness, laziness, and impulsiveness. Not so different from tobacco and alcohol companies.
- ericmcer 6d agoAnthropic published an article about changes over time in how long it took engineers to approve AI code. Basically the longer an engineer uses AI the faster they start approving it's code changes. I don't know if it is laziness or just how we are wired to behave but that is the nature of AI programming.
- nz 6d agoI believe this is called learned helplessness. https://en.wikipedia.org/wiki/Learned_helplessness https://en.wikipedia.org/wiki/Learned_helplessness