4 ms·
Having worked with people doing bringup of specialized chips, I am awed at how the world has changed. > When the first chips came back from the foundry in May,
by pama 14d ago
Having worked with people doing bringup of specialized chips, I am awed at how the world has changed.
> When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.
- wmf 14d agoBack in the day you'd write the code before the chip came back but I guess today it's faster to wait.
- threatripper 14d agoThe longer you wait the faster you will go.
- LoganDark 14d agoBack when teams proved their designs and actually understood them...
- saidnooneever 14d agocant wait for no one to really know whats in chips i mean, even intel hardly knows what all their reserved mem ranges are for. who will decap the chip and see if the docs were right? xD
- RussianBot9580 14d agoHaha - understood. Good one! They'd write a limited test for a feature based on an ask from the software team garbled by a five layer game of telephone. Claim that the module passed validation. A few months later the software folks would have to pull a few all nighters to figure out how to work around the resulting turd during bringup.
- brookst 14d agoMakes me wonder about AI and FPGAs. If the cost and effort to (re)program them goes to zero, maybe interesting new applications?
- drob518 14d agoHm. An interesting thought. Paired with RSI loops, that would allow rapid iteration in the hardware domain as well.
- brookst 14d agoYeah. Just spent an hour planning a closed-loop vision-based extrusion modulation system for 3d printing, with extensive telemetry and offline processing to iterate on the realtime system. Great, like I need another side project.
- Eridrus 13d agoI had this same thought and think this is a generally interesting direction, but I think we're in a bit of a weird spot where the compute heavy stuff is on GPUs already and most infra stuff is not compute bound (it's often I/O bound or memory bound in some way). It doesn't help that FPGAs are not made at the same scale as CPUs so don't benefit from the economies of scale. I'm super curious if you have thoughts on specific pieces of software that would be economically better because I've thought about this in my niche and sort of come to the conclusion that it won't help. I do think things like SIMD in CPUs will get more use and maybe we will get more difficult to program for CPU features, but I haven't found a use case where off the shelf FPGA components would help with typical software.
- brookst 13d agoI’m looking at realtime mechanical processes, like shaping extrusion beads from a clay 3d printer. Clay is heterogenous and pressure takes time, so hand tuning is never just right. But put an fpga with vision processing? Seems promising, with millisecond-level latency that I’d never get pushing to a remote system for processing.
- marcelo-earth 14d ago> “All our schedule assumptions are going to be based on the fact we have this capability now” is the world we live in, planning things while waiting for a more powerful LLM
- cindyllm 14d ago[dead]
- _zoltan_ 13d agoI optimize a lot of CUDA and it got really, really good at it.