Sort of. At the gate counts we're talking, it doesn't take long until you can't even fit the chip on a whole wafer of an older node. IIRC, an Nvidia GA100 doe

Sort of. At the gate counts we're talking, it doesn't take long until you can't even fit the chip on a whole wafer of an older node. IIRC, an Nvidia GA100 doesn't fit on a 300mm 65nm wafer (to say nothing of being able to clock it at a viable speed even if you were to cut it down). Going off chip destroys perf for a lot of designs too. It's not just a price component, but a performance and viability of new designs one, at a time where the DoD internally is pushing ML hard as a major pillar of investment in weapons technology.