3 ms·
I don't think we were wrong to look at this as a commodity problem and ask how many widgets we need. Most people will still get their access to this technology
by ecocentrik 2y ago
I don't think we were wrong to look at this as a commodity problem and ask how many widgets we need. Most people will still get their access to this technology through cloud services and nothing in this paper changes the calculations for inference compute demand. I still expect inference compute demand to be massive and distilled models aren't going to cut it for most agentic use cases.