4 ms·
Inference-only hardware like this could be a temporary cost saving solution to scale up AI infrastructure, but I think a chip the size of this should be able to
by m_mueller 3y ago
Inference-only hardware like this could be a temporary cost saving solution to scale up AI infrastructure, but I think a chip the size of this should be able to support training, otherwise it’s just a waste of money and energy. I predict inference will move to edge computing based on mobile chips like the ones from Qualcomm in the midterm.
- wrigby 3y agoThis. When I was still doing infra for a large recommendation engine, the training was what got the big GPUs. Inference was on CPU or (later) ASICs (may have been FPGAs?).
- mupuff1234 3y agoUnless you have a model that's good enough for your use case and it's mostly inference from here on.