I'm curious, what does cheaper mean exactly, and how much cheaper is it? Is that cheaper to acquire the inferencing hardware, or cheaper in terms of energy effi

I'm curious, what does cheaper mean exactly, and how much cheaper is it? Is that cheaper to acquire the inferencing hardware, or cheaper in terms of energy efficiency? Is your inferencing running on servers, or commodity hardware... in the cloud or on user/consumer devices? I'd agree with the sibling comment; this depends completely on what you are trying to do, and how fast you need to do it. There's nothing wrong with inferencing on a CPU, and I have no doubt it's much cheaper in certain ways, but it's also slower than what you can do on a GPU or custom ASIC, and there are reasons some people need it to go faster. One example that's in wide deployment would be Nvidia's DLSS for video games. It'd be pretty hard to run that in real time at 4k resolution on a CPU.