I think this is true for operations where inference time is not very critical. Think content generation. For example, my work in grad school is deploying AI-ba

I think this is true for operations where inference time is not very critical. Think content generation. For example, my work in grad school is deploying AI-based control on drones such that they are fault-tolerant. We can have a stronger model at the ground station communicating with the vehicle, which introduces lag. Or the ground station model is more supervisory, and bare metal control is deferred to your classical algorithms. Or we run a smaller, faster network(s) on board which can act and learn fast. The other thing is model interpretability. For some of my work with drones or smart buildings, the stakeholders wouldn't let a neural network run on a real vehicle or building until it was thoroughly certified through whatever criteria they had. Easier with a smaller network.