It's more likely that you want a lot of compute for a very little amount of time each day - which makes centralised/cloud processing the most obvious answer. I

It's more likely that you want a lot of compute for a very little amount of time each day - which makes centralised/cloud processing the most obvious answer. If I want a response within 100ms, and have 1000 AI-queries per day, that would only be about 2 minutes of aggregated processing time for your AI box per day. It's less than 1% utilised. If the same box is multiuser and on the internet, it can probably serve 50-100 peoples queries concurrently. The converse is that if you put something onto the cloud, for the same cost you might be able to effectively get 50x the hardware per user for the same cost (i.e. rather than have 1 AI box locally with 1 GPU for each of the 50 users, you could have 1 AI box with 50 GPU's which is usable by all 50 users).