I think it depends on how much you expect your model to be used, and how quickly the model needs to react. The higher either of those becomes the more likely yo

I think it depends on how much you expect your model to be used, and how quickly the model needs to react. The higher either of those becomes the more likely you'll want to specialize. If you expect your model to be used a lot, and you don't have a way to distribute that pain (for instance, having a mobile app run the model locally on people's phones instead of remotely on your data centers) then it ends up being a cost balancing method. A single DGX machine with 8 GPUs is going to cost you about the same as a single engineer would. If cutting the model size down means you can reduce your number of machines that makes increasing headcount easier. The nice thing about data cleaning is that it's also an investment- you can keep using most data for a long time afterwords, and if you're smart then you're building automated techniques for cleaning that can be applied to new data coming in.