I’m working on a formula that factors ML model run-time cost (speed / memory-usage) into its performance evaluation. I generated a very simple scoring function

I’m working on a formula that factors ML model run-time cost (speed / memory-usage) into its performance evaluation. I generated a very simple scoring function from economic first principles. Now, I’m trying to test performance on cumbersome transformer models published by Google / FB / Microsoft etc in order to prove that many of these models are not “state-of-the-art” if run-time cost is taken into account.