2 ms·
ML Engineer: Why does inference for this model take 0.9s per-call?! Data scientist: I have no idea, inferences take 0.1s on average in my environment? I jest
by monkeybutton 5y ago
ML Engineer: Why does inference for this model take 0.9s per-call?!
Data scientist: I have no idea, inferences take 0.1s on average in my environment?
I jest but I've also lived this experience with data scientists developing an algorithm on Windows with one set of wheels, and the same code being deployed to Linux with a different set of binaries and the whole thing running 10x slower. We fixed it, but it was an unnecessary headache.