4 ms·
I like the cut of your jib and your experience matches mine, but without real numbers this is all just piss in the wind (as far as online discussions go).
by Y_Y 1y ago
I like the cut of your jib and your experience matches mine, but without real numbers this is all just piss in the wind (as far as online discussions go).
- BoorishBears 1y agoYou're right, it's unfortunate I didn't keep the benchmarks around: I benchmark a lot of configurations and providers for my site and have a script I typically run that produces graphs for various batch sizes (https://ibb.co/0RZ78hMc https://ibb.co/0RZ78hMc) The performance with offloading was just so bad I didn't even bother proceeding to the benchmark (without offloading you get typical H100 speeds)