5 ms·

2xA6000 NVLinked Ampere can run 70B 8-bit which is almost as good as fp16. I bought another A6000 just for that.