3 ms·If you really need, 170B parameter model can infer a few tokens per minute on commodity hardware.by lostmsu 3y agoIf you really need, 170B parameter model can infer a few tokens per minute on commodity hardware.