4 ms·
How does the speed of this model compare to other LLMs? I see lots of accuracy benchmarks, like HellaSwag, but are there performance benchmarks out there as wel
by jsf01 3y ago
How does the speed of this model compare to other LLMs? I see lots of accuracy benchmarks, like HellaSwag, but are there performance benchmarks out there as well?
- logicchains 3y agoIt entirely depends on the speed of your hardware, but roughly we'd expect it to be 3.5 times slower than Falcon 40B.
- jsf01 3y agoEither on a standardized set of hardware or relative to other models. Performance benchmarks exist for all sorts of compute intensive things, so surely there’s at least one for LLMs?