3 ms·
I'd love to see how this compares against Llama.cpp on speed. Do anyone any benchmarks that compare PyTorch.compile vs Llama.cpp?
by syrusakbary 3y ago
I'd love to see how this compares against Llama.cpp on speed. Do anyone any benchmarks that compare PyTorch.compile vs Llama.cpp?
- chillee 3y agoThis person claims to have compared llama.cpp against gpt-fast on a 4090, and found gpt-fast about 20% faster. https://twitter.com/zeuxcg/status/1730450360895242355 https://twitter.com/zeuxcg/status/1730450360895242355
- syrusakbary 3y agoAwesome, thanks!