3 ms·
Looks their performance is better than llama.cpp - https://news.ycombinator.com/item?id=37018989 https://news.ycombinator.com/item?id=37018989 - and scales to b
by Lindon4290 3y ago
Looks their performance is better than llama.cpp - https://news.ycombinator.com/item?id=37018989 https://news.ycombinator.com/item?id=37018989 - and scales to batches of prompts.
- polishgladiator 3y agoActually no -- that post shows they are not performing measurements and comparisons correctly. These are not serious people.