3 ms·
It is a bug, it's been clearly discussed under the issues of LLaMA.cpp. The wait time is far from atrocious, with a 5600x and the 13B I would wait 15 seconds be
by loufe 4y ago
It is a bug, it's been clearly discussed under the issues of LLaMA.cpp. The wait time is far from atrocious, with a 5600x and the 13B I would wait 15 seconds before input starts for a relatively complex prompt.