3 ms·
(afaik) it does not support batching, and some other server focused features. I was talking more about a high throughput server, which is not appropriate for o
by brucethemoose2 3y ago
(afaik) it does not support batching, and some other server focused features.
I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.