2 ms·
It depends if you want ease or speed and if you are batching. Ease? Probably ollama Speed and you are batching on gpu? vLLM
by whimsicalism 3y ago
It depends if you want ease or speed and if you are batching.
Ease? Probably ollama
Speed and you are batching on gpu? vLLM