3 ms·
Pretty sure llama.cpp can already do that
by happyPersonR 1y ago
Pretty sure llama.cpp can already do that
- TYMorningCoffee 1y agoI forgot to clarify dealing with the network bottleneck
- moralestapia 1y agoJust my two cents from experience, any sufficiently advanced LLM training or inference pipeline eventually figures out that the real bottleneck is the network!