3 ms·
Right, I meant "fits" in the sense of I can load the whole thing into some combination of system RAM and GPU at llama-server launch. 15 tk/s isn't useless if y
by walrus01 1mo ago
Right, I meant "fits" in the sense of I can load the whole thing into some combination of system RAM and GPU at llama-server launch.
15 tk/s isn't useless if you can give it big tasks to do overnight, or like ask it to do something and check back 3-4 hours later.
- nicman23 1mo agoyeah specially if you have it do some long task that also has to wait for ie compilation anyways