3 ms·
You could keep multimodal projector (understanding of audio, images & PDFs) in system RAM with `--no-mmproj-offload` in llama.cpp. Of course, then it is not acc
by srigi 5mo ago
You could keep multimodal projector (understanding of audio, images & PDFs) in system RAM with `--no-mmproj-offload` in llama.cpp.
Of course, then it is not accelerated with GPU, but you save its VRAM.
- msp26 5mo agoInteresting, I might try that, thanks!