4 ms·
From my experience if it doesn't fit on vram it is rarely worth to bother except for a few narrow tasks. For example make an essay about something where you do
by DanielHB 2mo ago
From my experience if it doesn't fit on vram it is rarely worth to bother except for a few narrow tasks.
For example make an essay about something where you don't actively engage with the LLM after the initial prompt. So mostly one-shot prompts.