3 ms·
The vRam for example limits you running local llms. Basically, the formula for this is: 1GB of vRAM can handle 1B parameters. There are ways to overcome this, b
by exiguus 1y ago
The vRam for example limits you running local llms. Basically, the formula for this is: 1GB of vRAM can handle 1B parameters. There are ways to overcome this, but the easiest way for the best performance, is just to have enough vRAM.
- marssaxman 1y agoIf running local LLMs were a thing I ever wanted to do, I'm sure that would be something that mattered to me.