3 ms·
Can system RAM be used for context (albeit at lower parsing speeds)?
by datameta 1y ago
Can system RAM be used for context (albeit at lower parsing speeds)?
- ivape 1y agoYeah, but it sucks. In fact, if you get the wrong graphics card and the memory bandwidth/speeds suck, things will suck too, so RAM is even worse (other than m1/m2/m3 stuff).
- datameta 1y agoWhat if one is okay with walking away and letting it run (crawl)? Then context can be 128 or 256GB system RAM while the model is jammed into what precious VRAM there is?