4 ms·
> I think the main advantage here is you COULD run it, even it it takes a while. I mean, you COULD run it before as well, even if you don't have enough RAM or
by diggan 2y ago
> I think the main advantage here is you COULD run it, even it it takes a while.
I mean, you COULD run it before as well, even if you don't have enough RAM or VRAM, by using something like `zram`. It'd probably be even slower (and border-line usable, depending on the use case), but it's not impossible to get things to run.
- prometheon1 2y agoZram compresses part of the data in ram, right? Can an LLM be compressed?
- xena 2y agoNot really. The weights are mostly random numbers.