3 ms·
We can do that, but then we will need to close all applications to run it because it will need 20GB RAM or VRAM to run. It is still more practical to visit sta
by kramerger 3y ago
We can do that, but then we will need to close all applications to run it because it will need 20GB RAM or VRAM to run.
It is still more practical to visit stackoverflow when I have a question.
(still impressive how much this improves every week. Maybe in 2-3 months we have something more useful)
- moonchrome 3y agoI got 64gb ram in my desktop for few hundred €. Laptops overprice RAM (as a price discriminator I guess). But unlike Apple SoC, most GPUs have separate memory - getting a GPU with 24gb ram is very very expensive in comparison.
- midasuni 3y agoI just upgraded my 2020 era thinkpad (t470, which I bought this time last year for less than £300) to 32g memory for £70. 20g of ram doesn’t feel like a lot today.
- 0x6c6f6c 3y agoAlso today: soldered memory and 8GB MBPs that START at $1300.
- deleted 3y ago[deleted]
- fancyfredbot 3y agoThe magic of swap means you can leave those applications open and as long as they're idle they'll just quietly wait for you on disk until you've finished your LLM adventure.
- kramerger 3y agoOn many modern systems swap is quite small and not large enough to hold the entire system. For a 32GB machine, Ubuntu recommends 6GB and redhat recommends 6.4GB (20%). In a VM that number is much much lower. Besides, some libraries allocate memory in such way that you get artificial memory spikes in physical memory. I know this because I spent this weekend debugging crashes in an LLM pipeline :(
- fancyfredbot 3y agoUbuntu recommends a /minimum/ of 6GB, not sure it's fair to say they recommend you actually use the minimum though.