3 ms·
Amazing writeup. And what a heavy hitter of an inaugural blog entry... This might be the right time to ask: So, on the one hand, this is what it takes to pack
by schaefer 2y ago
Amazing writeup. And what a heavy hitter of an inaugural blog entry...
This might be the right time to ask: So, on the one hand, this is what it takes to pack 192gb of Nvidia flavored vram into a home server.
I'm curious, is there any hope of doing any interesting work on a MacBook Pro Which currently can be max-spaced at 128 GB of unified memory (for the low, low price of $4.7k).
I know there's no hope of running cuda on the macbook, and I'm clearly out of my depth here. But the possibly naive day-dream of tossing a massive LLM into a backpack is alluring...
- Eisenstein 2y agoDownload kobodlcpp and give it a try. It is a single exec and uses metal acceleration with an Apple Arm CPU.
- schaefer 2y agoThanks!