3 ms·
Not really, a 13B 4 bit fits in about 8GB of RAM. But you have to factor in the unified memory aspect of M series chips. Your “vram” and ram are from the same p
by syntaxing 3y ago
Not really, a 13B 4 bit fits in about 8GB of RAM. But you have to factor in the unified memory
aspect of M series chips. Your “vram” and ram are from the same pool. So your computer shares the same pool to run the OS and application hence why you can only load a 13B 4bit for 16GB RAM.