4 ms·
Does each parameter need double the allocation size?
by lynguist 3y ago
Does each parameter need double the allocation size?
- syntaxing 3y agoNot really, a 13B 4 bit fits in about 8GB of RAM. But you have to factor in the unified memory aspect of M series chips. Your “vram” and ram are from the same pool. So your computer shares the same pool to run the OS and application hence why you can only load a 13B 4bit for 16GB RAM.