A reasonable quantization (Q4_K_M) is 4GB on disk and uses 6.5 GB of RAM. It will run at reasonable speed on an iPhone 15 Pro today. Using a smaller model or a

A reasonable quantization (Q4_K_M) is 4GB on disk and uses 6.5 GB of RAM. It will run at reasonable speed on an iPhone 15 Pro today. Using a smaller model or a more efficient quantization on a phone two years from now (aka whatever's in their lab today) will be easier than running the latest First Person Shooter. Welcome to the future. Edit: I see a few comments about hundreds of GB. That works for storage, but DRAM consumes battery just by existing, and that's been keeping RAM sizes from increasing the way storage has.