On a 3090 (24gb vram), same prompt & quant, I can report more than double the tokens per second, and significantly faster prompt eval. total_duration:

On a 3090 (24gb vram), same prompt & quant, I can report more than double the tokens per second, and significantly faster prompt eval. total_duration: 10530451000 load_duration: 54350253 prompt_eval_count: 36 prompt_eval_duration: 29000000 prompt_token/s: 1241.38 eval_count: 460 eval_duration: 10445000000 response_token/s: 44.04 Fast prompt eval is important when feeding larger contexts into these models, which is required for almost anything useful. GPUs have other advantages for traditional ML, whisper models, vision, and image generation. There's a lot of flexibility that doesn't really get discussed when folks trot out the 'just buy a mac' line. Anecdotally I can share my revealed preference. I have both an M3 (36gb) as well as a GPU machine, and I went through the trouble of putting my GPU box online because it was so much faster than the mac. And doubling up the GPUs allows me to run models like the deepseek-tuned llama 3.3, with which I have completely replaced my use of chatgpt 4o.