1. I wouldn't consider Mac Studio ($7,000) a customer product. 2. Yes, and my MBP M1 Pro can run quantized 34b models. My point was that when you do MoE, memor

1. I wouldn't consider Mac Studio ($7,000) a customer product. 2. Yes, and my MBP M1 Pro can run quantized 34b models. My point was that when you do MoE, memory requirements suddenly become too challenging. A 7b Q8 is roughly 7GB (7b parameters × 8 bits each). But 8x of that would be 56GB, and all of that must be in memory to run.