long story short, for a machine that has about 5+ years of shelf life it isn't a bad value prop to go and buy the absolute top end, especially if you expect som

long story short, for a machine that has about 5+ years of shelf life it isn't a bad value prop to go and buy the absolute top end, especially if you expect some roi out of it, just be aware that if you don't want the top end for llm the m2 currently provides more value because m2 has memory bandwith than the m3 at the middle range. memory bandwidth is the key to model speed, and memory size is what enable you to use larger model (quantization let you push thing further, to a point) so one thing to note is that on the M3 pro/max only the top end model gets the full bandwidh, while the m1/m2 pro enjoy full bandwidth from a smaller memory size. this may be important if you value speed above model size or vice versa. M2 Pro, M2 Max get approximately 200 GB/s and 400 GB/s, but things are more complicated for m3: M3 Pro gets 150mb/s, and M3 max gets 300mb/s at 36gb and 400mb/s at 48gb few more things to note: it's absolutely fine to go and play around with llm but even with a llm monster machine there's nothing wrong in starting with smaller models and learning how to squeeze the maximum amount of work out of them. the learning do transfer to larger model. this may or may not be important if at some point you'll want to monetize or deploy to production what you learned. while the mac itself is a good investment for personal use, once you move to servers, cost skyrockets with model size, because of supply constraints on 40gb+ memory gpus. if you are dependent to a 70b parameter model, you'll have a hard time to make a cost effective solution. if it's stricly to playing around, you can disregard this concern even if you're playing around, a 70b is going to run at 7 tokens / second, which is fine for a local chat, but if you are writing a program and need inference in bulk, it's fairly slow. another thing of note is that while the field is still undecided on which size and architecture is good enough, the moltitude of small fish experimenting with tuning and mix of instructions are largely experimenting on smaller models. currently my favorite is openhermes-2.5-mistral-7b-16k, but it's not an indication that mistrals are strictly better than llama2, more an indication that experimenting with 7b is more cost effective for third parties without access to gpu than experimenting with 13b, and so you'll find 13b model kinda stagnating, with many of them trained in a period where people didn't really know the best parameters for finetuning and are so to say a bit behind the curve. a few tuners are working at 70b models, but these seems to be pivoting to mixtrals and the likes, which will cause a similar stagnation on the top end, that is, until llama3 or the next mixtral size drops, then, who knows