I have a M2 with 64GB and find that it’s fine. Larger models that don’t fit are only moderately better, while much slower. I’d want higher memory bandwidth ove

I have a M2 with 64GB and find that it’s fine. Larger models that don’t fit are only moderately better, while much slower. I’d want higher memory bandwidth over more memory. The next five years will push for better models that are smaller. Smaller is faster and more useful. I feel like we are in the brute force phase of model development and that it will pass. Running 405B param model in 16bit on my laptop would be neat, but I’d stop after the novelty wore off.