3 ms·
96GB of unified RAM. How much of that is available to the graphics cores? I haven't tested a later model but the M1 Max would max out at 16GB VRAM regardless of
by paxys 2y ago
96GB of unified RAM. How much of that is available to the graphics cores? I haven't tested a later model but the M1 Max would max out at 16GB VRAM regardless of how much the machine had.
There's a reason companies are setting up clusters of A100s, not MacBooks.
- fragmede 2y agohttps://techobsessed.net/2023/12/increasing-ram-available-to-gpu-on-apple-silicon-macs-for-running-large-language-models/ https://techobsessed.net/2023/12/increasing-ram-available-to... says it's tunable via the terminal down to 2GiB for reserved for the OS with the rest allocated for GPU use.
- fennecfoxy 2y agoNot only that but Apple's ram is 0.5TB/s pretty much, a 4090 gets 1TB/s. I feel like the discrete card is the better value proposition because: nobody should need to be running 80GB models on a laptop, I feel this is more in the high perf/research area, you could argue that it could be a useful tool as a co-pilot but you've tuned your machine to use all ram for the model...you can't do anything else. Additionally, it's such a specific use case for the machine that trying to sell it would be hard, whereas I can hock off a GPU to someone doing data, ML, gaming, video editing, etc.