2 ms·
I think it depends on your model size Fits into 32gb: 5090 Fits into 64gb - 96gb: Mac Studio Fits into 128gb: for now 395+ $/token/s, Mac Studio
by KingOfCoders 1y ago
I think it depends on your model size
Fits into 32gb: 5090
Fits into 64gb - 96gb: Mac Studio
Fits into 128gb: for now 395+ $/token/s,
Mac Studio if you don't care about $
but don't have unlimited money for Hxxx
This could be great for models that fit 128gb and you want best $/token/s (if it is faster than a 395+).
- timc3 1y agoThe 395 although it can be supplied with 128GB can’t use all that for VRAM (unless something has changed in the last couple of weeks).
- lhl 1y agoIn Linux, you can set it as high as you want, although you should probably have a swap drive and still be prepared for you system to die if you set it to 128GiB. Here's how you'd set it to 120GiB: # This is deprecated, but can still be referenced options amdgpu gttsize=122800 # This specifies GTT by # of 4KB pages: # 31457280 * 4KB / 1024 / 1024 = 120 GiB options ttm pages_limit=31457280
- KingOfCoders 1y agoFrom YouTube it seems up to 105gb Models disksize work, yes.