3 ms·
Even a small startup, a researcher or a tinkerer can get a cloud instance with a beefy GPU. Also of note, Apple's M1 Max/Ultra should be be able to run it on th
by iknowstuff 3y ago
Even a small startup, a researcher or a tinkerer can get a cloud instance with a beefy GPU. Also of note, Apple's M1 Max/Ultra should be be able to run it on their GPUs given their 64/128GB of memory, right? That's an order of magnitude cheaper.
- mejutoco 3y agoI am confused. Those amounts are ram, not gpu ram, aren‘t they? Macs cpus are impressive, but not for ml. A most realistic one for a consumer is a 4090 rtx 24 GB. A lot of models do not fit in that, so A6000 48GB and over for some professional cards. That might be around 9000€ already.
- piperswe 3y agoApple Silicon has unified memory - all memory is accessible to both the CPU and GPU parts of the SoC.
- karmasimida 3y agoBut they comes at max 32GB model?
- mkl 3y agoMac Studio (desktop) is up to 128GB, and Macbook Pro is up to 96GB.
- deleted 3y ago[deleted]
- codedokode 3y ago> Macs cpus are impressive, but not for ml On Mac GPU has access to all memory.
- himlion 3y agoI overlooked the unified memory on those machines. Can it really run this performantly?
- ukuina 3y agoI run Vicuna quite well with my M1 Pro, 32GB.