3 ms·
How about the iMac Pro? Would that work? I was able to put 128gb in it (not as easy as the regular iMac but possible).
by NSUserDefaults 4mo ago
How about the iMac Pro? Would that work? I was able to put 128gb in it (not as easy as the regular iMac but possible).
- wazoox 4mo agoI've been running various models on a Mac Pro 2013 (8 cores, 32 GB RAM) at about 8 to 10 t/s for months. It's not fast, but it's more than enough for many actual tasks, in particular background tasks. An iMac pro will do just as well I suppose.
- fooker 4mo agoWhat are the tasks that do well with 8-10 t/s ?
- wazoox 4mo agoThe sort of task you don't expect to end immediately. If extracting data from a bunch of PDFs takes 1 hour or the whole night, that doesn't make much difference to me. It's not fast enough for auto completion and slightly too slow for chat (but bearable IMO).
- neverartful 4mo agoI have and use a Mac Pro 2013 too. Mine is 8 cores with 64 GB RAM. I haven't used mine for any LLM workloads, but it does just fine for most stuff. My biggest concern with it is the OS. I'm still running macOS (the latest supported version) but it's getting continually further out-of-date security wise all the time.