4 ms·
Anyone have insight as to whether these will be useful for ML tasks - mostly on pre-trained models? I have bulk jobs that I like to do for things like transcri
by n8cpdx 3y ago
Anyone have insight as to whether these will be useful for ML tasks - mostly on pre-trained models?
I have bulk jobs that I like to do for things like transcription and I'd like to add summarization as well (Whisper + Llama basically). My GTX 1070 is fine for proof-of-concept, but I was planning to build a 4090 box for this stuff.
Nvidia is notorious for nerfing the RTX cards for ML tasks on RAM specifically. Is there any universe in which it makes sense to put my 4090 box budget ($4000+ all in) towards an updated MacBook? I can get a laptop with 128Gb RAM for $4600 and the convenience of not having to run things remote and deal with another OS would be a big win.
- blovescoffee 3y agoIf you want to run ML tasks, a 4090 will smoke any laptop plain and simple. You get to decide about convenience of having a laptop. I wouldn’t want to deal with the inconvenience of a non-nvidia stack.
- threeseed 3y ago4090 has 32GB of available memory versus up to 128GB on the Mac. That allows for some training to be slow but at least possible. When you look at benchmarks the memory can make a difference especially for LLMs: https://www.reddit.com/r/LocalLLaMA/comments/16o4ka8/running_ggufs_on_an_m1_ultra_is_an_interesting/ https://www.reddit.com/r/LocalLLaMA/comments/16o4ka8/running...
- NorwegianDude 3y agoThats not true. RTX 4090s have 24GB.
- rubyn00bie 3y agoHuh? The 4090 has 24GB of RAM which is independent of the system RAM.
- langarus 3y agoHave you tried running remote via parsec (if you want the full desktop experience)? With proper internet connection you won't realize you are on a remote connection.
- n8cpdx 3y agoI will make a note to try Parsec. I've been using RDP with both devices connected to the same LAN over ethernet. The experience is very good, the biggest annoyance is that I have to deal with Windows.