4 ms·
2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.
by rootlocus 4mo ago
2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.
- horsawlarway 4mo agoYes, today is not a great time to purchase hardware. When I bought, I paid $850 a piece. And I needed one anyways for the gaming I was going to do. My guess is the next good time to buy is going to be 24-36 months from now, depending on how the AI bubble goes. --- I'll add to this, I personally don't like Apple hardware (not so much related to the hardware as their company philosophy) but their machines with unified memory (or AMDs latest unified memory offerings) get pretty equivalent speeds to my 3090s, and are probably a much better modern entrypoint to local llms. There's a reason the joke is that Silicon Valley software devs bought up all the Mac minis for OpenClaw. You can get a 48gb unified RAM M4 pro mac mini for ~2k. If you're not going to do much else with the machine, it's what I'd pick as my budget inference device right now. Spend a year of claude now, get ~150tok/s for the next decade (plus) for ~free. If you want more capable and are willing to spend a little more, go with the newer Ryzen AI Max+ 395 machines. You'll spend less on power too. My last suggestion would be to go buy an RTX3090 at this point. You can do a lot better for a lot cheaper.
- tracker1 4mo agoIf you're willing to go the AMD route, the AMD Radeon Pro R9700 definitely looks interesting for the price compared to NVidia.
- felooboolooomba 4mo agoCan we also run LLMs on Radeon?
- lloyd-christmas 4mo agoI run qwen 27B:Q4 @ 130k context at 50 t/s on a single R9700, and have a 7900XT that runs mellum 12B:Q8 as its subagent. R9700s do really well at low wattage and underclocking as well. It's designed to run at 300W, mine is throttled at 210W, and only had an 8% slowdown. If I had somewhere else to put my desktop in my house, I'd bump it up to 240W and there would be zero perf degradation.
- nyrikki 4mo agoYou can get 60tps with three 1080tis and the sparse model, and I bet two 16gb 5060tis would do the same for ~1200. One 3090 is enough for a useful system, even on an old am4 host.
- jononor 4mo agoDual 5060ti 16gb does over 100 tok/s on 35B A3B. Even with PCIE Gen 4 x4, which quite a lot of motherboards can do. Though Gen 4 x8 or Gen 5 x4 is slightly faster. Misc working notes on this hardware combo here, https://github.com/jonnor/embeddedml/tree/master/handson/micropython-vibes#testing-slower-pcie-connectivity-dual-5060 https://github.com/jonnor/embeddedml/tree/master/handson/mic...
- freetonik 4mo agoThat's also years of top tier PC gaming, if you're into that.
- augusto-moura 4mo ago2x RTX3090 is extremely overkill for gaming, you can run any released game on earth on ultra for much less
- googletron 4mo agowhat?
- kakacik 4mo agoAFAIK nvidia cards dont work in tandem (aka sli in the past) very well these days. So that aint true. Also, 2 gens old means bad performance at ray tracing, abysmal path tracing if at all. Pretty sure it can't run smoothly CP2077 in native 4k without dlss upscalers with all on ultra.
- himata4113 4mo agoYou can have the 2nd card as an offload for upscaling, frame generation and whatnot.
- irishcoffee 4mo agoWhen I'm not running models I use the 2nd one in a pass-thru configuration to a windows vm for various things, usually gaming.
- overgard 4mo agoHaving a second card doesn't really work well for gaming.
- drnick1 4mo ago1x RTX3090 is absolutely not overkill for gaming however. Nowadays it's barely enough to get 60FPS in 4K in some recently released games. But the shocking part is that my 3090 is still probably worth as much as when I bought it about 4 years ago.
- sieabahlpark 4mo ago[dead]
- jmuguy 4mo agoOr a really excellent experience playing Satisfactory with the settings cranked up, which is priceless.
- flowerthoughts 4mo agoIn 3.6 years, chances are they are still worth $3k. Unless some new chip fab pops up that can spam the chip market. Even if the AI bubble bursts, I doubt we'll see high-RAM GPUs sell off.
- tripleee 4mo agoChrist GPU prices have gotten crazy How do AMD cards perform with LLMs? A 9070 is sold for ~$600 and has 16GB VRAM
- lambda 4mo agoThat should do pretty well. Memory bandwidth is the biggest bottleneck for token generation, at 644 GB/s you should be able to do pretty well on a 9070, while prompt proessing is more compute bound and Nvidia tends to have the edge there. 16 GiB won't fit you much, so you'd probably want at least 2x, and preferably 3x of those, and then you need a motherboard, power, etc. that can handle that.
- overgard 4mo agoIn my personal experience, I wouldn't bother with 16GB cards for coding -- the useful models are _slightly_ too large to work at any reasonable speed
- toyg 4mo agoThat's not my experience, and the trajectory is good anyway - what doesn't work perfectly today will be just fine in a few months. In a quickly moving field, it's amazing how much money one can save by overcoming FOMO and not living on the bleeding edge. It's like waiting for Steam sales, the games will be just as good.
- 4mo ago
- overgard 4mo agoAssuming the $100/m claude subscription is still around in three years.
- reddalo 4mo ago[dead]
- booi 4mo agowe will be lucky if it's still around in 3 months..
- oofbey 4mo agoI think there’s a reasonable argument that a burst bubble will cause prices to drop. Prices are very high because they’re trying to justify these trillion dollar valuations on IP alone. If that fantasy goes away then prices will fall down to just silicon and electricity, which looks more like Chinese model prices. Hard to say how it will play out but the direction isn’t obvious to me.
- matheusmoreira 4mo agoThose GPUs can also play video games or mine cryptocurrency. They can also be sold later. We should own things, not rent them. We should all do what we can to keep the fabled 2030 agenda at bay.
- driverdan 4mo agoIf you pay $2200 for a 3090 you're a sucker. They're not worth anything close to that.
- fluoridation 4mo agoLook in the used market, not new. There must some that can be had for much, much less than that.
- picofarad 4mo agoMy 3090 was $700 shipped. Shop around. I'd expect to pay "thousands" for the nodded 3090 with 48+ gigs of vram.
- 650REDHAIR 4mo agoIt was ~8mo ago. Not anymore.
- throwawayffffas 4mo agoI cheaped out and got 2 7900XT I get about 80 tps on qwen3.6 35b a3b. The cost when I got them before the memory crunch was $1400ish. On retrospect I should have forked over a couple hundred more and gotten the 7900XTX for the extra VRAM.