4 ms·
I did something similar but using a K80 and M40 I dug up from eBay for pennies. Be advised though, stay as far away as possible from the K80 - the drivers were
by axegon_ 2y ago
I did something similar but using a K80 and M40 I dug up from eBay for pennies. Be advised though, stay as far away as possible from the K80 - the drivers were one of the most painful tech things I've ever had to endure, even if 24GB of VRAM for 50 bucks sounds incredibly appealing. That said, I had a decent-ish HP workstation laying around with 1200 watt power supply so I had where to put those two in. The one thing to note here is that these types of GPUs do not have a cooling of their own. My solution was to 3d print a bunch of brackets and attach several Noctua fans and have them blow at full speed 24/7. Surprisingly it worked way better than I expected - I've never gone above 60 degrees. As a side efffect, the CPUs are also benefiting from this hack: at idle, they are in the mid-20 degrees range. Mind you, the noctua fans are located on the front and the back of the case: the ones on the front act as an intake and the ones on the back as exhaust and there's two more inside the case that are stuck in front of the GPUs.
The workstation was refurbished for just over 600 bucks, and another 120 bucks for the GPUs and another ~60 for the fans.
Edit: and before someone asks - no I have not uploaded the STL's anywhere cause I haven't had the time but also since this is a very niche use case, though I might: the back(exhaust) bracket came out brilliant the first try - it was a sub-millimeter fit. Then I got cocky and thought that I'd also nail it first try on the intake and ended up re-printing it 4 times.
- JKCalhoun 2y agoCurious what HP workstation you have?
- egorfine 2y ago> K80 - the drivers were one of the most painful tech things I've ever had to endure Well, for a dedicated LLM box it might be feasible to suffer with drivers a bit, no? What was your experience like with the software side?
- deadbabe 2y agoWhat’s the most pain you’ve ever felt?
- BizarroLand 2y agoWhat kind of performance did you get out of that?
- yjftsjthsd-h 2y ago> Be advised though, stay as far away as possible from the K80 - the drivers were one of the most painful tech things I've ever had to endure, even if 24GB of VRAM for 50 bucks sounds incredibly appealing. I thought the problem was that those cards have loads of RAM but lack really important compute capabilities such that they're kind of useless for actually running AI workloads on. Is that not the case?
- almostgotcaught 2y ago> Is that not the case? it is - they're laughably slow and not even supported by latest CUDA > NVIDIA Driver support for Kepler is removed beginning with R495. CUDA Toolkit development support for Kepler continues through CUDA 11.x.
- GTP 2y agoBut Deepseek R1 doesn't use CUDA, so maybe for this specific case, it isn't a big deal?
- almostgotcaught 2y ago> it isn't a big deal? friend you shouldn't make comments like this unless you understand the definitions of the words. Deepseek wrote some parts of their kernels using PTX. newsflash: PTX support for features is lockstep with CUDA support for the same features ie the fact that CUDA doesn't support it means you couldn't write the PTX to use those features either.
- therealfiona 2y agoIt is poor form to condemn someone from asking a question. Thank you for providing the information to clear up ignorance though.
- almostgotcaught 2y agothis is a question: > is deepseak's use of PTX instead of CUDA relevant here? this is a conclusion/assumption thinly veiled as a question > Deepseek R1 doesn't use CUDA, so ... it isn't a big deal? note, genuine questions don't already presuppose an answer.