5 ms·
I recently saw a rack server of 8x SXM2 P100's (so, the 2016 equivalent of this) for sale for $3600 on ebay. And you can buy individual P100 pci-e versions for
by mk_stjames 4y ago
I recently saw a rack server of 8x SXM2 P100's (so, the 2016 equivalent of this) for sale for $3600 on ebay. And you can buy individual P100 pci-e versions for about $300 now. They retailed for about $7600 each back in 2016.
I wonder how long it'll be until A100's depreciate that amount.
I see articles from "Serve The Home" pop up now often and I've yet to see one discussing hardware that I've ever come across in... someone's home.
- Patrick-STH 4y agoSTH has been around for 14 years and so we have tested these big servers from the P100 days (see our DeepLearning12 build) and newer. Home was the /home/ directory in Linux. We do have some content for homes. Check out our TinyMiniMicro series as an example.
- trvr 4y ago>> Home was the /home/ directory in Linux. I have never (until reading this) thought of it in that context. Was that the original meaning for the name, or did it shift over time?
- Patrick-STH 4y agoIt is in the "About" page :-) I always get confused by why folks ask why we have server content. The Wall Street Journal covers more than just one road in NYC.
- trvr 4y agoUnderstood. But what about this? https://web.archive.org/web/20110201124803im_/http://www.servethehome.com/wp-content/uploads/2010/08/STH-Logo102.jpg.pagespeed.ce.ADlm7Tggpk.jpg https://web.archive.org/web/20110201124803im_/http://www.ser...
- Patrick-STH 4y agoNow that is a throwback! That old logo was created with GIMP at a table outside NetApp. STH started at a time when the SMB market (and VERY high-end homes) were using Dell, HP, and IBM rackmount gear off-lease. The original STH idea came from me learning Linux as an alternative to Windows for that segment. STH was what I used to learn the rest of the market outside of work. The first product we were sent for a review was a rackmount case and that started us on the path of reviewing new gear ~2010-2011 when it was a part time blog instead of having a team of folks working on the site. I still try to keep 15-20% of our content in the SMB/ high-end home arena.
- deleted 4y ago[deleted]
- metadat 4y agoDirect link: https://www.servethehome.com/introducing-project-tinyminimicro-home-lab-revolution/ https://www.servethehome.com/introducing-project-tinyminimic...
- smoldesu 4y agoThose P100s are so cheap because they're more trouble than they're worth. They don't mount easily in most cases, will consume a lot of power at idle, generate lots of heat without active cooling and require a Linux machine to use properly. It's probably possible to engineer a home-scale inferencing network like that, but I've had plenty of success just running GPT-Neo on off-the-shelf ARM chips. It's not worth the extra $800, exploding power bill and engineering effort to get diminishing returns.
- mk_stjames 4y agoOh I know for sure, it's still just amusing to me to look up these just slightly older cards and see them going for so little compared to what they were new; I do also look at various benchmarks and comparisons over and over because I would like to build a home gpu server for some machine learning fun and I find it interesting that the TFLOPS/$ numbers always come out very much linear... A card that gives 2x performance costs 2x more.... if cards depreciate 50%, it's because new cards have been launched that have double performance for the same price... etc... Even simpler, the CUDA cores-per-dollar is pretty much a constant. What isn't taken into account there is various optimizations new cards have that speed up training or inference, like better use of the tensor cores. At some point incompatibility with newer CUDA compute levels makes the older cards much less useable.
- dzdt 4y agoNot directly applicable for large language models, I was doing some searching for cards with lots of double-precision calc performance. Nvidia has been leaving that out of all of the consumer versions of hardware; cheap P100's seem to be by far the highest DP FLOPS per dollar solution available right now.
- mk_stjames 4y agoI have an interest in doing lattice Boltzmann fluid dynamics simulation on GPUs, and this is something I've noticed with the P100's as well. That and memory bandwidth is super important for such calcs, along with total memory. Commercial codes I've seen aren't using any ML ops compute optimizations or tensor cores and anything like that, it's mostly just OpenCL based and thus the P100s throughput per dollar is pretty high.
- rldjbpin 4y agoit might land in someone's home...after 7 years of launch like the p100s
- rsync 4y ago"I see articles from "Serve The Home" pop up now often and I've yet to see one discussing hardware that I've ever come across in... someone's home." You're just not visiting the right homes ...
- kamikaze675 4y agoAgree, I have 14 V100's in my home but its pretty rare. I use it with fiber backhaul to supply services for marketplaces like vast.ai
- asciimike 4y agoWith residential power costs, is it economically viable to do this? What's your utilization look like?
- buildbot 4y agoIn Washington power can be 2.95¢ per kWh so yes, very!
- asciimike 4y agoFrom the utility? That's crazy.
- buildbot 4y agoYep! Actually 2.7c: https://www.chelanpud.org/my-pud-services/rates-and-policies https://www.chelanpud.org/my-pud-services/rates-and-policies
- lostmsu 4y agoWhere? Is there also 1G or better 5G+ fiber?
- kamikaze675 4y agoYou can get 1Gb, 2GB and 5Gb fiber to the home from ATT, TDS, Google etc. for backhaul.
- lostmsu 4y agoI mean in the area with 3c/kWh electricity that the above poster mentioned.