8 ms·

Build a fast deep learning machine for under $1K

147 points by kqr2 10y ago

RichardHeart 10y ago
If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board. Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better. Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running. If you chose an ITX platform for deep learning fun, uh, you should really add the couple shoeboxes extra space to your platform and have 4x-7x the power available to upgrade into.
- siscia 10y ago
  Try to bring good in the conversation. What piece you would have use? Why? What tradeoffs you have considered? You sound expert in the field, try to share your knowledge with the community in a constructive way so that we can all benefit from it. I develop software for living and still I haven't any clue of what you said, while I followed quite well the article and I thought he was making reasonable choice. Please show me better.
  - crawfordcomeaux 10y ago
    I'd like to second this request and note it took me years to figure out that people saying "focus on the positive" in this context would be satisfied by me phrasing things in terms of what to do rather than what not to do. Concrete examples are key for this approach.
  - croon 10y ago
    Not parent, but I've built my own computers for almost 20 years. As the articles author himself questions, he should have gotten the marginally more expensive CPU, and definitely the GPU with more RAM. * The overclockable CPU doesn't just mean that you very easily could get a 10%+ performance boost without much work, but also that you often (depending on your specific chip) can lower the voltage and make it run much cooler/quieter, which is something I increasingly care about if I'm using it a lot. * He writes that fitting the model in RAM is basically the most important part, but then saves less than a dinner out by basically halving the RAM on the GPU he bought. The chip is otherwise the same though, so performance is only dependent on whether you fill that RAM or not. * The Noctua D15 the parent mentioned is a CPU cooler where Noctua is a long standing high performant brand, and D15 is a specific model with a 15 inch fan and comparable sized heat sink. There are of course other brands, but I myself also usually end up with Noctuas. The reason it's important is that however fast you can dissipate heat from the cpu/case, the less chance of throttling, and the larger headroom for potential overclocks you get. Airflow and room for larger heatsinks is also why he recommended not going for an ITX. A linked benefit is again the potential for a quieter system. I haven't gone much into ML (yet), but I currently have a system with: * I7 6700K (the difference to I5 6600K being higher base clock and hyperthreading, which is more important to computational work than to gaming, so if you have the money, definitely go for the I7) * 32GB DDR4 (as author mentioned, RAM is cheap). The clock/timings on RAM isn't really as important, but try to find the best you can find for a given price point. * An Nvidia GTX 1080: It's not Titan X or Z, but almost, at less price. It definitely blows the budget for a $1000 system, but I agree that the entire 10 series is good. If the limit is a firm $1000, I would get something like this: https://pcpartpicker.com/list/XHV9Fd https://pcpartpicker.com/list/XHV9Fd And if more funds is available, I'd get more storage and RAM, then a better CPU, then a better GPU, then maybe bump the chassis up to an R5 (same brand), possibly another motherboard. In that order. There's always something better, so you compromise based on budget.
    - IanCal 10y ago
      I've got a near identical build, and my process was the same as you. I added a decent pcie nvme drive, and spent more than I guess I could have done on a motherboard that supports up to 128G ram so I can extend it later. I'm waiting for ram to drop in price a bit, or to hit a problem where I need more than 32 sooner, before bumping that up and probably adding more storage. It's been great sshing in from my laptop, submitting a job that completes far more quickly and keeping my laptop cool.
      - croon 10y ago
        I kept my old 512GB Samsung 850 Pro, but will definitely get an nvme when I need more space/speed. Getting another 32 RAM will hold me over until I'll build a new one, so that part I'm not worried about. Yeah, it's supremely fast, my only regret is not having enough time to do something fun with it. For work I'm stuck with the clients platform approved machine, which is not what I would've picked. Tough to complain, but if I'm ever between contracts, I'll likely get into some fun project.
    - deleted 10y ago
      [deleted]
    - hermiti 10y ago
      I have a machine that I am using for deep learning and it has 32GB of ECC RAM paired with a Xeon CPU w/ Quadro, and that 32GB of RAM is gone almost instantly. Now I'm looking at building another machine as it has gotten to the point where I am spending a great deal of time waiting. I would be interested to see what other people are using for their setups, and how that can differ for things like high-resolution style transfer or generic neural networks, etc...and at what point they have to switch from geforce/quadro cards to tesla.
    - mamcx 10y ago
      Was years ago when I build my last PC and I'm out of the game. I'm tempted in build a hackintosh: https://www.tonymacx86.com/threads/hackintosh-cutting-edge-kaby-lake-nvme-ddr4-4k-viable.214249/#post-1433129 https://www.tonymacx86.com/threads/hackintosh-cutting-edge-k... - Intel i7 Kaby Lake - No decided on motherboard. The one that cause me less trouble (for hackintosh) is fine. - GTI 750ti (have) or buy a pascal nvidia. - NVMe drive if possible - 32 GB RAM. - Probably a Thermaltake CORE P3 case. Not decided. I was thinking in use a Liquid Cooler but wonder if the Noctua could be better/less noise?
- rspeer 10y ago
  > Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running. Deep learning toolkits basically have two modes of operation: the CPU way, and the proprietary NVIDIA way. There is no point to putting an AMD chip in a machine you hope to use for deep learning. Some toolkits may go through the motions of supporting AMD via OpenCL, but that's not going to be the case that they make sure works well, or works at all. This is a problem, as NVIDIA is awful at maintaining drivers, results are being published based on "well, my NVIDIA black box decided to do this", and it contributes to deep learning veering toward a local maximum. But specifically choosing to do deep learning with an AMD chip is a pointless sacrifice.
gcp 10y ago
Contrary to the claims there, the CPU does matter: if the network is relatively small, it's easy to get bottle-necked feeding the GPU. Same thing applies to storage, for the same reasons. There's a reason AMD is selling deep learning cards with Flash drives built in. The choice of CPU cooler is silly. The CPU chosen is a 65W model, which will be cooled quietly even by the boxed cooler (and we don't need to care about overclocking with a non-K). By his own admission, the rig draws 250W, so the idea that 650W must be cooled is nonsense, and even then, only a part of that is due to the CPU. It's probably worth looking more closely at the GPU, notably, if that cooler stays quiet when the card is under full load. You do not need a monitor, keyboard or mouse to use such a machine. You can just ssh into it. You might need to borrow a set to set up the BIOS. If you "run out of memory for your applications" on the GPU, have you considered simply lowering the mini-batch size? (See, this is why we got a good CPU, it removes the hurt from feeding smaller batches)
- nl 10y ago
  There's a reason AMD is selling deep learning cards with Flash drives built in. For those not familiar with them, this is the Radeon Pro SSG[1] This has a built in M2 drive, but this is used as video card memory, NOT main storage. It could possibly allow big (HUGE) models or batches, but doesn't prove fast IO is often a bottleneck. Practically no one[2] is using these for deep learning. They are marketed to the oil and gas modelling and visualization market and notably the product page (linked above) makes no mention of neural networks or deep learning, and it has never been mentioned in AMD's publicity. Yes, it might be good for deep learning. And yet, modern deep models do want more memory. But there is a lot that needs to happen for this to be useful: AMD needs to release something like CuDNN and they need to make sure OpenCL is supported as well as CUDA is. [1] https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-ssg-graphics-card-specs-price-release-date/ https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-ssg-g..., http://www.amd.com/en-us/press-releases/Pages/amd-radeon-pro-2016jul25.aspx http://www.amd.com/en-us/press-releases/Pages/amd-radeon-pro... [2] I'm sure you'll find someone. But show me a published paper or any kind, or even benchmarks showing the use of the extra memory somewhere.
  - gcp 10y ago
mrmondo 10y ago
My home server has an i7 4790K, 32GB DDR3 and... a GTX 1080 Gamer Edition that's doing... well, bugger all. Why is the graphics card in there you might ask? Well, it was sitting on my desk collecting dust as I wait patiently for a Pascal driver for eGPUs on macOS but Nvidia had delayed these for long enough that I thought I'd chuck it in my home server, it is being used by ffmpeg for x265/HVEC transcoding with Plex media server but I'm really not even touching it's capabilities - not even close when transcoding a 120GB+ 4K 120FPS video and streaming it wirelessly to my TV that's running a Plex client. The server is running CentOS 7 and has 10x 8TB SATA drives in BTRFS RAID 10 for storage, two Intel NVMe 750 series 1.2 TB SSDs for the OS as well as a couple of 2TB Crucial MX300 SATA SSDs that aren't currently in use for much. So, I have been sitting here wondering how I can make better use of it while it's practically parked in my server while I wait for some Pascal drivers to drop. I'm not so interested in robots / live video processing but it'd be neat if I could leverage its power for something fun or to assist the servers compute performance in some other ways that might be useful or interesting. I was running KVM with PCIe pass through passing the GPU to a guest VM running Steam as a steam streaming box but I got sick of having an OS on the network that I had to worry about Virus's, Malware and annoyingly slow and unreliable updates being installed when you least want it to happen and I got lazy and went back to playing PlayStation when I needed some time out from constructive work / research. Open to ideas to any software I could tinker with if anyone has any interesting suggestions?
- kharms 10y ago
  Not something for you to work on, but something you can do in the mean time: https://en.m.wikipedia.org/wiki/List_of_distributed_computing_projects https://en.m.wikipedia.org/wiki/List_of_distributed_computin...
- j_s 10y ago
  Can you ballpark today's price tag for such a setup? There seem to be several pricing tiers floating around in this discussion and I'd appreciate the chance to see how your setup fits in without having to shop for it.
  - mrmondo 10y ago
singularity2001 10y ago
In Europe it's also very common to just choose the components in a web shop and have them assembled, also for about $1000. (Not looking at Dell) Anyone interested in deep learning should go that path instead of burning their money on Amazon.
- ProxCoques 10y ago
  +1 on that. There are a number of online build-to-order shops here in the UK which offer a truly vast range of component combinations. Better still, the order process automatically checks for compatibility with, for example, your choice of motherboard. Once you have the configuration you want, you just pay up and it arrives professionally assembled ready to rock. I don't think I'll ever by an off-the-shelf machine again.
  - IanCal 10y ago
    I found the premium on these was quite high, although even just buying all the components from a single place added a fair amount to my recent build. Compatibility, price checking and searching are all available with PcPartPicker (UK site: https://uk.pcpartpicker.com/ https://uk.pcpartpicker.com/) which saved me an astonishing amount of time. The hour or two building the machine were worth the few hundred it saved me, but everyone has different priorities.
    - Coding_Cat 10y ago
      I've found that the premium depends on the store and model. Some seem to do relative pricing, others absolute. In the latter case, if you want a big rig it's best to let someone build it for you. Often times in those cases you also get great service because a) if you build PCs for a living you're a computer geek and it's fun to build an insane PC and b) they often use their biggest systems as advertisement. At least, that's what I've seen. +cable management is like black magic to me. If I were to take the money saved as payment for me to get it as nice as those places get it, I'd be below minimum wage.
- jbmorgado 10y ago
amelius 10y ago
But how do you scale cheaply to more GPUs?
- mtanski 10y ago
  Get a ATX instead of ITX mobo. It's likely is cheaper as well.
PascLeRasc 10y ago
I'm not sure why mITX was chosen - in my experience it's almost always more expensive and leaves no room for additional GPU expansion (Or network adapters/whatever card you might need) in the future. You can get 1151 mATX motherboards for more like $60-70 on Amazon instead of the $125 they paid. Also, paying $125 for 16GB of ram but not spending the extra $20 to have your CPU be able to overclock? I'm not sure where OP is building but I can find that much RAM for $20 less than they paid, and overclocking isn't really that difficult or unreliable these days.
- coroxout 10y ago
  I was wondering that too. For something like this maximum room for future expansion seems good. In the past I've used mITX to build low-power, low-noise machines, rather than building a high-powered processing beast. On the other hand, it made the article more interesting for me. I can't really justify a home machine learning rig, but I was thinking I might soon replace that old mITX machine. (edit - just reached the pics - really not sure why you'd go with a mini form factor and then add a 6" heatsink/fan!)
  - PascLeRasc 10y ago
    I also prefer mITX and I have one I've tried to optimize for low noise and efficiency. Just seems ill-suited for this purpose.
- pdog 10y ago
  Most people don't want a giant box under their desk. If you're building a machine learning rig for under $1,000, you don't need room for expansion because additional $200, $400 or even $700 cards are a useless expense at that price range.
- gens 10y ago
  You don't need a powerful cpu for gpu loads.
brudgers 10y ago
If I were building a GPU box on the cheap, I'd start with a used Dell Precision T7xxx series off of Ebay for a <$300 including RAM and a Xeon or two. They have 1100 watt power supplies and lots of slots and the RAM is usually ECC. Odds are it will come with DVD burner and possibly a hard disk and a Windows license...again all for a couple of hundred dollars. Leaving $700 for GPU's while providing reliable high end hardware pretty much designed to run GPU's for the base platform.
throwawayish 10y ago
If you're on a budget it makes little sense if any to buy things like a motherboard, CPU or memory new.
dsacco 10y ago
I think there should be a disclaimer here that while, yes, this will work, it will quickly become suboptimal (perhaps even dysfunctional) for a lot of work as you scale your hobby into something resembling more professional work. I personally would not start with a thousand dollar machine if you have any intention of doing serious computation in the future. While you can trade out parts pretty easy on a custom built machine, you don't want to be in a position where you have to trade out most of the core parts 6 months in. For reference, I built a home PC that I successfully do deep learning and data analysis on (mostly tensorflow and scipy stack) for about ~$10k. It's liquid cooled, has 15 fans, four radiators, an i7-6900K CPU, 128GB RAM, four GTX 1080 GPUs (controversial), four TBs of HDD space and 1TB of SSD space. I don't recommend you start with this at all, but my point is that porting your hardware from point A to point B will be a pain if it comes to it. I used the guide here as a reference about 8 months ago when I built it: http://graphific.github.io/posts/building-a-deep-learning-dream-machine/ http://graphific.github.io/posts/building-a-deep-learning-dr.... My purpose in doing this was, essentially, to pay for electricity rather than AWS/GCP/Azure compute resources (and in that regard it's been very successful!). I know I'm hijacking a thread here to talk about building home machines for professional deep learning work when this story is clearly not intended for that, but I wanted to throw in this perspective so that it's understood this is very different from just "build this machine to start out and upgrade it later." There's a law of diminishing returns here, but in general my point is that I do not think this is a minimum for "start doing deep learning effectively at home." If you want to learn hands on deep learning cheaply, my opinion is that it would be more efficient to use compute resources from a cloud provider before diving into this with a home-based custom machine. tl;dr: The demographic of folks who probably want/should/need to build a home deep learning machine probably has little overlap with the demographic of folks who want to do it non-professionally, or at least with only $1k in resources.
- dbancajas 10y ago
  so do you earn money out of that 10K investment on machines? I didn't read the article but what's the benchmark on deep learning machines? Still TFLOPS?
  - 10y ago
kordless 10y ago
I've been considering a "thing" that allows monetization of an AI-enabled machine by others who need the capacity to do larger scale testing. Anyone interested in this? My contact info is in my profile.
- minimaxir 10y ago
  You won't be able to match price/expenses (fixed with hardware or variable with upkeep) of AWS/GCP due to their economy of scale. Even startups who are doing similar things sublicense from AWS.
- PascLeRasc 10y ago
  How is it better than AWS?

147 points by kqr2 10y ago

If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board. Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better. Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running. If you chose an ITX platform for deep learning fun, uh, you should really add the couple shoeboxes extra space to your platform and have 4x-7x the power available to upgrade into.

siscia 10y ago
Try to bring good in the conversation. What piece you would have use? Why? What tradeoffs you have considered? You sound expert in the field, try to share your knowledge with the community in a constructive way so that we can all benefit from it. I develop software for living and still I haven't any clue of what you said, while I followed quite well the article and I thought he was making reasonable choice. Please show me better.
- crawfordcomeaux 10y ago
  I'd like to second this request and note it took me years to figure out that people saying "focus on the positive" in this context would be satisfied by me phrasing things in terms of what to do rather than what not to do. Concrete examples are key for this approach.
- croon 10y ago
  Not parent, but I've built my own computers for almost 20 years. As the articles author himself questions, he should have gotten the marginally more expensive CPU, and definitely the GPU with more RAM. * The overclockable CPU doesn't just mean that you very easily could get a 10%+ performance boost without much work, but also that you often (depending on your specific chip) can lower the voltage and make it run much cooler/quieter, which is something I increasingly care about if I'm using it a lot. * He writes that fitting the model in RAM is basically the most important part, but then saves less than a dinner out by basically halving the RAM on the GPU he bought. The chip is otherwise the same though, so performance is only dependent on whether you fill that RAM or not. * The Noctua D15 the parent mentioned is a CPU cooler where Noctua is a long standing high performant brand, and D15 is a specific model with a 15 inch fan and comparable sized heat sink. There are of course other brands, but I myself also usually end up with Noctuas. The reason it's important is that however fast you can dissipate heat from the cpu/case, the less chance of throttling, and the larger headroom for potential overclocks you get. Airflow and room for larger heatsinks is also why he recommended not going for an ITX. A linked benefit is again the potential for a quieter system. I haven't gone much into ML (yet), but I currently have a system with: * I7 6700K (the difference to I5 6600K being higher base clock and hyperthreading, which is more important to computational work than to gaming, so if you have the money, definitely go for the I7) * 32GB DDR4 (as author mentioned, RAM is cheap). The clock/timings on RAM isn't really as important, but try to find the best you can find for a given price point. * An Nvidia GTX 1080: It's not Titan X or Z, but almost, at less price. It definitely blows the budget for a $1000 system, but I agree that the entire 10 series is good. If the limit is a firm $1000, I would get something like this: https://pcpartpicker.com/list/XHV9Fd https://pcpartpicker.com/list/XHV9Fd And if more funds is available, I'd get more storage and RAM, then a better CPU, then a better GPU, then maybe bump the chassis up to an R5 (same brand), possibly another motherboard. In that order. There's always something better, so you compromise based on budget.
  - IanCal 10y ago
    I've got a near identical build, and my process was the same as you. I added a decent pcie nvme drive, and spent more than I guess I could have done on a motherboard that supports up to 128G ram so I can extend it later. I'm waiting for ram to drop in price a bit, or to hit a problem where I need more than 32 sooner, before bumping that up and probably adding more storage. It's been great sshing in from my laptop, submitting a job that completes far more quickly and keeping my laptop cool.
    - croon 10y ago
      I kept my old 512GB Samsung 850 Pro, but will definitely get an nvme when I need more space/speed. Getting another 32 RAM will hold me over until I'll build a new one, so that part I'm not worried about. Yeah, it's supremely fast, my only regret is not having enough time to do something fun with it. For work I'm stuck with the clients platform approved machine, which is not what I would've picked. Tough to complain, but if I'm ever between contracts, I'll likely get into some fun project.
  - deleted 10y ago
    [deleted]
  - hermiti 10y ago
    I have a machine that I am using for deep learning and it has 32GB of ECC RAM paired with a Xeon CPU w/ Quadro, and that 32GB of RAM is gone almost instantly. Now I'm looking at building another machine as it has gotten to the point where I am spending a great deal of time waiting. I would be interested to see what other people are using for their setups, and how that can differ for things like high-resolution style transfer or generic neural networks, etc...and at what point they have to switch from geforce/quadro cards to tesla.
  - mamcx 10y ago
    Was years ago when I build my last PC and I'm out of the game. I'm tempted in build a hackintosh: https://www.tonymacx86.com/threads/hackintosh-cutting-edge-kaby-lake-nvme-ddr4-4k-viable.214249/#post-1433129 https://www.tonymacx86.com/threads/hackintosh-cutting-edge-k... - Intel i7 Kaby Lake - No decided on motherboard. The one that cause me less trouble (for hackintosh) is fine. - GTI 750ti (have) or buy a pascal nvidia. - NVMe drive if possible - 32 GB RAM. - Probably a Thermaltake CORE P3 case. Not decided. I was thinking in use a Liquid Cooler but wonder if the Noctua could be better/less noise?
rspeer 10y ago
> Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running. Deep learning toolkits basically have two modes of operation: the CPU way, and the proprietary NVIDIA way. There is no point to putting an AMD chip in a machine you hope to use for deep learning. Some toolkits may go through the motions of supporting AMD via OpenCL, but that's not going to be the case that they make sure works well, or works at all. This is a problem, as NVIDIA is awful at maintaining drivers, results are being published based on "well, my NVIDIA black box decided to do this", and it contributes to deep learning veering toward a local maximum. But specifically choosing to do deep learning with an AMD chip is a pointless sacrifice.

gcp 10y ago

Contrary to the claims there, the CPU does matter: if the network is relatively small, it's easy to get bottle-necked feeding the GPU. Same thing applies to storage, for the same reasons. There's a reason AMD is selling deep learning cards with Flash drives built in. The choice of CPU cooler is silly. The CPU chosen is a 65W model, which will be cooled quietly even by the boxed cooler (and we don't need to care about overclocking with a non-K). By his own admission, the rig draws 250W, so the idea that 650W must be cooled is nonsense, and even then, only a part of that is due to the CPU. It's probably worth looking more closely at the GPU, notably, if that cooler stays quiet when the card is under full load. You do not need a monitor, keyboard or mouse to use such a machine. You can just ssh into it. You might need to borrow a set to set up the BIOS. If you "run out of memory for your applications" on the GPU, have you considered simply lowering the mini-batch size? (See, this is why we got a good CPU, it removes the hurt from feeding smaller batches)

nl 10y ago
There's a reason AMD is selling deep learning cards with Flash drives built in. For those not familiar with them, this is the Radeon Pro SSG[1] This has a built in M2 drive, but this is used as video card memory, NOT main storage. It could possibly allow big (HUGE) models or batches, but doesn't prove fast IO is often a bottleneck. Practically no one[2] is using these for deep learning. They are marketed to the oil and gas modelling and visualization market and notably the product page (linked above) makes no mention of neural networks or deep learning, and it has never been mentioned in AMD's publicity. Yes, it might be good for deep learning. And yet, modern deep models do want more memory. But there is a lot that needs to happen for this to be useful: AMD needs to release something like CuDNN and they need to make sure OpenCL is supported as well as CUDA is. [1] https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-ssg-graphics-card-specs-price-release-date/ https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-ssg-g..., http://www.amd.com/en-us/press-releases/Pages/amd-radeon-pro-2016jul25.aspx http://www.amd.com/en-us/press-releases/Pages/amd-radeon-pro... [2] I'm sure you'll find someone. But show me a published paper or any kind, or even benchmarks showing the use of the extra memory somewhere.
- gcp 10y ago

mrmondo 10y ago

My home server has an i7 4790K, 32GB DDR3 and... a GTX 1080 Gamer Edition that's doing... well, bugger all. Why is the graphics card in there you might ask? Well, it was sitting on my desk collecting dust as I wait patiently for a Pascal driver for eGPUs on macOS but Nvidia had delayed these for long enough that I thought I'd chuck it in my home server, it is being used by ffmpeg for x265/HVEC transcoding with Plex media server but I'm really not even touching it's capabilities - not even close when transcoding a 120GB+ 4K 120FPS video and streaming it wirelessly to my TV that's running a Plex client. The server is running CentOS 7 and has 10x 8TB SATA drives in BTRFS RAID 10 for storage, two Intel NVMe 750 series 1.2 TB SSDs for the OS as well as a couple of 2TB Crucial MX300 SATA SSDs that aren't currently in use for much. So, I have been sitting here wondering how I can make better use of it while it's practically parked in my server while I wait for some Pascal drivers to drop. I'm not so interested in robots / live video processing but it'd be neat if I could leverage its power for something fun or to assist the servers compute performance in some other ways that might be useful or interesting. I was running KVM with PCIe pass through passing the GPU to a guest VM running Steam as a steam streaming box but I got sick of having an OS on the network that I had to worry about Virus's, Malware and annoyingly slow and unreliable updates being installed when you least want it to happen and I got lazy and went back to playing PlayStation when I needed some time out from constructive work / research. Open to ideas to any software I could tinker with if anyone has any interesting suggestions?

kharms 10y ago
Not something for you to work on, but something you can do in the mean time: https://en.m.wikipedia.org/wiki/List_of_distributed_computing_projects https://en.m.wikipedia.org/wiki/List_of_distributed_computin...
j_s 10y ago
Can you ballpark today's price tag for such a setup? There seem to be several pricing tiers floating around in this discussion and I'd appreciate the chance to see how your setup fits in without having to shop for it.
- mrmondo 10y ago

singularity2001 10y ago

In Europe it's also very common to just choose the components in a web shop and have them assembled, also for about $1000. (Not looking at Dell) Anyone interested in deep learning should go that path instead of burning their money on Amazon.

ProxCoques 10y ago
+1 on that. There are a number of online build-to-order shops here in the UK which offer a truly vast range of component combinations. Better still, the order process automatically checks for compatibility with, for example, your choice of motherboard. Once you have the configuration you want, you just pay up and it arrives professionally assembled ready to rock. I don't think I'll ever by an off-the-shelf machine again.
- IanCal 10y ago
  I found the premium on these was quite high, although even just buying all the components from a single place added a fair amount to my recent build. Compatibility, price checking and searching are all available with PcPartPicker (UK site: https://uk.pcpartpicker.com/ https://uk.pcpartpicker.com/) which saved me an astonishing amount of time. The hour or two building the machine were worth the few hundred it saved me, but everyone has different priorities.
  - Coding_Cat 10y ago
    I've found that the premium depends on the store and model. Some seem to do relative pricing, others absolute. In the latter case, if you want a big rig it's best to let someone build it for you. Often times in those cases you also get great service because a) if you build PCs for a living you're a computer geek and it's fun to build an insane PC and b) they often use their biggest systems as advertisement. At least, that's what I've seen. +cable management is like black magic to me. If I were to take the money saved as payment for me to get it as nice as those places get it, I'd be below minimum wage.
jbmorgado 10y ago

amelius 10y ago

But how do you scale cheaply to more GPUs?

mtanski 10y ago
Get a ATX instead of ITX mobo. It's likely is cheaper as well.

PascLeRasc 10y ago

I'm not sure why mITX was chosen - in my experience it's almost always more expensive and leaves no room for additional GPU expansion (Or network adapters/whatever card you might need) in the future. You can get 1151 mATX motherboards for more like $60-70 on Amazon instead of the $125 they paid. Also, paying $125 for 16GB of ram but not spending the extra $20 to have your CPU be able to overclock? I'm not sure where OP is building but I can find that much RAM for $20 less than they paid, and overclocking isn't really that difficult or unreliable these days.

coroxout 10y ago
I was wondering that too. For something like this maximum room for future expansion seems good. In the past I've used mITX to build low-power, low-noise machines, rather than building a high-powered processing beast. On the other hand, it made the article more interesting for me. I can't really justify a home machine learning rig, but I was thinking I might soon replace that old mITX machine. (edit - just reached the pics - really not sure why you'd go with a mini form factor and then add a 6" heatsink/fan!)
- PascLeRasc 10y ago
  I also prefer mITX and I have one I've tried to optimize for low noise and efficiency. Just seems ill-suited for this purpose.
pdog 10y ago
Most people don't want a giant box under their desk. If you're building a machine learning rig for under $1,000, you don't need room for expansion because additional $200, $400 or even $700 cards are a useless expense at that price range.
gens 10y ago
You don't need a powerful cpu for gpu loads.

brudgers 10y ago

If I were building a GPU box on the cheap, I'd start with a used Dell Precision T7xxx series off of Ebay for a <$300 including RAM and a Xeon or two. They have 1100 watt power supplies and lots of slots and the RAM is usually ECC. Odds are it will come with DVD burner and possibly a hard disk and a Windows license...again all for a couple of hundred dollars. Leaving $700 for GPU's while providing reliable high end hardware pretty much designed to run GPU's for the base platform.

throwawayish 10y ago

If you're on a budget it makes little sense if any to buy things like a motherboard, CPU or memory new.

dsacco 10y ago

I think there should be a disclaimer here that while, yes, this will work, it will quickly become suboptimal (perhaps even dysfunctional) for a lot of work as you scale your hobby into something resembling more professional work. I personally would not start with a thousand dollar machine if you have any intention of doing serious computation in the future. While you can trade out parts pretty easy on a custom built machine, you don't want to be in a position where you have to trade out most of the core parts 6 months in. For reference, I built a home PC that I successfully do deep learning and data analysis on (mostly tensorflow and scipy stack) for about ~$10k. It's liquid cooled, has 15 fans, four radiators, an i7-6900K CPU, 128GB RAM, four GTX 1080 GPUs (controversial), four TBs of HDD space and 1TB of SSD space. I don't recommend you start with this at all, but my point is that porting your hardware from point A to point B will be a pain if it comes to it. I used the guide here as a reference about 8 months ago when I built it: http://graphific.github.io/posts/building-a-deep-learning-dream-machine/ http://graphific.github.io/posts/building-a-deep-learning-dr.... My purpose in doing this was, essentially, to pay for electricity rather than AWS/GCP/Azure compute resources (and in that regard it's been very successful!). I know I'm hijacking a thread here to talk about building home machines for professional deep learning work when this story is clearly not intended for that, but I wanted to throw in this perspective so that it's understood this is very different from just "build this machine to start out and upgrade it later." There's a law of diminishing returns here, but in general my point is that I do not think this is a minimum for "start doing deep learning effectively at home." If you want to learn hands on deep learning cheaply, my opinion is that it would be more efficient to use compute resources from a cloud provider before diving into this with a home-based custom machine. tl;dr: The demographic of folks who probably want/should/need to build a home deep learning machine probably has little overlap with the demographic of folks who want to do it non-professionally, or at least with only $1k in resources.

dbancajas 10y ago
so do you earn money out of that 10K investment on machines? I didn't read the article but what's the benchmark on deep learning machines? Still TFLOPS?
- 10y ago

kordless 10y ago

I've been considering a "thing" that allows monetization of an AI-enabled machine by others who need the capacity to do larger scale testing. Anyone interested in this? My contact info is in my profile.

minimaxir 10y ago
You won't be able to match price/expenses (fixed with hardware or variable with upkeep) of AWS/GCP due to their economy of scale. Even startups who are doing similar things sublicense from AWS.
PascLeRasc 10y ago
How is it better than AWS?