9 ms·
Facebook to open-source AI hardware design
- sawwit 11y agoHow come they aim for GPU technology rather than for neuromorphic computing ASICs?
- kitcar 11y agoJust a guess, but they project the cost of large scale GPU manufacturing to decline faster than ASIC due to the shared applications of gaming/VR?
- blackguardx 11y agoYou would think AI would be bigger than VR. Edit: going with GPUs makes sense now, but it doesn't indicate a big bet on Facebook's part. I don't think opensourcing a server cabinet is that interesting, but I'm a hardware engineer. In 10 years I think more people will touch AI than VR but I don't know how that translates to investment.
- unchocked 11y agoGaming is a proven market. Capital hates risk. (Real capital, not the fringe type we talk about here.)
- argonaut 11y agoAI might be bigger than VR, but the timeframes are messed up. VR is an engineering problem now; I don't think there are major field-shattering breakthroughs needed to make reasonably realistic, cheap VR within the next 5 years. AI is nowhere close to strong AI or reasonable general-purpose intelligent programs that don't require loads of human tuning. It will take several breakthroughs.
- vonnik 11y agoWhich neuromorphic ASICs are you thinking of? GPUs are fast and they work now. Some "neuromorphic" chips, like IBM's True North, are just low-energy GPUs, so neuromorphic is in the eyes of the beholder...
- emcq 11y agoThis is not very accurate. The true North chip places memory next to compute elements on chip. GPUs have most memory off chip. The true North hardware is specialized to neural net stuff, and doesn't allow general floating point compute, have any sort of instruction set decoder, etc. That turns into big performance and power wins.
- modeless 11y agoContrary to what you might read in puff pieces from IBM PR, all existing "neuromorphic" hardware is terrible. The neural nets that work don't look anything like the brain, and neural nets that look like the brain don't work. People constructing neural nets that look like the brain are engaged in cargo cult science, because the truth is we have no idea yet how the brain works and attempting to imitate it without knowing that is doomed to failure.
- dangirsh 11y agoI'm curious to hear your opinions on Numenta's work on emulating parts of the neocortex.
- howlin 11y agoHe just gave it to you. Their system is not really competitive in any metric people are measuring.
- chimtim 11y agoA good measure of any of the techniques are the results. We see record object recognition and speech recognition results. How many of these results are from Numenta or IBM neomorphic chips? Most successes have been on deep learning architectures over GPUs (and large datasets). While these fancy architectures may have their applications, they have nothing to do with recent advancements in last 3 years.
- emcq 11y agoTo be fair, the goal of those new computing architectures is not to advance algorithms. That at best is a side benefit. Those algorithmic performance improvements seen in DNN have come from improved datasets and training. Numenta and IBM arent focusing on training AFAIK. Google's Quantum Annealing [0] is the only hardware I'm aware of focused on training, although there are rumors Nervana Systems may produce something [1]. I'm sure there are others; accelerating training of DNN isnt a particularly new idea. The goal of these other computing architectures is typically to provide lower power, higher frequency/lower latency, or smaller form factor execution of trained models, but there is a question of how much value they can provide over more conventional chips to be worth the chip design costs. However without these architectures becoming as mainstream as say a GPU, I think we will continue to see advances come from the typical everyday computer. The ML community seems to be much more democratic than others. [0] http://googleresearch.blogspot.com/2015/12/when-can-quantum-annealing-win.html http://googleresearch.blogspot.com/2015/12/when-can-quantum-... [1] http://www.nervanasys.com/about/ http://www.nervanasys.com/about/
- nickpsecurity 11y agoBasically, you need hardware designers and a ton of money to develop ASIC's that will accelerate as good as current GPU's. I've seen designs along lines of SIMD's, DSP's, etc that might be a nice alternative in custom hardware. Still tons of money to get them to work. Doesn't even count developing compilers and such to utilize them properly. Existing tools on GPU's are easier despite me wanting more work on FPGA's and ASIC's in this area. ;)
- michael_h 11y agoI was heavily into neuromorphic hardware ~5 years ago when the hype seemed to be at its peak. We funneled a load of time and money into it. It was working okay for what we were doing and everybody was generally happy. Then the NVidia Tesla C2050 came out and CUDA reached a somewhat stable release. The neuromorphic plane has crashed into the proverbial mountain. EDIT: Maybe I should elaborate - we accomplished the same task using the Tesla card, but it was about 100x faster and each card was $2500 and usable for other tasks. The general rule of thumb became: design a neuromorphic system, wait for the next Tesla chip, then simulate it in software.
- dang 11y agoMore at https://news.ycombinator.com/item?id=10712195 https://news.ycombinator.com/item?id=10712195.
- vonnik 11y agoI'm glad they're open-sourcing this, but I have to say that making 8 GPUs work together is not that big of a deal. Companies like Cirrascale are making up to 16 GPUs scale linearly with a blade.
- glxc 11y agoso if it's not the best, then it's not worth it?
- spiantino 11y agoWe have distributed implementations that allow us to use more than 8 GPUs to train a single network. These machines have 8 cards each, but we aren't limited to a single machine (though having more GPUs connected via PCI than via Ethernet helps)
- mrlase 11y agoAny particular reason you don't have Infiniband on these for interconnects? Having this chassis with Infiniband and a local disk would be a dream if the manufacturing cost is right as we scale up in our local datacenter where I'm at.
- vonnik 11y agoThat's interesting. But from what I understand, the problem isn't getting many GPUs to train a single network -- the problem is to get them to scale linearly; i.e. to get each additional GPU to accelerate the process as much as the last. Throwing boxes at neural net training is easy, but people run into serious plateaus scaling GPUs.
- dgacmu 11y agoIt's irrelevant if you can't balance the cost, CPU, shared DRAM, and - possibly most importantly - PCIe and network connectivity in and out of it. A typical mid-range Intel Xeon has 40 PCIe lanes. A high-end GPU requires 16. You can do the math. Facebook has decided upon a balance that works for them and their workloads; it's quite likely that going to 16 GPUs in the chassis resulted in overall worse utilization because of PCIe bandwidth limits, or socket and QPI count, etc. What's a big deal is someone having gone and done a lot of work to calculate and experiment with finding a balanced design that's also cost-effective and easy to maintain.
- daveguy 11y agoCan we please not call open sourcing a build plan of off the shelf components, open-source "hardware design". I don't think backblaze called their case designs open source hardware. It just doesn't fit when we have open source circuit board designs. How about open-source build specification instead?
- nickpsecurity 11y agoI agree we should differentiate between the types of open-sourcing. Open-source hardware should either be the umbrella term or the specific one for open-source circuits. Open-source build is one phrase for this sort of thing. Maybe configuration would be another. Seems tricky given we usually call a collection of components on PCB's and such a computer, hardware design, server, or so on. Putting open-source in front of them doesn't make it clear how much is open-source. That might be what we need to resolve. Anybody have ideas?
- atYevP 11y agoYev from Backblaze here -> Call it whatever you want to call it, as long as it's not "late for dinner". Wakka wakka wakka!
- Throwaway10323 11y agoDo you just watch ycombiner, reddit, and other sites all the time or do you use something to crawl the sites to see whenever someone says "Backblaze"?
- atYevP 11y agoHi! No, I use a program for that, it's actually kinda cool, though I don't get "alerts" I just monitor it. It's mention.net. Highly recommend it if you have a lot of internet chatter you want to stay ahead of. Funny story, Ycombinator is one of the reasons I got it. Since there's no great way to keep track of threads, it was a way for me to catalog and follow up with threads in these posts.
- nickpsecurity 11y agoGood work FB on another great deliverable for open HW configurations and improved datacenter design! :)
- scott_s 11y agoBig Sur is twice as fast as our previous generation, which means we can train twice as fast and explore networks twice as large. Can a machine learning expert comment on this claim? There are many domains in computing where doubling your compute power does not halve your execution time, or allow you to double the size of your input. (For example, your algorithms are worse than O(n), or you have hardware communication bottlenecks.)
- willvarfar 11y agoThese are machine learning experts making the claim? Machine learning is usually embarrassingly parallel.
- scott_s 11y agoIf I was reviewing a paper with bits on machine learning (presumably written by machine learning experts), I would ask machine learning experts I know to verify the claims I was not familiar with. I do know that a lot of ML algorithms become, at the low-level, matrix operations. Matrix operations are typically not embarrassingly parallel. Now, I can imagine a bunch of independent matrix operations, but whether you'd call the entire problem "embarrassingly parallel" would depend on how coarse the independent bits are.
- hedgehog 11y agoIn many cases you can train multiple instances of a neural net simultaneously in increments, average the weights together, and repeat. Making the underlying compute more efficient is an interesting problem.
- spiantino 11y agoAuthor here - in this case we are talking mostly about ~halving the time of a single minibatch in training a variety of standard neural networks. That type of training tends to be very hard to distribute. The servers also have 8 gpus in a single host, which is an additional performance boost and isn't included in the 2x above.
- guelo 11y agoI was hoping it was the AI that was designing the hardware.
- deleted 11y ago[deleted]
- chimtim 11y agoIs this a machine with GPUs? You can order a similar looking design from acmemicro since last 4 years atleast?
- mryan 11y agoDo you think there might be a reason that Facebook invested time and money in to producing this design, rather than buying from acmemicro?