4 ms·
Can someone explain the "intersection" of neural networks and FPGAs/GPUs. I feel like I'm seeing this referenced a fair amount but haven't heard a good explanat
by bogomipz 11y ago
Can someone explain the "intersection" of neural networks and FPGAs/GPUs. I feel like I'm seeing this referenced a fair amount but haven't heard a good explanation on why this hardware is attractive for these workload.
- danielvf 11y agoBecause neural networks use "simple" math that is massively parallelizable. Your computer's CPU may have eight cores, but my older, mid range graphics card has over a thousand cores. If your work can be broken down into a thousand peices that can be computed independently, than using a GPU can get you ludicrous speedups.
- lucaspiller 11y agoThis is a bit of an over simplification, because the 'cores' of a GPU aren't really comparable to a CPU core. It's more like scaling up a factory - at a single point on the production like you go from 1 person doing a job to 100 people doing the same job with the same tools. It means you can do more of the same thing, but can't do 100 completely different things. Here is a good overview: http://gamedev.stackexchange.com/a/17255 http://gamedev.stackexchange.com/a/17255
- bogomipz 11y agoThanks
- deleted 11y ago[deleted]
- SixSigma 11y agoI think it is something people just like to say because both sound a big magical. Neural nets are matrix multiplications and addition. They are generally memory constrained on GPUs and Main <> GPU memory is also a bottleneck. I don't see how an FPGA will really help there.
- mtgx 11y agoI can't find the link right now, but I remember reading a more recent article on NextPlatform.com about how FPGAs are much more memory constrained than GPUs are, which essentially negates any slight power efficiency advantage they might have over GPUs.
- stillworks 11y agoAlso, don't FPGAs bring along additional operational constraints ? Modern large/vast FPGAs will take anywhere from 2 to 4 hours to be Pd in the F. This makes managing new releases even more challenging.
- bogomipz 11y ago"Pd in the F"? What does that mean?
- stillworks 11y agoFPGA = Field Programmable Gate Arrays, Pd in the F => Programmed in the Field