3 ms·
Correct me if I'm wrong, but that looks like it needs specialized hardware. I'm thinking about how FPGAs with now being available on AWS can be used for neural
by ktta 9y ago
Correct me if I'm wrong, but that looks like it needs specialized hardware. I'm thinking about how FPGAs with now being available on AWS can be used for neural networks.
They are also much better than GPUs for in-house deployment because they consume a lot less power which in turn reduces cooling power. They also don't get 'refreshed' every year which can be a good or bad thing because you don't have to upgrade so frequently and possibly lag being on performance compared to rest of the industry.
FPGAs have often been acknowledged as being a low power solution for the same precision but they're tremendously difficult to program. Also when you want FP operations on FPGA you're using up a LOT of chip space, because they were meant for GPUs.
So this kind of light weight neural nets will be much more suited for FPGAs. I've tried some basic nets but they are better for GPUs since you can fit larger nets without problem and RAM is another problem with FPGAs.
I'm thinking that these nets would be perfect since they take up much less space on the FPGA and consume a lot less RAM. (The lower RAM consumption is talked about here[1]) Also, the simple construction of the net with basic operations would help the programming difficulty.
[1]: https://youtu.be/Q17HwA5oY4w?t=1m35s https://youtu.be/Q17HwA5oY4w?t=1m35s
- regularfry 9y agoThe original implementation used custom hardware, but the original implementation was in 1984, so I think that's forgivable. I don't think the custom hardware is necessary, though. Fundamentally the concept is that you replace a thumping great matrix operation with a set of lookup tables in memory. FPGA's are fun, I'm just not sure you need them for this. RAM is plenty fast.