5 ms·
Efficient Recurrent Neural Networks using Structured Matrices in FPGAs
- cs702 9y agoThe main idea is to take each weight matrix W, divide it into smaller blocks, each of dimension, say, n×n, and make each block a circulant matrix that can be specified by a vector of only n elements.[a] This reduces W's memory consumption by a factor of n, and makes other gains in computational efficiency possible. Read the paper for details. However, as far as I can tell, it appears the authors have tested this technique only with one RNN architecture, in one task. It's hard to know whether the technique will hold up well in a broad range of RNN architectures/tasks. [a] https://en.wikipedia.org/wiki/Circulant_matrix https://en.wikipedia.org/wiki/Circulant_matrix
- godelmachine 9y agoSpeech recognition be the task they implemented this for.
- lnsru 9y agoWhat about image recognition? Too complex? Or doable at 320x240 resolution?
- lnsru 9y agoHow does this compare to GPU? These FPGAs are high end ones and not cheap for sure.
- godelmachine 9y agoRegarding GPU, they have only said - "The ESE achieves higher energy efficiency than GPU, but its performance is lower and it cannot operate in real time" So maybe we can assume the performance is quite fast - as compared to the GPU they used in ESE, and this work can operate in real time?
- dr_zoidberg 9y agoTable 2 has latencys (measured in microseconds) and a frames-per-second measure too (which places it a ~250k-400k fps). According to them, ESE has a 17k fps "theoretical max", but in reality has a lower performance. I don't really get the fps measure -- without context, I can't say above which threshold they'd be able to perform real-time operation (this is surely due to my lack of knowledge in this domain).
- alain94040 9y agoAccording to [1], a high-end FPGA can do "AlexNet inference performance: int16 over 2,400 img/s, int8 over 4,500 img/s.". Plug in your favorite nVidia numbers and compare. [1] http://mipsology.com/zebra.html http://mipsology.com/zebra.html
- lnsru 9y agoThank you very much for the link!
- godelmachine 9y agoHi Insru - how do you think GPUs would fare taking into consideration alain94040 's metrics?
- lnsru 9y agoPlease look at this pdf: https://www.nvidia.com/content/tegra/embedded-systems/pdf/jetson_tx1_whitepaper.pdf https://www.nvidia.com/content/tegra/embedded-systems/pdf/je... There is a table on page 9 and it claims, that Titan X can make 3216 img/s while TUL-KU115 FPGA board can make only >1000 img/s. This KU115 chip has a project price of ~$2000. Real time capability is nice, but that’s still expensive.
- godelmachine 9y agoThanks for this PDF :) Do you think TitanX has real time capability?
- lnsru 9y agoI am FPGA developer and have not much knowledge about GPUs. I just can make a guess, that no one can guarantee execution time on GPU. There are all little details well defined in FPGA design and result is always available after defined delay.
- RobertDeNiro 9y agoFrom the original paper: ESE outperforms Core i7 CPU and Pascal Titan X GPU by factors of 43× and 3× on speed, and it is 40× and 11.5× more energy efficient than the CPU and GPU respectively.