6 ms·

Recurrent Neural Networks Hardware Implementation on FPGA

54 points by Katydid 11y ago

dang 11y ago
Url changed from http://hgpu.org/?p=14968 http://hgpu.org/?p=14968, which points to this.
iaw 11y ago
Is this unexpected? I feel like we're heading towards more hardware implementations of Machine Learning to work around the current bottlenecks.
- spooningtamarin 11y ago
  There are extremely large amounts of hardware based implementations. Any more complicated TV upscaler (Sony 4K TVs certainly) probably has a neural network embedded in it. Any algorithms working with images need FPGA implementations to be quick enough, and there are a lot of convnets in use there. There's a chapter on convnets here: http://www.cambridge.org/us/academic/subjects/computer-science/pattern-recognition-and-machine-learning/scaling-machine-learning-parallel-and-distributed-approaches?format=HB http://www.cambridge.org/us/academic/subjects/computer-scien... RNNs are nothing special in particular, at least these with small number of layers and nodes. HMMs or CRFs are easier to handle for sequences, and would probably work well, and there are FPGAs all over the place of these models.
  - iaw 11y ago
    Wow, I was completely unaware that consumer products like TV upscalars were utilizing neural networks. I assume that the upscalar uses the result of a prior training session and doesn't need periodic retraining.
    - spooningtamarin 11y ago
      Yeah, I was surprised too when I heard that, but if you had a huge database of HD movies, you could train a NN yourself, see how it upscales, use some algorithms or different error functions to correct weird artifacts and put the trained weights on an FPGA.
  - p1esk 11y ago
    Any links to TV upscaler chips using NN algorithms? I couldn't find any. Also, my impression was that FPGAs are much slower than GPUs for neural nets. Unless you're talking about really high end chips like Stratix 10 from Altera, which cost over $30k. Power consumption is a different matter though.
    - lightcatcher 11y ago
      One of my favorite named startups ever is working on something likely related to video upscaling: http://www.magicpony.technology/ http://www.magicpony.technology/
      - p1esk 11y ago
        That page says they are a software company. Are they working on an upscaler chip?
    - spooningtamarin 11y ago
      https://community.sony.co.uk/t5/blog-news-from-sony/inside-4k-what-is-x-reality-pro/ba-p/1346224 https://community.sony.co.uk/t5/blog-news-from-sony/inside-4... They won't tell you it's NNs but it is. Sony distributes a lot of movies, they use their database of movies to train the upscaling models (which is obviously NNs https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x ) and then put the chip in the TV. It's something almost equivalent to storing Pride and Prejudice and Zombies in your TV in 4K, and then reproducing it when they match it with what's playing on TV.
      - p1esk 11y ago
        It does not mention anything about neural networks in that link. How do you know? I looked up that Sony chip, and I can't find anything "neural" about it either. I don't quite understand what you mean by: "It's something almost equivalent to storing Pride and Prejudice and Zombies in your TV in 4K, and then reproducing it when they match it with what's playing on TV." Can you explain? What exactly do they have to store in the TV?
ilurk 11y ago
> We implemented a RNN with 2 layers and 128 hidden units in hardware and it has been tested using a character level language model. The implementation is more than 21× faster than the ARM CPU embedded on the Zynq 7020 FPGA. I'm left curious on the performance gain factor when scaling the network in terms of layers and units. Would the performance gap widen as the RNN grows?
- leonardt 11y ago
  > Figure 8: The execution time is projected to decrease with the increase of number of LSTM cells running in parallel. This can lead to significant performance improvement. They do say in the text that "Figure 8 shows the expected speed up, assuming the data throughput is high enough to handle the parallel processing" so take it with a grain of salt. There could and most likely will be (as there always is) other factors that prevent ideal scaling.
nuand 11y ago
Seems like an interesting approach. However the 21x speedup seems a little underwhelming considering the speed of the Zynq's programmble logic fabric and how parallizable NNs are. They're quoting a 21x speedup over the ARM processor on the Zynq 7020, which is on par with what powers the RaspberryPi 2. My guess is they didn't pipeline their design enough or appropriately causing one of the datapaths to significantly limit their throughput.

54 points by Katydid 11y ago

dang 11y ago

Url changed from http://hgpu.org/?p=14968 http://hgpu.org/?p=14968, which points to this.

iaw 11y ago

Is this unexpected? I feel like we're heading towards more hardware implementations of Machine Learning to work around the current bottlenecks.

spooningtamarin 11y ago
There are extremely large amounts of hardware based implementations. Any more complicated TV upscaler (Sony 4K TVs certainly) probably has a neural network embedded in it. Any algorithms working with images need FPGA implementations to be quick enough, and there are a lot of convnets in use there. There's a chapter on convnets here: http://www.cambridge.org/us/academic/subjects/computer-science/pattern-recognition-and-machine-learning/scaling-machine-learning-parallel-and-distributed-approaches?format=HB http://www.cambridge.org/us/academic/subjects/computer-scien... RNNs are nothing special in particular, at least these with small number of layers and nodes. HMMs or CRFs are easier to handle for sequences, and would probably work well, and there are FPGAs all over the place of these models.
- iaw 11y ago
  Wow, I was completely unaware that consumer products like TV upscalars were utilizing neural networks. I assume that the upscalar uses the result of a prior training session and doesn't need periodic retraining.
  - spooningtamarin 11y ago
    Yeah, I was surprised too when I heard that, but if you had a huge database of HD movies, you could train a NN yourself, see how it upscales, use some algorithms or different error functions to correct weird artifacts and put the trained weights on an FPGA.
- p1esk 11y ago
  Any links to TV upscaler chips using NN algorithms? I couldn't find any. Also, my impression was that FPGAs are much slower than GPUs for neural nets. Unless you're talking about really high end chips like Stratix 10 from Altera, which cost over $30k. Power consumption is a different matter though.
  - lightcatcher 11y ago
    One of my favorite named startups ever is working on something likely related to video upscaling: http://www.magicpony.technology/ http://www.magicpony.technology/
    - p1esk 11y ago
      That page says they are a software company. Are they working on an upscaler chip?
  - spooningtamarin 11y ago
    https://community.sony.co.uk/t5/blog-news-from-sony/inside-4k-what-is-x-reality-pro/ba-p/1346224 https://community.sony.co.uk/t5/blog-news-from-sony/inside-4... They won't tell you it's NNs but it is. Sony distributes a lot of movies, they use their database of movies to train the upscaling models (which is obviously NNs https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x ) and then put the chip in the TV. It's something almost equivalent to storing Pride and Prejudice and Zombies in your TV in 4K, and then reproducing it when they match it with what's playing on TV.
    - p1esk 11y ago
      It does not mention anything about neural networks in that link. How do you know? I looked up that Sony chip, and I can't find anything "neural" about it either. I don't quite understand what you mean by: "It's something almost equivalent to storing Pride and Prejudice and Zombies in your TV in 4K, and then reproducing it when they match it with what's playing on TV." Can you explain? What exactly do they have to store in the TV?

ilurk 11y ago

> We implemented a RNN with 2 layers and 128 hidden units in hardware and it has been tested using a character level language model. The implementation is more than 21× faster than the ARM CPU embedded on the Zynq 7020 FPGA. I'm left curious on the performance gain factor when scaling the network in terms of layers and units. Would the performance gap widen as the RNN grows?

leonardt 11y ago
> Figure 8: The execution time is projected to decrease with the increase of number of LSTM cells running in parallel. This can lead to significant performance improvement. They do say in the text that "Figure 8 shows the expected speed up, assuming the data throughput is high enough to handle the parallel processing" so take it with a grain of salt. There could and most likely will be (as there always is) other factors that prevent ideal scaling.

nuand 11y ago

Seems like an interesting approach. However the 21x speedup seems a little underwhelming considering the speed of the Zynq's programmble logic fabric and how parallizable NNs are. They're quoting a 21x speedup over the ARM processor on the Zynq 7020, which is on par with what powers the RaspberryPi 2. My guess is they didn't pipeline their design enough or appropriately causing one of the datapaths to significantly limit their throughput.