14 ms·

> Combining memory and processing resources in a single device has huge potential to increase the performance and efficiency of DNNs as well as others forms of

by femto 10y ago

> Combining memory and processing resources in a single device has huge potential to increase the performance and efficiency of DNNs as well as others forms of machine learning systems. Isn't that describing a Field Programmable Gate Array: memory embedded in a sea of logic cells? What's the best resource to get someone who knows FPGAs, a decent amount of maths and 1990s style neural nets up to speed on today's deep neural networks?

jacquesm 10y ago
http://course.fast.ai/ http://course.fast.ai/
- femto 10y ago
  Thanks for prompting me to do further research on this. I didn't realise from the initial posting, that this course is based on prerecorded videos and can be completed asynchronously, to fit in with life. The site say students "need access to a computer with an Nvidia GPU", but I reckon it could be an exceedingly interesting exercise to try and do the course with an FPGA in place of the GPU.
  - jacquesm 10y ago
    Alternatively you could use some GPU cloud instances but the end result of that is probably that it ends up costing you more than it would to buy a low-end GPU that you can complete the course with. It'd be very interesting to see how you do this with an FPGA, that would be quite the accomplishment, especially given the monstrous compute capability that GPUs give, and the communications between high level (say: python) and low level (a GPU or some other co-processor) have all been worked out and debugged for GPUs. If there is room for FPGA work in there it would probably be some specialized function. Regardless I'd love a write-up if you do this.
    - dnautics 10y ago
      If my understanding is correct, FPGAs aren't that fast, and while the really small ones are cheap, they don't fit that much. I synthesized a 32-bit multiplier on a spartan 6, and it took up about 1/5 of the footprint. I figure maybe you could fit 4 16-bit fused-multiply adds (which ought to be able to do machine learning) onto a spartan 6, which is not that great. What is really going to slaughter you for ML performance on FPGAs is memory bandwidth. If you have access to a really big FPGAs with lots of memory (on the order of $10-30k...) it might make a difference.
      - jacquesm 10y ago
        I'd see more applications on the evaluation than on the training side but even in training there might be a few situations where it could pay off, you'd still be stuck with a huge communications issue to solve.
posterboy 10y ago
> Isn't that describing a Field Programmable Gate Array hard to say, my wristwatch has processing and memory, too.
zebrafish 10y ago
http://neuromorphic.eecs.utk.edu/pdfs/2016-IJCNN-Schuman.pdf http://neuromorphic.eecs.utk.edu/pdfs/2016-IJCNN-Schuman.pdf These include SNN implementations on field-programmable gate arrays (FPGAs), one that is trained using a mixture of unsupervised/supervised learning [61] and another that is trained using a GA [62] An initial implementation of DANNA on field programmable gate arrays (FPGAs)

by femto 10y ago

jacquesm 10y ago

http://course.fast.ai/ http://course.fast.ai/

femto 10y ago
Thanks for prompting me to do further research on this. I didn't realise from the initial posting, that this course is based on prerecorded videos and can be completed asynchronously, to fit in with life. The site say students "need access to a computer with an Nvidia GPU", but I reckon it could be an exceedingly interesting exercise to try and do the course with an FPGA in place of the GPU.
- jacquesm 10y ago
  Alternatively you could use some GPU cloud instances but the end result of that is probably that it ends up costing you more than it would to buy a low-end GPU that you can complete the course with. It'd be very interesting to see how you do this with an FPGA, that would be quite the accomplishment, especially given the monstrous compute capability that GPUs give, and the communications between high level (say: python) and low level (a GPU or some other co-processor) have all been worked out and debugged for GPUs. If there is room for FPGA work in there it would probably be some specialized function. Regardless I'd love a write-up if you do this.
  - dnautics 10y ago
    If my understanding is correct, FPGAs aren't that fast, and while the really small ones are cheap, they don't fit that much. I synthesized a 32-bit multiplier on a spartan 6, and it took up about 1/5 of the footprint. I figure maybe you could fit 4 16-bit fused-multiply adds (which ought to be able to do machine learning) onto a spartan 6, which is not that great. What is really going to slaughter you for ML performance on FPGAs is memory bandwidth. If you have access to a really big FPGAs with lots of memory (on the order of $10-30k...) it might make a difference.
    - jacquesm 10y ago
      I'd see more applications on the evaluation than on the training side but even in training there might be a few situations where it could pay off, you'd still be stuck with a huge communications issue to solve.

posterboy 10y ago

> Isn't that describing a Field Programmable Gate Array hard to say, my wristwatch has processing and memory, too.

zebrafish 10y ago

http://neuromorphic.eecs.utk.edu/pdfs/2016-IJCNN-Schuman.pdf http://neuromorphic.eecs.utk.edu/pdfs/2016-IJCNN-Schuman.pdf These include SNN implementations on field-programmable gate arrays (FPGAs), one that is trained using a mixture of unsupervised/supervised learning [61] and another that is trained using a GA [62] An initial implementation of DANNA on field programmable gate arrays (FPGAs)