5 ms·

Yeah, what's a good intro on understanding all of this? I've got an EE/Chem/Math background.

• deepnotderp 10y ago
  For deep learning or chips for deep learning? With an EE/Chem/Math background you should be set to go :) For deep learning, I highly recommend the cs231n course materials (available for free online) and the Deep Learning Book by Goodfellow et al. For chip design for deep learning specifically, it's a fairly new field, and a lot of commercial interests, hence why a lot of it isn't available in the form of an "Intro to Deep Learning Chip Design" course. But, the basic point is that deep learning can both train and perform inference in astonishingly low precision. For training: https://arxiv.org/abs/1502.02551 https://arxiv.org/abs/1502.02551 For inference, there are so many papers confirming this fact that there really isn't one key paper to point to. This article is a good introduction however: https://petewarden.com/2015/05/23/why-are-eight-bits-enough-for-deep-neural-networks/ https://petewarden.com/2015/05/23/why-are-eight-bits-enough-... Other than that, stripping out the cache hierarchy, HBM memory, etc. are the obvious steps to take in targeting deep learning. Note that it's my opinion that deep learning chip startups (such as ours) need another "secret sauce" beyond simply lowering precision, we're not like Intel which can just say "okay, here's $100mil, make me an 8-bit GPU, Go."
  • lowglow 10y ago
    Thanks! Mind if I connect with you over email?
    • deepnotderp 10y ago
      Sure, my personal email is sixsamuraisoldier [at] g(mail) (dot) [com]