5 ms·

Oh man, if you like that paper do I have the list for you. We used this list while doing some literature review when we were doing research on HPs memristors ht

Oh man, if you like that paper do I have the list for you. We used this list while doing some literature review when we were doing research on HPs memristors http://webee.technion.ac.il/people/skva/new_full_reference_list.htm http://webee.technion.ac.il/people/skva/new_full_reference_l.... That covers a pretty wide range of topics related to memristors

deepnotderp 9y ago
Beautiful! I have a question for you, lots of people have been claiming memristors are right around the corner for almost half a century at this point, what do you think about the viability of the memristor in the future?
- smalley 9y ago
  In the enterprise storage market I think they're probably quite close to being a volume product. The versions I was working with were a few years back and admittedly a different chemistry than is used these days. The issue I think they'll have in the short run is that these will be quite expensive relative to NAND. I don't think you're going to have a memristor based disk in your home PC for quite a while. I do know that there are already some space systems that utilize memristors and reram already. The really novel applications (as in not just big hunks of fast memory) are probably much further out if they even go anywhere.
  - kovrik 9y ago
    So do memristors exist or not? Wiki claims they are still 'hypothetical'.
    - smalley 9y ago
      They 100% exist. We had a wafer worth of cutup die of the TiO_2-x type devices all the way back in 2007. HP officially published results of a physical device as well (http://www.hpl.hp.com/news/2008/apr-jun/memristor.html http://www.hpl.hp.com/news/2008/apr-jun/memristor.html)
      - aidenn0 9y ago
        My understanding of the TiO2-x devices is that they are strongly non-linear; which makes them very useful for binary storage, but less so for analogue memristor-y uses.
        elcritch 9y ago
        Wouldn't non-linear mem-resistors make pretty ideal neural nets? Most activation functions are modeled as non-linear functions anyway. Edit: Appears some HP associated labs had success with just such a chip! Surprised it hasn't been more success in that field. https://www.technologyreview.com/s/537211/a-better-way-to-build-brain-inspired-chips/ https://www.technologyreview.com/s/537211/a-better-way-to-bu...
        deepnotderp 9y ago
        Because no one wants neuormorphic hardware, we want 16-bit hardware that allows us to do math. Contrary to popular opinion, neural networks is more about math than biology. Having 1-bit or analog weights/activations is a royal pain.
        elcritch 9y ago
        That's a good point. Most of our current CS / Hardware tech and experience relies on digital computing. Makes me wonder if instead of using neuormorphic hardware as the deployment vehicle, it'd make sense to use it as a specialized high speed trainer. Whereas most of these research projects seem to be assuming that they'd be deployed as endpoint/client devices. The alternative would be to use the annoying/tedious analog hardware to significantly (possibly) reduce the amount of time/energy costs to train a neural net. Then have special equipment to measure the learned weights and convert them to 16-bit weights that'd be easier to deploy. Facebook/Google are showing that custom chips for deep learning make economic sense. That market feature could tip the scales for specialized neuormorphic training chips which deal with analog circuitry but offset those costs & complexities by driving efficiency and/or speed while still deploying via traditional digital chips. Of course, this path relies on the assumption that an analog circuit could be much faster or energy efficient. It's not too far fetched though based on the current numbers and time for training vs biological system energy costs. [edit: grammar]
    - WaxProlix 9y ago
      My understanding is that the current 'memristor' technologies are essentially combinations of materials which satisfy the linkage between flux and charge, thereby meeting the definition. From the same wikipedia article: > There is no such thing as a standard memristor. Instead, each device implements a particular function, wherein the integral of voltage determines the integral of current, and vice versa. A linear time-invariant memristor, with a constant value for M, is simply a conventional resistor.[1] Manufactured devices are never purely memristors (ideal memristor), but also exhibit some capacitance and resistance. Just reading the 'Background' and following 'Memristor definition and criticism' sections gives a super clear and lucid answer that even a goober like me can mostly degest. https://en.wikipedia.org/wiki/Memristor#Background https://en.wikipedia.org/wiki/Memristor#Background
    - igk 9y ago
      Basically, in the science world, there is a lot of debate going on about exact definitions (memristive device vs. memristor). I can see if I can find the relevant papers if there is interest. Prof. Chua has updated and loosened his criteria a bit in response to the criticisms. In the engineering world, if it talks like a memristor, and walks like a memristor if you squint a bit, we are damn well going to use it like a memristor. Hence we are happily building and testing nonvolatile storage (MRAM, ReRAM, PCRAM) and other applications - I'm gonna start research on applications as ML accelerators in May :-)
    - deepnotderp 9y ago
      They absolutely exist. The question is whether they can be reliably fabricated in volume without having loads of faulty units and hopefully, to not be ridiculously expensive.
dayve 9y ago
Thank you!

deepnotderp 9y ago

Beautiful! I have a question for you, lots of people have been claiming memristors are right around the corner for almost half a century at this point, what do you think about the viability of the memristor in the future?

smalley 9y ago
In the enterprise storage market I think they're probably quite close to being a volume product. The versions I was working with were a few years back and admittedly a different chemistry than is used these days. The issue I think they'll have in the short run is that these will be quite expensive relative to NAND. I don't think you're going to have a memristor based disk in your home PC for quite a while. I do know that there are already some space systems that utilize memristors and reram already. The really novel applications (as in not just big hunks of fast memory) are probably much further out if they even go anywhere.
- kovrik 9y ago
  So do memristors exist or not? Wiki claims they are still 'hypothetical'.
  - smalley 9y ago
    They 100% exist. We had a wafer worth of cutup die of the TiO_2-x type devices all the way back in 2007. HP officially published results of a physical device as well (http://www.hpl.hp.com/news/2008/apr-jun/memristor.html http://www.hpl.hp.com/news/2008/apr-jun/memristor.html)
    - aidenn0 9y ago
      My understanding of the TiO2-x devices is that they are strongly non-linear; which makes them very useful for binary storage, but less so for analogue memristor-y uses.
      - elcritch 9y ago
        Wouldn't non-linear mem-resistors make pretty ideal neural nets? Most activation functions are modeled as non-linear functions anyway. Edit: Appears some HP associated labs had success with just such a chip! Surprised it hasn't been more success in that field. https://www.technologyreview.com/s/537211/a-better-way-to-build-brain-inspired-chips/ https://www.technologyreview.com/s/537211/a-better-way-to-bu...
        deepnotderp 9y ago
        Because no one wants neuormorphic hardware, we want 16-bit hardware that allows us to do math. Contrary to popular opinion, neural networks is more about math than biology. Having 1-bit or analog weights/activations is a royal pain.
        elcritch 9y ago
        That's a good point. Most of our current CS / Hardware tech and experience relies on digital computing. Makes me wonder if instead of using neuormorphic hardware as the deployment vehicle, it'd make sense to use it as a specialized high speed trainer. Whereas most of these research projects seem to be assuming that they'd be deployed as endpoint/client devices. The alternative would be to use the annoying/tedious analog hardware to significantly (possibly) reduce the amount of time/energy costs to train a neural net. Then have special equipment to measure the learned weights and convert them to 16-bit weights that'd be easier to deploy. Facebook/Google are showing that custom chips for deep learning make economic sense. That market feature could tip the scales for specialized neuormorphic training chips which deal with analog circuitry but offset those costs & complexities by driving efficiency and/or speed while still deploying via traditional digital chips. Of course, this path relies on the assumption that an analog circuit could be much faster or energy efficient. It's not too far fetched though based on the current numbers and time for training vs biological system energy costs. [edit: grammar]
  - WaxProlix 9y ago
    My understanding is that the current 'memristor' technologies are essentially combinations of materials which satisfy the linkage between flux and charge, thereby meeting the definition. From the same wikipedia article: > There is no such thing as a standard memristor. Instead, each device implements a particular function, wherein the integral of voltage determines the integral of current, and vice versa. A linear time-invariant memristor, with a constant value for M, is simply a conventional resistor.[1] Manufactured devices are never purely memristors (ideal memristor), but also exhibit some capacitance and resistance. Just reading the 'Background' and following 'Memristor definition and criticism' sections gives a super clear and lucid answer that even a goober like me can mostly degest. https://en.wikipedia.org/wiki/Memristor#Background https://en.wikipedia.org/wiki/Memristor#Background
  - igk 9y ago
    Basically, in the science world, there is a lot of debate going on about exact definitions (memristive device vs. memristor). I can see if I can find the relevant papers if there is interest. Prof. Chua has updated and loosened his criteria a bit in response to the criticisms. In the engineering world, if it talks like a memristor, and walks like a memristor if you squint a bit, we are damn well going to use it like a memristor. Hence we are happily building and testing nonvolatile storage (MRAM, ReRAM, PCRAM) and other applications - I'm gonna start research on applications as ML accelerators in May :-)
  - deepnotderp 9y ago
    They absolutely exist. The question is whether they can be reliably fabricated in volume without having loads of faulty units and hopefully, to not be ridiculously expensive.

dayve 9y ago

Thank you!