19 ms·
Memristor Breakthrough: First Single Device to Act Like a Neuron
- robinduckett 6y agoAt 800 degrees C
- api 6y agoCan we make AI computers for Venus surface use now?
- mocmoc 6y agoOf course
- elsonrodriguez 6y agoProtogen wants to know your location.
- Tade0 6y agoIt's actually still too cold there for them to work.
- astrea 6y agoVenus seems sufficient: https://solarsystem.nasa.gov/resources/681/solar-system-temperatures/ https://solarsystem.nasa.gov/resources/681/solar-system-temp...
- Judgmentality 6y agoIt's above 800 degrees Fahrenheit, but not even 500 degrees Celsius.
- astrea 6y agoOh, my mistake. I clearly can't read.
- deleted 6y ago[deleted]
- jacobush 6y agoTo be fair, it's usually easier to heat something to above ambient temperature than to cool it to below ambient temp.
- Tade0 6y agoThis is definitely the "in mice" of materials science. Solid state batteries, hydrocarbon fuel cells etc. all work - at high temperatures.
- w-m 6y agoSolid-state batteries have officially left the lab, "Mercedes-Benz’s newest electric city bus uses solid-state batteries": https://arstechnica.com/cars/2020/09/mercedes-benzs-newest-electric-city-bus-uses-solid-state-batteries/ https://arstechnica.com/cars/2020/09/mercedes-benzs-newest-e...
- ForHackernews 6y agoThis is interesting, but I'm kind of sceptical. We don't fully understand everything that's going on in a neuron well enough to say if we're replicating that behaviour or not. We don't even really know how many synapses an average neuron in the human brain has.
- deleted 6y ago[deleted]
- indrax 6y agoI think it's worth considering the hypothesis that neurons work because they have been evolving towards memristor like behavior.
- sroussey 6y agoFor neural networks, it makes sense to skip pure digital design. When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number, it made me want to just use superposition for addition, which is physical and instantaneous. Of course, that has all sorts of other problems that make it worse (so much worse). Once you learn how to slow down the earlier bits, you end up with all the bits arriving at the same time, and you can have up to n adds pipelined and timed to the clock when output matters. But with NN I think now would be a fun time (and likely the last decade as well) to rethink the basics right down to the basics. Everything need not be a NAND gate. :)
- dnautics 6y ago> When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number No, it's O(1). The bit size of how you represent a single datum does not grow with size of your model.
- RedComet 6y agoThat doesn't really contradict what he said. O(n) where n is a positive constant is O(1).
- morei 6y agoThis runs into the problem that the power efficiency of analogue circuitry is dramatically worse. A FET dissipates the least heat when it's fully on or fully off. Operating in the resistive regime will result in orders of magnitude more dissipation. It's difficult to over-come that, particularly because it's not a comparison between 'analogue v 64-bit float', but 'analogue versus 8-bit int'. (it's tough for scale analogue circuits to operate even when 8-bit accuracy).
- DennisP 6y agoThat just means an FET isn't the right device for analog circuitry. Maybe this memristor device will be more efficient.
- DesiLurker 6y agoI have followed this on & off. The thing that I got excited about was the amount of integration density and how little energy it takes to signal. I wonder how this fares on those aspects.
- lacker 6y agoThis is really interesting work, but I feel like the way the article summarizes it is getting it wrong. First Single Device To Act Like a Neuron One thing that’s kept engineers from copying the brain’s power efficiency and quirky computational skill is the lack of an electronic device that can, all on its own, act like a neuron. It would take a special kind of device to do that, one whose behavior is more complex than any yet created. This is just getting it wrong. Modern neural networks run as vectorized operations on a GPU. They are efficient because they do not use a single device to act like a single neuron, they can do massively parallelized work by optimizing for the hardware we have. If we had memristors, it doesn't seem like GPUs are the first thing they'd replace. More likely they would replace some sort of memory, since what makes them unique is storing information rather than performing analog operations faster.
- quicklyfrozen 6y agoThe brain is still way more efficient and parallel then a GPU (e.g. 12W compared to 280W), so being able to duplicate it closer sounds like a pretty compelling advancement.
- xyproto 6y agoTo me, this is one the most important news items this year. I've been excited about memristors since I first heard about them, and this sounds like an excellent application. These single device "neurons" also hints of things to come. Even larger neural nets with more efficient structures may, perhaps, cross a threshold in terms of what it can learn and express. I'm thinking Alpha Go Zero and beyond. I believe this is only the start of this type of hardware.
- sillysaurusx 6y agoI've heard about memristors for ... a decade? ... so I'm skeptical. What makes "this time different"? They were going to revolutionize the speed of data access, right? https://past.date-conference.com/proceedings-archive/2017/pdf/7095.pdf https://past.date-conference.com/proceedings-archive/2017/pd... Making them behave like neurons is a cool hack, but sort of tangential to why anyone cares about memristors. I keep an open mind though, so if this time really is different, then that's valid. It just seems like more hype.
- Judgmentality 6y agoI was also very interested in memristors when I first discovered them over 10 years ago. I actually wanted to go to graduate school to work on them, but was surprisingly unlucky in finding willing advisors with funding. In fact I even personally knew one of the researchers at HP and asked about working there, and he basically told me "you should do something else with your life." > I believe this is only the start of this type of hardware. I agree, but I also suspect a pragmatic solution is still decades away, much like nuclear fusion. Of course I no longer follow this area closely and could be completely wrong.
- pfdietz 6y ago"Acts like a neuron" is BS. Neurons are more like individual computers themselves, with all sorts of activity at the subcellular level.
- neolog 6y agoInteresting, where can I read more?
- pfdietz 6y agohttps://www.quantamagazine.org/neural-dendrites-reveal-their-computational-power-20200114/ https://www.quantamagazine.org/neural-dendrites-reveal-their...
- p1esk 6y agoAbstract Current hardware approaches to biomimetic or neuromorphic artificial intelligence rely on elaborate transistor circuits to simulate biological functions. However, these can instead be more faithfully emulated by higher-order circuit elements that naturally express neuromorphic nonlinear dynamics1,2,3,4. Generating neuromorphic action potentials in a circuit element theoretically requires a minimum of third-order complexity (for example, three dynamical electrophysical processes)5, but there have been few examples of second-order neuromorphic elements, and no previous demonstration of any isolated third-order element6,7,8. Using both experiments and modelling, here we show how multiple electrophysical processes—including Mott transition dynamics—form a nanoscale third-order circuit element. We demonstrate simple transistorless networks of third-order elements that perform Boolean operations and find analogue solutions to a computationally hard graph-partitioning problem. This work paves a way towards very compact and densely functional neuromorphic computing primitives, and energy-efficient validation of neuroscientific models.
- jcims 6y agoFun fact: there are about a hundred trillion atoms in every neuron.
- eutropia 6y ago
- woeirua 6y agoI'm not convinced that spiking behavior is all that's needed to really emulate what's going on in our neurons. Modern hardware designs are not really amenable to replicating the number or structure of synapses that we have, muchless the extremely complicated biochemistry that controls how and when they fire. It's good to see progress being made, but we really need to tone down the hyperbole in the popular scientific press.
- Balgair 6y agoNo, they mean that the memristor acts like the synapse. It was the last remaining basic two wire circuit element (the others being the resistor, capacitor, and inductor). We could model cells, up to the memristive portion, entirely with basic circuit components (albeit in very complicated arrangements). We were missing that part of our bits boxes. Now we aren't, well, sorta. It'll be a while before we get these in our hands. Also, these things are a bit bigger than just modern hardware. It's as if you just added a new primary color to Bob Ross; everything changes. We're going to need to redesign computers from the electrical-engineering-ground up. Well maybe, it'll be a few years (hopefully not decades)
- 1024core 6y ago12 years ago, on HN: "The memristor—the functional equivalent of a synapse—could revolutionize circuit design" https://news.ycombinator.com/item?id=397335 https://news.ycombinator.com/item?id=397335
- k__ 6y agoDidn't HP want to build a new type of computer on it?
- vbezhenar 6y agoIt was called The Machine: computer with unified memory. They released it without memristors in the end: HPE unveils The Machine, a single-memory computer with 160 terabytes of memory: https://news.ycombinator.com/item?id=14350396 https://news.ycombinator.com/item?id=14350396
- mywittyname 6y agoIt's fascinating that none of the comments mention neural network applications. Seems like these people came a bit too early to the party.
- freeone3000 6y agoWe've left biomimicry in the dust a fair time ago - neurons do not work like relus after all (nor do they work like memristors), but relus are great at solving problems. None of our current NN applications would even want something that behaves like a neuron. You'd essentially have to start from scratch. It's not even clear if behaves-like-a-neuron is better or not to be worth the effort. (And that's disregarding the entire fact that memristors do not behave like neurons either!)
- andy_ppp 6y agoDo we even understand how a neuron works well enough to say that either thing is correct. I thought a relu was just the activation function that decides what weight to triggers the next part of the network with?
- bryan0 6y agoIf you find water analogies helpful for circuit elements, here is one for the memristor: "The closest analogy I can think of is a sand filter, an item of apparatus used in water-purification plants. As contaminated water flows through a bed of sand and gravel, sediment gradually clogs the pores of the filter and thereby increases resistance. Reversing the flow flushes out the sediment and reduces resistance." -- https://www.americanscientist.org/article/the-memristor https://www.americanscientist.org/article/the-memristor
- chongli 6y agoIsn’t that like a capacitor? As it charges the voltage increases and the current drops until it reaches zero. Then if you discharge it you can start over again.
- Konohamaru 6y agoA capacitor is like a rubber membrane that has elasticity. When the capacitor/rubber membrane is stretched to its limit, it either breaks (dielectric breakdown) or it releases that energy causing a flow in the opposite direction. A sand filter has no limit (as it gets more clogged, it starts resembling a closed valve) and it can't release built up energy (flowing in the opposite direction just "opens the valve")
- freeone3000 6y agoCapacitors bridge voltage and charge, yes. In exactly the same way, memristors bridge charge and flux.
- 29athrowaway 6y agoActs like an artificial neuron model, not a biological neuron.
- deleted 6y ago[deleted]
- stephc_int13 6y agoWhat is clear to me is that our current nand based hardware is not ideal to model artificial neurones. But from my point of view, the difficult part is the many-to-many connectivity.
- IdiocyInAction 6y agoUnfortunately, we do not know how neurons learn, compute and represent information in biological neural networks. Though I feel that there would be great potential in figuring this out - the energy efficiency and capabilities of the brain are truly amazing.
- Balgair 6y agoHuh? >Learn Depends on the neuron, but things like NMDA and AMPA receptors are very well studied at this point and are the primary portion of synaptic plasticity, again, depending on the neuron: https://en.wikipedia.org/wiki/AMPA_receptor https://en.wikipedia.org/wiki/AMPA_receptor >Compute Again, depends on the neuron, but dendritic integration is a classic example: https://www.sciencedirect.com/topics/neuroscience/dendritic-integration https://www.sciencedirect.com/topics/neuroscience/dendritic-... > Represent information The do so in the frequency space. But again, it's highly dependent on the system of neurons : https://en.wikipedia.org/wiki/Neural_coding https://en.wikipedia.org/wiki/Neural_coding That said, holy cow yes. The brain is really something when it comes to power usage.
- IdiocyInAction 6y agoNeural coding is not a solved problem. Rate codes are not sufficient to explain all experimental data. Neither is learning; we know some things, sure, we have models for long- and short-term plasticity, but we have no idea how learning actually works. With computation, there are many hypothesis and dendrites do indeed do more than you would think, but again, these are just small details.
- VonGuard 6y agoI was just wondering what had happened to the Duke Nukem Forever of tech products. So.... nothing, really. Just more lab success. At this rate, HP will be dead long before this comes to fruition.
- imwm 6y agoFitting this article was written by a fella named Moore
- adatavizguy 6y agoOff topic. When something is on the tip of my tongue, I don't search for it because I have a strange belief that what my mind is trying to do to remember it is fire off neurons associated with the thought or idea, near it, to create the action potential. It is strange that I will eventually remember it within hours or the next day. When the action potential fires the neuron, dendrites are formed between neurons strengthening the memory. I wonder if there is any truth to that notion? That to solidify a memory, there can't be outside help, that electric chemical signal has to trigger firing the neurons associated with the memory to strengthen their connections.
- xxxtentachyon 6y agoI feel like if this were true (searching doesn’t strengthen the original memory), it would lead to a dramatic difference in a cohort’s ability to remember trivial information that they learned a long time ago, but it also seems really hard to perform a trial for something like this.
- ficklepickle 6y agoWhen I started learning programming, my French became much easier to recall. This was after ~ a decade of very infrequent French language use. Much of it has subsequently come back to me. Sample size of 1, yadda, yadda. Make of it what you will. I think it is interesting.
- 542354234235 6y agoI seem to remember reading about how thinking about something on the tip of your tongue and failing makes it harder to remember in the future, because concentrating on trying to remember it strengthens pathways that don’t lead back to the memory. So when you try in the future, those failed paths are stronger and more likely to be used again, further reinforcing the wrong association. Letting it go and hoping it comes to you also might not work, but it doesn’t strengthen bad pathways, so is more likely to allow you to remember in the future.
- visarga 6y agoWhat neural nets need in order to improve is lots of data, and especially simulated environments. We have had the advantage of billions of years of evolution, they start the same journey from scratch. Human brain is not smart in and of itself - it learns everything from the environment, including fundamental concepts and reasoning. We are not just surrounded by nature, with all its glorious detail, but also by the human society and culture. Our environment is very complex and rich. That's something neural nets have to replicate some other way, and simulation is one. It shows that just inventing a better artificial neuron does not meaningfully advance the problem of artificial intelligence. It's just one leg, it needs two legs to stand on.
- Flibble21 6y agoPlato has entered the chat
- mdturnerphys 6y agoThe news article mentions that the student on the paper has passed away. He died of suicide in his lab just a couple weeks after the paper was submitted to Nature [0]. Grad school is hard for a lot of people. Find someone to talk to if you're struggling. [0] https://padailypost.com/2019/02/16/engineering-student-found-dead-at-stanford/ https://padailypost.com/2019/02/16/engineering-student-found...
- mikorym 6y agoI've had an idea for a while, coming from a developing continent, that it would be good to have an alternative for students where they can start working when they are doing an advanced degree, but also simply a place for high school students to acclimate to some of the aspects of the work place. I know it sounds tangential, but this does resonate with my own roundabout way of going from studies -> work and the gross lack of fundamental science companies in Africa as a whole. I did not have depression, but rather my own cocktail of problems as a student, and a lot of it (at least for me) stemmed from clear dissonance between your current situation and an inability to anticipate a future self that is acceptable to your current self. With depression, the situation is similar and more importantly, you become more and more subjective and people outside your situation are needed, as probably is medication, to bring you back to personal assonance with what you want in life. The article doesn't say what the reasons were, so Wang may have had different reasons.
- pritovido 6y agoWhat is needed is removing the source of the problem in the first place. You are in a foreign country, with no friends, no family, working over time, in some places extremely little money. If you are burnout the first thing that you need to do is to stop. Make friends and lovers, travel and work outside for a year or so. To use medication or drugs is not going to solve your problems,if you continue living such an unnatural lifestyle. And I said that having lived that lifestyle myself, but I took my breaks from the toxic system.
- speeder 6y ago