4 ms·

I am currently working on my master's thesis on the topic of applying this type of knowledge to build a reinforcement learning system with spiking neural networ

I am currently working on my master's thesis on the topic of applying this type of knowledge to build a reinforcement learning system with spiking neural networks. The role of dopamine is crucial in learning. By combining spike-timing-dependent plasticity (STDP) with the reward (R-STDP), it is possible to address the spatial eligibility trace (which synapse caused the reward?). However, the design of the reward/utility function is one core issue, where this paper probably advances the field. I am currently looking into ways to balance the reward. Without balancing each trial causes more long term depression than long term potentiation, so that after a while the end the network dies. A different reward function results in the opposite. Another issue is the distal reward problem: Which event correspondents to which activity? I would be very glad to discuss related questions or exchange ideas with other experts or newcomers to this growing field. My mail is in my profile. This paper also got me thinking if this mechanism might explain why we enjoy listening to music. Music follows and breaks rules, which our brain continuously tries to predict and most of the time it succeeds.

longtom 7y ago
> This paper also got me thinking if this mechanism might explain why we enjoy listening to music. Music follows and breaks rules, which our brain continuously tries to predict and most of the time it succeeds. The link between predictability and art has been raised various times in the literature, even before Schmidhuber, but he had particularly interesting ideas about it: http://people.idsia.ch/~juergen/beauty.html http://people.idsia.ch/~juergen/beauty.html
- jcims 7y ago
  Great episode with Schmidhuber on Lex Fridman’s podcast. https://youtu.be/3FIo6evmweo https://youtu.be/3FIo6evmweo I’m coming at this from a career in infosec, never heard of the guy but really enjoyed his thought processes.
- temporaryPotato 7y ago
  > This paper also got me thinking if this mechanism might explain why we enjoy listening to music. Music follows and breaks rules, which our brain continuously tries to predict and most of the time it succeeds. I think that this explains my style to some extent [1]. One of the main themes behind what I write is that it should sound 1) original (less familiar) and 2) there needs to be an interplay between musical closure and tension, especially with half tones. [1] https://soundcloud.com/phillip-janvanzyl/spiral https://soundcloud.com/phillip-janvanzyl/spiral
so_tired 7y ago
Is there any research that spiking/dopamine type learning is good at "animal level" behaviour, but abstract and complex thinking is enabled by different mechanisms ?
- deephony 7y ago
  On the AI side of the fence the approach has been "let's see just how far Reinforcement Learning can take us, and then start making up stories (hypothesis) about what the secret ingredients are that are missing." On the neuroscience side of things my sense is that that's not a question that can be empirically answered any time soon. This experiment was interesting because they new what they were looking for going in. "What algo are these cells running?" is a hard question, "are these cells' firing activities consistent with this given algo is comparatively easy. Inference vs hypothesis testing.

longtom 7y ago

> This paper also got me thinking if this mechanism might explain why we enjoy listening to music. Music follows and breaks rules, which our brain continuously tries to predict and most of the time it succeeds. The link between predictability and art has been raised various times in the literature, even before Schmidhuber, but he had particularly interesting ideas about it: http://people.idsia.ch/~juergen/beauty.html http://people.idsia.ch/~juergen/beauty.html

jcims 7y ago
Great episode with Schmidhuber on Lex Fridman’s podcast. https://youtu.be/3FIo6evmweo https://youtu.be/3FIo6evmweo I’m coming at this from a career in infosec, never heard of the guy but really enjoyed his thought processes.
temporaryPotato 7y ago
> This paper also got me thinking if this mechanism might explain why we enjoy listening to music. Music follows and breaks rules, which our brain continuously tries to predict and most of the time it succeeds. I think that this explains my style to some extent [1]. One of the main themes behind what I write is that it should sound 1) original (less familiar) and 2) there needs to be an interplay between musical closure and tension, especially with half tones. [1] https://soundcloud.com/phillip-janvanzyl/spiral https://soundcloud.com/phillip-janvanzyl/spiral

so_tired 7y ago

Is there any research that spiking/dopamine type learning is good at "animal level" behaviour, but abstract and complex thinking is enabled by different mechanisms ?

deephony 7y ago
On the AI side of the fence the approach has been "let's see just how far Reinforcement Learning can take us, and then start making up stories (hypothesis) about what the secret ingredients are that are missing." On the neuroscience side of things my sense is that that's not a question that can be empirically answered any time soon. This experiment was interesting because they new what they were looking for going in. "What algo are these cells running?" is a hard question, "are these cells' firing activities consistent with this given algo is comparatively easy. Inference vs hypothesis testing.