4 ms·

I'm confused. I don't get what the novelty in this is. It looks like all they do is include an input that identifies different tasks and then trains one neural

by rayuela 10y ago

I'm confused. I don't get what the novelty in this is. It looks like all they do is include an input that identifies different tasks and then trains one neural network to learn a separate distributions for each task, with some weight sharing...

Filligree 10y ago
It may be an obvious solution, but has anyone done that before? While retaining the ability to have said weight sharing?
- rayuela 10y ago
  Of course, people have done this before [1]. There is quite of bit of research looking into Multi-task learning. Just look through some of the references in that Luong et all paper. Deepmind has been putting out some amazing research lately, but this paper definitely does not fall in that category. 1. http://nlp.stanford.edu/pubs/luong2016iclr_multi.pdf http://nlp.stanford.edu/pubs/luong2016iclr_multi.pdf
- gnaritas 10y ago
  > but has anyone done that before? The answer that question is nearly always YES, no matter what you ask it about.
- etiam 10y ago
  I know Jeff Hawkins' Numenta have been tracking "permanence" in their artificial synapses for many years now.
eutectic 10y ago
And relativity is just the idea that the laws of physics are the same in every inertial reference frame.
- rayuela 10y ago
  To compare this to the introduction of relativity is just silly.
  - jacquesm 10y ago
    Less silly than it looks at first sight. After all, for day-to-day use (except for GPS I guess, and most people probably don't even realize it would not work without a correction for relativistic effects) relativity is very little gain over classical Newtonian physics and a lot more complex to work out from a mathematical point of view. So even though 'that's how it really should work' we tend to take the shortcut because it is 'good enough' for almost all use cases. Which caused us to miss the wood for the trees for a long time. This minor change is what enables learning in the first place, and as such it could easily be a game changer.
    - posterboy 10y ago
      Nuclear reactors?
  - eutectic 10y ago
    Making an analogy is not the same as implying equivalence.

by rayuela 10y ago

Filligree 10y ago

It may be an obvious solution, but has anyone done that before? While retaining the ability to have said weight sharing?

rayuela 10y ago
Of course, people have done this before [1]. There is quite of bit of research looking into Multi-task learning. Just look through some of the references in that Luong et all paper. Deepmind has been putting out some amazing research lately, but this paper definitely does not fall in that category. 1. http://nlp.stanford.edu/pubs/luong2016iclr_multi.pdf http://nlp.stanford.edu/pubs/luong2016iclr_multi.pdf
gnaritas 10y ago
> but has anyone done that before? The answer that question is nearly always YES, no matter what you ask it about.
etiam 10y ago
I know Jeff Hawkins' Numenta have been tracking "permanence" in their artificial synapses for many years now.

eutectic 10y ago

And relativity is just the idea that the laws of physics are the same in every inertial reference frame.

rayuela 10y ago
To compare this to the introduction of relativity is just silly.
- jacquesm 10y ago
  Less silly than it looks at first sight. After all, for day-to-day use (except for GPS I guess, and most people probably don't even realize it would not work without a correction for relativistic effects) relativity is very little gain over classical Newtonian physics and a lot more complex to work out from a mathematical point of view. So even though 'that's how it really should work' we tend to take the shortcut because it is 'good enough' for almost all use cases. Which caused us to miss the wood for the trees for a long time. This minor change is what enables learning in the first place, and as such it could easily be a game changer.
  - posterboy 10y ago
    Nuclear reactors?
- eutectic 10y ago
  Making an analogy is not the same as implying equivalence.