3 ms·
Thank you for taking the time to write a substantial reply! You might be right about deep learning having stumbled upon something deep and profound. Or it may
by YeGoblynQueenne 9y ago
Thank you for taking the time to write a substantial reply!
You might be right about deep learning having stumbled upon something deep and
profound. Or it may just be the case of "big machine performs well at task
that is hard for humans". Like you say, we will have to wait and see.
It's just that we won't be seeing much, unless the field focuses real effort on
the task of coming up with some kind of "calculus of (deep) learning". As
things go right now, it might take more than ten years to see the progress
you're hoping for.
On a personal note, I should say that I do quite like your idea, in principle.
But that's because you're proposing a grammar of design spaces; I think we
should use grammars everywhere :P
On the demonstration of your technique in other domains than vision- well,
that would be really interesting to see. I watched a presentation by a
gentleman called Willem Zuidema recently, whose work is on computational
linguistics. His team had worked to interpret their deep learning models by
visualising their hidden unit activations; he said that it was extremely
painful and didn't scale well (he was talking at CoCoSym 2018, a workshop that
featured much work on the prospects of combining deep learning with symbolic
techniques, mainly for interpretation). If your method can work well in other
domains it will definitely be useful to many people. It's still not the kind
of theoretical result I'm hoping for, but it would be nice to see a principled
way to extract symbolic representations from a continuous space - if that's
what you're talking about.
Anyway, I'll keep an eye out :)