3 ms·

I think this is the most interesting direction of research in CS one can get involved with today. In 10-20 years, half of CS graduate will be working in comput

I think this is the most interesting direction of research in CS one can get involved with today. In 10-20 years, half of CS graduate will be working in computational biology and the other half developing SDR-based machine intelligence.

kmike84 12y ago
It is not clear that these methods work better than "traditional" deep learning methods; in fact, they haven't produced any good results yet, and most experts think the hype around CLA is not because of its technical properties. See e.g. * http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_yann_lecun/chigv2l http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_... * http://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clw89b4 http://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_... * http://www.reddit.com/r/MachineLearning/comments/2iejpg/systematically_compared_with_other_pattern/ http://www.reddit.com/r/MachineLearning/comments/2iejpg/syst...
- java-man 12y ago
  Can "traditional deep learning methods" replicate the kind of unsupervised learning demonstrated by cortical.io's semantic retina (fox eats rodent example in the video)?
  - kmike84 12y ago
    Sure they can, see e.g. http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pdf http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pd... or http://arxiv.org/pdf/1301.3618v2.pdf http://arxiv.org/pdf/1301.3618v2.pdf. Check the first paper - when working on this problem researchers from Stanford developed a way to measure the quality of an approach, evaluated the results on 30k+ relation examples from 2 different datasets and compared their algorithm with 4 other algorithms. The problem with cortical.io or numenta (both commercial companies) is that they don't compare their approaches with existing approaches and don't evaluate them on public datasets. And when people do such comparison existing approaches turn out to be better. It is totally possible these algorithms are good and they provide something that "traditional" methods don't provide. But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning. Also, they use their own terminology and usually refer only to their own papers or to some obscure papers from ten years ago, which doesn't help.
    - zo1 12y ago
      >"But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning." You'd have to provide them then with a decent argument about what economic/competitive benefit they would get by what you suggest. You say they're commercial companies, so then don't be surprised when their approach is based on financial incentives. But trust me, they're probably begging to have someone show them a better alternative that will give them a competitive edge. So, either no one like you has given them that alternative/idea. Someone has already, and they rejected the financial benefit. Or, finally, someone already told them your idea but they discovered there was no financial benefit and the only benefit was for the greater society.
      - kmike84 12y ago
        A cynical view on this could be the following: Numenta sells licenses, cortical.io sells api requests, they benefit from more developers using their tools. Commercial companies which fund deep learning research (like Microsoft and Google ) develop their own products which are based on machine learning, they benefit from advancing state of the arts, from better algorithms. So we have quality publications and algorithms from Microsoft or Google and quality marketing from Numenta or cortical.io.
  - Houshalter 12y ago
    The idea of representing words as vectors and testing was originally a deep learning thing. See Google's word2vec which is famous for being able to do things like "king"-"man"+"woman"= "queen".
    - biomimic 12y ago
      Even before Googles word2vec at Berkeley Lab they were experimenting with this kind of vector space 'a search engine that thinks, almost' http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-thinks-almost/ http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-th...
arketyp 12y ago
As far as this technology will deliver I belive it will become a engineering or scientific branch in its own right. I suspect it will be a bit like economics, with lots of clever people deriving sophisticated models but expert knowledge is all about heuristics (at least this is my layman view of economics): anyone can understand the basic principles but one would be hard pushed to claim there is anyone who really grasps the thing in entirety.
- java-man 12y ago
  ... except those who control it. :-)

kmike84 12y ago

It is not clear that these methods work better than "traditional" deep learning methods; in fact, they haven't produced any good results yet, and most experts think the hype around CLA is not because of its technical properties. See e.g. * http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_yann_lecun/chigv2l http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_... * http://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clw89b4 http://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_... * http://www.reddit.com/r/MachineLearning/comments/2iejpg/systematically_compared_with_other_pattern/ http://www.reddit.com/r/MachineLearning/comments/2iejpg/syst...

java-man 12y ago
Can "traditional deep learning methods" replicate the kind of unsupervised learning demonstrated by cortical.io's semantic retina (fox eats rodent example in the video)?
- kmike84 12y ago
  Sure they can, see e.g. http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pdf http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pd... or http://arxiv.org/pdf/1301.3618v2.pdf http://arxiv.org/pdf/1301.3618v2.pdf. Check the first paper - when working on this problem researchers from Stanford developed a way to measure the quality of an approach, evaluated the results on 30k+ relation examples from 2 different datasets and compared their algorithm with 4 other algorithms. The problem with cortical.io or numenta (both commercial companies) is that they don't compare their approaches with existing approaches and don't evaluate them on public datasets. And when people do such comparison existing approaches turn out to be better. It is totally possible these algorithms are good and they provide something that "traditional" methods don't provide. But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning. Also, they use their own terminology and usually refer only to their own papers or to some obscure papers from ten years ago, which doesn't help.
  - zo1 12y ago
    >"But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning." You'd have to provide them then with a decent argument about what economic/competitive benefit they would get by what you suggest. You say they're commercial companies, so then don't be surprised when their approach is based on financial incentives. But trust me, they're probably begging to have someone show them a better alternative that will give them a competitive edge. So, either no one like you has given them that alternative/idea. Someone has already, and they rejected the financial benefit. Or, finally, someone already told them your idea but they discovered there was no financial benefit and the only benefit was for the greater society.
    - kmike84 12y ago
      A cynical view on this could be the following: Numenta sells licenses, cortical.io sells api requests, they benefit from more developers using their tools. Commercial companies which fund deep learning research (like Microsoft and Google ) develop their own products which are based on machine learning, they benefit from advancing state of the arts, from better algorithms. So we have quality publications and algorithms from Microsoft or Google and quality marketing from Numenta or cortical.io.
- Houshalter 12y ago
  The idea of representing words as vectors and testing was originally a deep learning thing. See Google's word2vec which is famous for being able to do things like "king"-"man"+"woman"= "queen".
  - biomimic 12y ago
    Even before Googles word2vec at Berkeley Lab they were experimenting with this kind of vector space 'a search engine that thinks, almost' http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-thinks-almost/ http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-th...

arketyp 12y ago

As far as this technology will deliver I belive it will become a engineering or scientific branch in its own right. I suspect it will be a bit like economics, with lots of clever people deriving sophisticated models but expert knowledge is all about heuristics (at least this is my layman view of economics): anyone can understand the basic principles but one would be hard pushed to claim there is anyone who really grasps the thing in entirety.

java-man 12y ago
... except those who control it. :-)