5 ms·

Can "traditional deep learning methods" replicate the kind of unsupervised learning demonstrated by cortical.io's semantic retina (fox eats rodent example in th

Can "traditional deep learning methods" replicate the kind of unsupervised learning demonstrated by cortical.io's semantic retina (fox eats rodent example in the video)?

kmike84 12y ago
Sure they can, see e.g. http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pdf http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pd... or http://arxiv.org/pdf/1301.3618v2.pdf http://arxiv.org/pdf/1301.3618v2.pdf. Check the first paper - when working on this problem researchers from Stanford developed a way to measure the quality of an approach, evaluated the results on 30k+ relation examples from 2 different datasets and compared their algorithm with 4 other algorithms. The problem with cortical.io or numenta (both commercial companies) is that they don't compare their approaches with existing approaches and don't evaluate them on public datasets. And when people do such comparison existing approaches turn out to be better. It is totally possible these algorithms are good and they provide something that "traditional" methods don't provide. But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning. Also, they use their own terminology and usually refer only to their own papers or to some obscure papers from ten years ago, which doesn't help.
- zo1 12y ago
  >"But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning." You'd have to provide them then with a decent argument about what economic/competitive benefit they would get by what you suggest. You say they're commercial companies, so then don't be surprised when their approach is based on financial incentives. But trust me, they're probably begging to have someone show them a better alternative that will give them a competitive edge. So, either no one like you has given them that alternative/idea. Someone has already, and they rejected the financial benefit. Or, finally, someone already told them your idea but they discovered there was no financial benefit and the only benefit was for the greater society.
  - kmike84 12y ago
    A cynical view on this could be the following: Numenta sells licenses, cortical.io sells api requests, they benefit from more developers using their tools. Commercial companies which fund deep learning research (like Microsoft and Google ) develop their own products which are based on machine learning, they benefit from advancing state of the arts, from better algorithms. So we have quality publications and algorithms from Microsoft or Google and quality marketing from Numenta or cortical.io.
Houshalter 12y ago
The idea of representing words as vectors and testing was originally a deep learning thing. See Google's word2vec which is famous for being able to do things like "king"-"man"+"woman"= "queen".
- biomimic 12y ago
  Even before Googles word2vec at Berkeley Lab they were experimenting with this kind of vector space 'a search engine that thinks, almost' http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-thinks-almost/ http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-th...

Can "traditional deep learning methods" replicate the kind of unsupervised learning demonstrated by cortical.io's semantic retina (fox eats rodent example in the video)?

kmike84 12y ago

Sure they can, see e.g. http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pdf http://nlp.stanford.edu/pubs/SocherChenManningNg_NIPS2013.pd... or http://arxiv.org/pdf/1301.3618v2.pdf http://arxiv.org/pdf/1301.3618v2.pdf. Check the first paper - when working on this problem researchers from Stanford developed a way to measure the quality of an approach, evaluated the results on 30k+ relation examples from 2 different datasets and compared their algorithm with 4 other algorithms. The problem with cortical.io or numenta (both commercial companies) is that they don't compare their approaches with existing approaches and don't evaluate them on public datasets. And when people do such comparison existing approaches turn out to be better. It is totally possible these algorithms are good and they provide something that "traditional" methods don't provide. But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning. Also, they use their own terminology and usually refer only to their own papers or to some obscure papers from ten years ago, which doesn't help.

zo1 12y ago
>"But this is yet to be shown; for some reason authors decide not to "compete" on a same ground. Instead of "promoting" their methods in scientific community via publications / comparisions with existing approaches they seem to focus on people who have little knowledge of modern machine learning." You'd have to provide them then with a decent argument about what economic/competitive benefit they would get by what you suggest. You say they're commercial companies, so then don't be surprised when their approach is based on financial incentives. But trust me, they're probably begging to have someone show them a better alternative that will give them a competitive edge. So, either no one like you has given them that alternative/idea. Someone has already, and they rejected the financial benefit. Or, finally, someone already told them your idea but they discovered there was no financial benefit and the only benefit was for the greater society.
- kmike84 12y ago
  A cynical view on this could be the following: Numenta sells licenses, cortical.io sells api requests, they benefit from more developers using their tools. Commercial companies which fund deep learning research (like Microsoft and Google ) develop their own products which are based on machine learning, they benefit from advancing state of the arts, from better algorithms. So we have quality publications and algorithms from Microsoft or Google and quality marketing from Numenta or cortical.io.

Houshalter 12y ago

The idea of representing words as vectors and testing was originally a deep learning thing. See Google's word2vec which is famous for being able to do things like "king"-"man"+"woman"= "queen".

biomimic 12y ago
Even before Googles word2vec at Berkeley Lab they were experimenting with this kind of vector space 'a search engine that thinks, almost' http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-thinks-almost/ http://newscenter.lbl.gov/2005/03/31/a-search-engine-that-th...