8 ms·

Meet the algorithm that can learn “everything about anything”

175 points by ossama 12y ago

cscurmudgeon 12y ago
One of the hallmarks of bad science is overly grand claims paired with aggressive marketing. Bad times are coming for AI again.
- zwieback 12y ago
  Judging by the amount of "deep learning" submissions to HN bad times for AI are already here although maybe they never left. In defense of the LEVAN thing, though, I didn't see any claims that this is science at all, more like an exploratory application illustrating an algorithm.
  - _delirium 12y ago
    I think the increase in deep-learning links in this particular community is partly because Google recently dropped a huge pile of money on a deep-learning startup (Deep Mind), which gave it more visibility in the startup scene. Though in that case, are we talking about a separate AI frenzy, or is it just an appendage of the current startup/acquisitions frenzy?
  - agibsonccc 12y ago
    Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box that takes a really long time to train with little to no reward. The hype isn't all for naught either. I'll elaborate if asked, but won't bore you guys otherwise. The results coming out from different tasks are currently blowing away many of the old school algorithms in tasks like sentiment analysis, speech to text, object recognition, among others. [1] http://deeplearning4j.org/ http://deeplearning4j.org/
    - ScottBurson 12y ago
      Wow, this is very cool! I take it your business model is basically consulting?
      - agibsonccc 12y ago
        Yes. I'll have a cleaner website with that going up next week. I just got done running a 20 node cluster on amazon training a few different datasets, it hummed ;). The overall idea is a support, services, training model. Support: Install/Infrastructure Services: Help with data etl/onboarding, tuning Training: How to use deep learning, how to think about it, and how to run/setup clusters. This will be run through the machine learning academy I teach at[1] [1]: http://zipfianacademy.com/ http://zipfianacademy.com/
    - cscurmudgeon 12y ago
      Look up Thinking Machines :) My point is that mixing business goals and scientific truth is dangerous if not handled carefully. That said all the best with both your goals.
      - agibsonccc 12y ago
        Sure! You're absolutely right. There are just a lot of myths about deep learning that I like clearing up, that being: it's a real world algorithm with actual merits, not just some marketing hype. Marketing hype and machine learning really does make things convoluted. That being said, what DOESN'T the press exaggerate? The track record for AI has definitely been over promise and under deliver. I think the hardware is getting there where we can start making real progress though. In this case, google is putting its money where its mouth is. Thanks for the wishes, it's been a great ride so far. My overall goal with this is to bring deep learning to everyone else. I see real merits in auto discovery of trends to automate some of the worst parts of machine learning and would like others to see these benefits as well. Seeing cool apps built with it is another side goal of this as well.
        escape_goat 12y ago
- fhars 12y ago
  And it is [patent pending] See http://levan.cs.washington.edu/?state=show_about http://levan.cs.washington.edu/?state=show_about
  - judk 12y ago
    Which means it is illegal for a practitioner to read about this work, and it is best left ignored by the scientific and technical community
    - bsenftner 12y ago
      illegal? No.
      - xerophtye 12y ago
        I think what he is trying to say is that if you read about it, it would influence you. And you may end up building something similar (possibly remotely) to it. And then you can be sued for it. I am not saying that i agree with him, i am just trying to clarify his point
- slashcom 12y ago
  FWIW, the paper is much more modest and honest than the gigaom article. It's certainly a hot topic and getting wide coverage, but most scientists are being fairly conservative about their claims.* * Grant applications and future work sections excluded :)
- bo1024 12y ago
  Nah, I think you're overreacting. It seems like every pop-sci article on CS research reads like this one (e.g. "breakthrough in cryptography", "scientists solve quantum computing", etc.). Usually it's just weak journalism rather than researchers trying to overinflate.
- varelse 12y ago
  It's easy to demonstrate that this is a huge advance in AI. All these guys need to do is start entering Kaggle competitions and win, and win, and win, and win... That said, given that Google et al. are scraping every corner of the bottom of the barrel in search of advances in search, I think the good times for AI peeps will continue for some time...
  - kunstmord 12y ago
    Most of the Kaggle competitions (if not all) do not allow any use of external data, as far as I know.
mikkom 12y ago
Here is the actual paper for those interested http://levan.cs.washington.edu/ngrams/objectNgrams_cvpr14.pdf http://levan.cs.washington.edu/ngrams/objectNgrams_cvpr14.pd...
- _flag 12y ago
  Summary of the paper for those who don't want to read it: So basically there are two categories of "learning" involved in this sort of research, supervised and unsupervised. In supervised learning, someone gives the computer a long list of concepts and their attributes ("frog", "green frog", "jumping frog") and a set of pictures to go with each item, and feeds them into a visual-recognition algorithm. In unsupervised learning, the computer is given a concept like "frog" but then has to discover all the variations itself and get its own visual data to match. The claim in this paper is that they have made the unsupervised learning as strong as the supervised learning. That is, they give the computer a concept ("frog"), it goes and searches through Google Books for common variations ("green frog", "jumping frog") and then uses Google image search to fetch images for each of those queries. They can then remove the obvious false positives (they test to see which images seem to screw up their learning algorithm and leave those out), and the result they get is on par with the supervised learning methods. ---------------------- In my opinion, this is only mildly interesting because Google Image Search functions based on human input anyway -- Google knows the difference between a "frog" and a "jumping frog" or even a "camel" simply because people on the internet caption such images and Google can make associations between images and their captions. Essentially, what the researchers have managed to do is outsource the work of some grad student to millions of people around the world through Google. Of course, it could be argued that there is some sort of parallel with what humans actually do (we know what things are called because we hear other people call them that), but even if I didn't know the name of an animal I could still tell you when the same animal is in different pictures, and I can also tell you when it's jumping and what colour it is. I don't need to have someone caption the image for me to understand the broad range of situations to which the caption "jump" applies.
  - xerophtye 12y ago
    >I don't need to have someone caption the image for me to understand the broad range of situations to which the caption "jump" applies. I wonder if this has anything to do with the fact that we can jump too. That we can translate the frog's position into something we do as well. of course one can argue that we can do the same for non-anthro-moprhic things as well. What i think is that we dont directly relate pictures, as the software is taught. What we do is translate that 2D picture into something we'd see in the 3d world. And that 3d "vision" isn't just another image. It represents an object in our world. something that has shape, existence etc. something which we can observe from other senses as well. For us a picture doesn't always represent an abstract thing, an arbitary pattern of colours. It usually represents something concrete. Something about which we have tons of other pieces of knowledge as well. So we relate pictures by checking if they map to the same real-world object. And here that "object" is a sort of nexus of many pieces of information we have on it which is a product of many direct and indirect human experiences. So i don't really think that we are in a position to teach a computer to do anything like that.
cyborgx7 12y ago
The problem with all approaches to machine learning I see today is that they only focus on grouping and separating concepts based on certain characteristics. They seem to be all fundamentally statistical. None of them seem to work towards a fundamental understanding of what the concepts mean. I'm not sure how that could be accomplished though. Is there even a meaningful distinction to be made between beeing able to identify a concept and understanding what a concept actually means?
- pavpanchekha 12y ago
  This is in fact a classic criticism of modern AI. Classic approaches tried to replicate human reasoning mechanisms. The work was very cool, but far harder to understand and scale, compared to modern statistical algorithms. To some extent, you are faced with a choice: a small palette of mediocre techniques whose workings are beautiful theories of the human mind; or, a broadly-applicable, incredibly practical set of tools with little motivation or spirit.
  - PurplePanda 12y ago
    I understand that other being hard to scale in a machine efficiency sense, the old techniques were also rather limited in applicable domain. They were often based on unjustified models of whatever the author decided was a good model of thought/reasoning, whether "frames", predicate calculus, constraint propagation or whatever. It seems to me that although you say statistical methods have little motivation, the motivation of the alternative - heuristics - is very questionable. Can anyone correct me on this?
    - pavpanchekha 12y ago
      That's actually what I meant by scale—that they were slow isn't really a problem if you're interested in AI for the raw fun of it. While justifications for using logic or constraints or probability theory were often lacking from classical techniques, do understand that logic itself was originally developed as a crystallization of proper human thought. And many of the classic AI folks did care deeply about understanding how humans achieved certain results. For example, take a look at Marvin Minsky's work. To the contrary, modern techniques aren't interested in replicating human thought; they are simply interested in replicating human results.
cedias 12y ago
(Disclaimer: I am not an native english speaker) The title of the article and of the algorithm (learn everything about anything) is a bit misleading. You might believe that it learns everything, period. Actually it's more about finding every variation of a "concept", I quote their website: "a fully automated method that given any concept, e.g., horse, discovers an exhaustive vocabulary for it that explains all variations (i.e., actions, interactions, attributes, etc) that modify its appearance."
- Shizka 12y ago
  I wonder. If they have an exhaustive vocabulary would it be possible to generate a picture of what the system believes an object to look like? I know that there is something called generative models in machine learning and my guess is that it could be applied here.
  - cedias 12y ago
    Well you'd have to build a probabilistic model for each concepts, whether on pixels or on features, and you could use it to generate images randomly. It might show up some good shapes.
  - Houshalter 12y ago
    It's possible, but generally generative models have to be trained in a specific way. If not, you could do something like for every layer of the neural net, you train another NN which can "predict" the layer below it, it's input. Then you can work your way down each layer to try to find an input which would produce that output. Another way is to use some kind of optimization to find an input which produces that pattern (e.g. backprop to the pixels themselves.) This will give you the image that most strongly triggers that output. Not necessarily a typical example.
sparky_z 12y ago
This strongly reminds me of the first chapter of Greg Egan's Diaspora [1] (which I can't recommend enough) in which newly-formed AIs bootstrap their way to consciousness in part by connecting randomly to an online library and using the various data streams to build up an associative model of the world. [1] http://gregegan.customer.netspace.net.au/DIASPORA/01/Orphanogenesis.html http://gregegan.customer.netspace.net.au/DIASPORA/01/Orphano...
- resdirector 12y ago
  Seconded. Diaspora is one of the best sci if books I've read. Highly recommend it (and all of Egan's work) to the HN community.
  - wfn 12y ago
    Thirded. Before Diaspora, I first read "Wang's Carpets"[1] which is a short story of his. Then found out this story had later been incorporated as a chapter into the book. I remember basically immediately ordering said book that night. fwiw, that "Webly-Supervised Visual Concept Learning" reminds me of the stuff that Hinton et al. do re: unsupervised (concept, etc.) learning (using restricted Boltzmann machines, and so on.) Good talk on the subject (of deep learning, etc.): https://www.youtube.com/watch?v=AyzOUbkUf3M https://www.youtube.com/watch?v=AyzOUbkUf3M [1]: read online here: http://bookre.org/reader?file=222997 http://bookre.org/reader?file=222997
    - MachineElf 12y ago
      Umm.. fourthed? I just couldn't help but jump in and also recommend Greg Egan's "Permutation City". That book is just wonderful... think simulation, cellular automata as a model for computation, artificial life and all that other good stuff :). Also, about the LEVAN thing... given the amount of data available online, both in various structured formats and unstructured formats, don't be surprised if deep learning will yield better and better results moving forward. To me though, they mostly seem evolutionary rather than revolutionary. I mean if you look back at the AI field, during the days before the "AI winter" came, huge amounts of data is one thing researchers back then didn't have available. This is not to say that there haven't been advances in learning algorithms at all recently. ..
deleted 12y ago
[deleted]
motyar 12y ago
Where is the code? is that opensource? and which programming language it uses?
- cedias 12y ago
  You have everything here: http://levan.cs.washington.edu/?state=show_about http://levan.cs.washington.edu/?state=show_about I quote the readme: > This is an implementation of the "Learning Everything about Anything" system. The system is implemented in MATLAB, with various helper functions written in Shell, Python, MEX C++ for efficiency reasons. For details about the method, please see [1]. This readme contains instructions on using the code, as well as accessing/using already trained models for various concepts. For questions concerning the code please contact Santosh Divvala (http://homes.cs.washington.edu/~santosh http://homes.cs.washington.edu/~santosh) at santosh@cs.washington.edu. The software has been tested on Linux using MATLAB versions R2011a. There may be compatibility issues with older versions of MATLAB. At least 4GB of memory (plus an additional 0.75GB for each parallel matlab worker) is assumed.
nawitus 12y ago
AIXI-mc can also learn everything about anything. And you can download the source code online[1]. 1. http://jveness.info/software/default.html http://jveness.info/software/default.html
ZeroFries 12y ago
This doesn't really help the symbol grounding problem: it uses pre-human-sorted data (who use their own ability to match symbols and meaning) to form its associative network. So, it's using human consciousness as part of the input to form its own consciousness. You could argue that humans use other humans' consciousness to develop its own, but now you have the infinite regress which seem to be the fate of all symbol grounding contemplations. Surely there has to be a starting point, some "axioms" you initially accept about the world to start the process. Maybe these are embedded in our DNA and have evolved to be a practical start (eg: sharp sensory input from nerves on your skin is automatically linked with pain, which we automatically avoid).
clubhi 12y ago
// learn everything wget google.com?search=random_string() >> everything.txt
deleted 12y ago
[deleted]
graycat 12y ago
If we define 'learning' in an appropriate way, then any good research library already knows "everything about anything". Okay, let the thing 'learn' about the Kuhn-Tucker conditions by searching on Google and reading, say, Wikipedia or some books at Google or Amazon. Then have the thing show that for problems in functional form the Zangwill and Kuhn-Tucker constraint qualifications are independent. Do that and I will start to believe that the terminology 'deep learning' is appropriate. I'm not holding my breath. Yes, it may be that in some rough sense the kind of 'learning' it is doing is roughly like some of the learning of a child of, say, 2 as it is starting to learn about language and things. Yes, it may be that such 'learning' is a significant part of the intelligence of, say, a child of 3-5. Maybe. Big, huge maybe. When I was working in AI, I noticed the terminology had been cooked up to imply much more than was being accomplished. Now, as I understand it, there is a specific definition for the current AI term 'deep learning' and has to do with the 'depth' of where adjust parameters in a neural network, not how 'deep' the 'learning' is about the subject in question. Cute terminology.
dang 12y ago
This is such a dumb title. Can any of you suggest a better one?
bra-ket 12y ago
I'm surprised NELL learner is not cited as it's closely related: http://rtw.ml.cmu.edu/rtw/ http://rtw.ml.cmu.edu/rtw/

175 points by ossama 12y ago

One of the hallmarks of bad science is overly grand claims paired with aggressive marketing. Bad times are coming for AI again.

zwieback 12y ago
Judging by the amount of "deep learning" submissions to HN bad times for AI are already here although maybe they never left. In defense of the LEVAN thing, though, I didn't see any claims that this is science at all, more like an exploratory application illustrating an algorithm.
- _delirium 12y ago
  I think the increase in deep-learning links in this particular community is partly because Google recently dropped a huge pile of money on a deep-learning startup (Deep Mind), which gave it more visibility in the startup scene. Though in that case, are we talking about a separate AI frenzy, or is it just an appendage of the current startup/acquisitions frenzy?
- agibsonccc 12y ago
  Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box that takes a really long time to train with little to no reward. The hype isn't all for naught either. I'll elaborate if asked, but won't bore you guys otherwise. The results coming out from different tasks are currently blowing away many of the old school algorithms in tasks like sentiment analysis, speech to text, object recognition, among others. [1] http://deeplearning4j.org/ http://deeplearning4j.org/
  - ScottBurson 12y ago
    Wow, this is very cool! I take it your business model is basically consulting?
    - agibsonccc 12y ago
      Yes. I'll have a cleaner website with that going up next week. I just got done running a 20 node cluster on amazon training a few different datasets, it hummed ;). The overall idea is a support, services, training model. Support: Install/Infrastructure Services: Help with data etl/onboarding, tuning Training: How to use deep learning, how to think about it, and how to run/setup clusters. This will be run through the machine learning academy I teach at[1] [1]: http://zipfianacademy.com/ http://zipfianacademy.com/
  - cscurmudgeon 12y ago
    Look up Thinking Machines :) My point is that mixing business goals and scientific truth is dangerous if not handled carefully. That said all the best with both your goals.
    - agibsonccc 12y ago
      Sure! You're absolutely right. There are just a lot of myths about deep learning that I like clearing up, that being: it's a real world algorithm with actual merits, not just some marketing hype. Marketing hype and machine learning really does make things convoluted. That being said, what DOESN'T the press exaggerate? The track record for AI has definitely been over promise and under deliver. I think the hardware is getting there where we can start making real progress though. In this case, google is putting its money where its mouth is. Thanks for the wishes, it's been a great ride so far. My overall goal with this is to bring deep learning to everyone else. I see real merits in auto discovery of trends to automate some of the worst parts of machine learning and would like others to see these benefits as well. Seeing cool apps built with it is another side goal of this as well.
      - escape_goat 12y ago
fhars 12y ago
And it is [patent pending] See http://levan.cs.washington.edu/?state=show_about http://levan.cs.washington.edu/?state=show_about
- judk 12y ago
  Which means it is illegal for a practitioner to read about this work, and it is best left ignored by the scientific and technical community
  - bsenftner 12y ago
    illegal? No.
    - xerophtye 12y ago
      I think what he is trying to say is that if you read about it, it would influence you. And you may end up building something similar (possibly remotely) to it. And then you can be sued for it. I am not saying that i agree with him, i am just trying to clarify his point
slashcom 12y ago
FWIW, the paper is much more modest and honest than the gigaom article. It's certainly a hot topic and getting wide coverage, but most scientists are being fairly conservative about their claims.* * Grant applications and future work sections excluded :)
bo1024 12y ago
Nah, I think you're overreacting. It seems like every pop-sci article on CS research reads like this one (e.g. "breakthrough in cryptography", "scientists solve quantum computing", etc.). Usually it's just weak journalism rather than researchers trying to overinflate.
varelse 12y ago
It's easy to demonstrate that this is a huge advance in AI. All these guys need to do is start entering Kaggle competitions and win, and win, and win, and win... That said, given that Google et al. are scraping every corner of the bottom of the barrel in search of advances in search, I think the good times for AI peeps will continue for some time...
- kunstmord 12y ago
  Most of the Kaggle competitions (if not all) do not allow any use of external data, as far as I know.

mikkom 12y ago

Here is the actual paper for those interested http://levan.cs.washington.edu/ngrams/objectNgrams_cvpr14.pdf http://levan.cs.washington.edu/ngrams/objectNgrams_cvpr14.pd...

_flag 12y ago
Summary of the paper for those who don't want to read it: So basically there are two categories of "learning" involved in this sort of research, supervised and unsupervised. In supervised learning, someone gives the computer a long list of concepts and their attributes ("frog", "green frog", "jumping frog") and a set of pictures to go with each item, and feeds them into a visual-recognition algorithm. In unsupervised learning, the computer is given a concept like "frog" but then has to discover all the variations itself and get its own visual data to match. The claim in this paper is that they have made the unsupervised learning as strong as the supervised learning. That is, they give the computer a concept ("frog"), it goes and searches through Google Books for common variations ("green frog", "jumping frog") and then uses Google image search to fetch images for each of those queries. They can then remove the obvious false positives (they test to see which images seem to screw up their learning algorithm and leave those out), and the result they get is on par with the supervised learning methods. ---------------------- In my opinion, this is only mildly interesting because Google Image Search functions based on human input anyway -- Google knows the difference between a "frog" and a "jumping frog" or even a "camel" simply because people on the internet caption such images and Google can make associations between images and their captions. Essentially, what the researchers have managed to do is outsource the work of some grad student to millions of people around the world through Google. Of course, it could be argued that there is some sort of parallel with what humans actually do (we know what things are called because we hear other people call them that), but even if I didn't know the name of an animal I could still tell you when the same animal is in different pictures, and I can also tell you when it's jumping and what colour it is. I don't need to have someone caption the image for me to understand the broad range of situations to which the caption "jump" applies.
- xerophtye 12y ago
  >I don't need to have someone caption the image for me to understand the broad range of situations to which the caption "jump" applies. I wonder if this has anything to do with the fact that we can jump too. That we can translate the frog's position into something we do as well. of course one can argue that we can do the same for non-anthro-moprhic things as well. What i think is that we dont directly relate pictures, as the software is taught. What we do is translate that 2D picture into something we'd see in the 3d world. And that 3d "vision" isn't just another image. It represents an object in our world. something that has shape, existence etc. something which we can observe from other senses as well. For us a picture doesn't always represent an abstract thing, an arbitary pattern of colours. It usually represents something concrete. Something about which we have tons of other pieces of knowledge as well. So we relate pictures by checking if they map to the same real-world object. And here that "object" is a sort of nexus of many pieces of information we have on it which is a product of many direct and indirect human experiences. So i don't really think that we are in a position to teach a computer to do anything like that.

cyborgx7 12y ago

The problem with all approaches to machine learning I see today is that they only focus on grouping and separating concepts based on certain characteristics. They seem to be all fundamentally statistical. None of them seem to work towards a fundamental understanding of what the concepts mean. I'm not sure how that could be accomplished though. Is there even a meaningful distinction to be made between beeing able to identify a concept and understanding what a concept actually means?

pavpanchekha 12y ago
This is in fact a classic criticism of modern AI. Classic approaches tried to replicate human reasoning mechanisms. The work was very cool, but far harder to understand and scale, compared to modern statistical algorithms. To some extent, you are faced with a choice: a small palette of mediocre techniques whose workings are beautiful theories of the human mind; or, a broadly-applicable, incredibly practical set of tools with little motivation or spirit.
- PurplePanda 12y ago
  I understand that other being hard to scale in a machine efficiency sense, the old techniques were also rather limited in applicable domain. They were often based on unjustified models of whatever the author decided was a good model of thought/reasoning, whether "frames", predicate calculus, constraint propagation or whatever. It seems to me that although you say statistical methods have little motivation, the motivation of the alternative - heuristics - is very questionable. Can anyone correct me on this?
  - pavpanchekha 12y ago
    That's actually what I meant by scale—that they were slow isn't really a problem if you're interested in AI for the raw fun of it. While justifications for using logic or constraints or probability theory were often lacking from classical techniques, do understand that logic itself was originally developed as a crystallization of proper human thought. And many of the classic AI folks did care deeply about understanding how humans achieved certain results. For example, take a look at Marvin Minsky's work. To the contrary, modern techniques aren't interested in replicating human thought; they are simply interested in replicating human results.

cedias 12y ago

(Disclaimer: I am not an native english speaker) The title of the article and of the algorithm (learn everything about anything) is a bit misleading. You might believe that it learns everything, period. Actually it's more about finding every variation of a "concept", I quote their website: "a fully automated method that given any concept, e.g., horse, discovers an exhaustive vocabulary for it that explains all variations (i.e., actions, interactions, attributes, etc) that modify its appearance."

Shizka 12y ago
I wonder. If they have an exhaustive vocabulary would it be possible to generate a picture of what the system believes an object to look like? I know that there is something called generative models in machine learning and my guess is that it could be applied here.
- cedias 12y ago
  Well you'd have to build a probabilistic model for each concepts, whether on pixels or on features, and you could use it to generate images randomly. It might show up some good shapes.
- Houshalter 12y ago
  It's possible, but generally generative models have to be trained in a specific way. If not, you could do something like for every layer of the neural net, you train another NN which can "predict" the layer below it, it's input. Then you can work your way down each layer to try to find an input which would produce that output. Another way is to use some kind of optimization to find an input which produces that pattern (e.g. backprop to the pixels themselves.) This will give you the image that most strongly triggers that output. Not necessarily a typical example.

sparky_z 12y ago

This strongly reminds me of the first chapter of Greg Egan's Diaspora [1] (which I can't recommend enough) in which newly-formed AIs bootstrap their way to consciousness in part by connecting randomly to an online library and using the various data streams to build up an associative model of the world. [1] http://gregegan.customer.netspace.net.au/DIASPORA/01/Orphanogenesis.html http://gregegan.customer.netspace.net.au/DIASPORA/01/Orphano...

resdirector 12y ago
Seconded. Diaspora is one of the best sci if books I've read. Highly recommend it (and all of Egan's work) to the HN community.
- wfn 12y ago
  Thirded. Before Diaspora, I first read "Wang's Carpets"[1] which is a short story of his. Then found out this story had later been incorporated as a chapter into the book. I remember basically immediately ordering said book that night. fwiw, that "Webly-Supervised Visual Concept Learning" reminds me of the stuff that Hinton et al. do re: unsupervised (concept, etc.) learning (using restricted Boltzmann machines, and so on.) Good talk on the subject (of deep learning, etc.): https://www.youtube.com/watch?v=AyzOUbkUf3M https://www.youtube.com/watch?v=AyzOUbkUf3M [1]: read online here: http://bookre.org/reader?file=222997 http://bookre.org/reader?file=222997
  - MachineElf 12y ago
    Umm.. fourthed? I just couldn't help but jump in and also recommend Greg Egan's "Permutation City". That book is just wonderful... think simulation, cellular automata as a model for computation, artificial life and all that other good stuff :). Also, about the LEVAN thing... given the amount of data available online, both in various structured formats and unstructured formats, don't be surprised if deep learning will yield better and better results moving forward. To me though, they mostly seem evolutionary rather than revolutionary. I mean if you look back at the AI field, during the days before the "AI winter" came, huge amounts of data is one thing researchers back then didn't have available. This is not to say that there haven't been advances in learning algorithms at all recently. ..

deleted 12y ago

[deleted]

motyar 12y ago

Where is the code? is that opensource? and which programming language it uses?

cedias 12y ago
You have everything here: http://levan.cs.washington.edu/?state=show_about http://levan.cs.washington.edu/?state=show_about I quote the readme: > This is an implementation of the "Learning Everything about Anything" system. The system is implemented in MATLAB, with various helper functions written in Shell, Python, MEX C++ for efficiency reasons. For details about the method, please see [1]. This readme contains instructions on using the code, as well as accessing/using already trained models for various concepts. For questions concerning the code please contact Santosh Divvala (http://homes.cs.washington.edu/~santosh http://homes.cs.washington.edu/~santosh) at santosh@cs.washington.edu. The software has been tested on Linux using MATLAB versions R2011a. There may be compatibility issues with older versions of MATLAB. At least 4GB of memory (plus an additional 0.75GB for each parallel matlab worker) is assumed.

nawitus 12y ago

AIXI-mc can also learn everything about anything. And you can download the source code online[1]. 1. http://jveness.info/software/default.html http://jveness.info/software/default.html

ZeroFries 12y ago

This doesn't really help the symbol grounding problem: it uses pre-human-sorted data (who use their own ability to match symbols and meaning) to form its associative network. So, it's using human consciousness as part of the input to form its own consciousness. You could argue that humans use other humans' consciousness to develop its own, but now you have the infinite regress which seem to be the fate of all symbol grounding contemplations. Surely there has to be a starting point, some "axioms" you initially accept about the world to start the process. Maybe these are embedded in our DNA and have evolved to be a practical start (eg: sharp sensory input from nerves on your skin is automatically linked with pain, which we automatically avoid).

clubhi 12y ago

// learn everything wget google.com?search=random_string() >> everything.txt

deleted 12y ago

[deleted]

graycat 12y ago

If we define 'learning' in an appropriate way, then any good research library already knows "everything about anything". Okay, let the thing 'learn' about the Kuhn-Tucker conditions by searching on Google and reading, say, Wikipedia or some books at Google or Amazon. Then have the thing show that for problems in functional form the Zangwill and Kuhn-Tucker constraint qualifications are independent. Do that and I will start to believe that the terminology 'deep learning' is appropriate. I'm not holding my breath. Yes, it may be that in some rough sense the kind of 'learning' it is doing is roughly like some of the learning of a child of, say, 2 as it is starting to learn about language and things. Yes, it may be that such 'learning' is a significant part of the intelligence of, say, a child of 3-5. Maybe. Big, huge maybe. When I was working in AI, I noticed the terminology had been cooked up to imply much more than was being accomplished. Now, as I understand it, there is a specific definition for the current AI term 'deep learning' and has to do with the 'depth' of where adjust parameters in a neural network, not how 'deep' the 'learning' is about the subject in question. Cute terminology.

dang 12y ago

This is such a dumb title. Can any of you suggest a better one?

bra-ket 12y ago

I'm surprised NELL learner is not cited as it's closely related: http://rtw.ml.cmu.edu/rtw/ http://rtw.ml.cmu.edu/rtw/