7 ms·

Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly

Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly disconnected from machine and humanity merging.

Smerity 14y ago
I was going to make the same request. The singularity should be discussed where relevant, not added to everything. This paper is producing high level features from noisy data in an unsupervised fashion -- a human still needs to indicate the task it should be targeted for and a human still needs to provide labelled training data for these high level features to be of use. This work is interesting enough to warrant detailed discussion on the topic at hand, large scale machine learning, rather than just rehashing discussions of the singularity. Added: As I can't reply to the comment below I'll do it here =] The network provides learned representations that are discriminative. The aim of the network is to learn high level features representative of the content. One of the many features it produced was one which accurately indicated the presence of a face in the image. Note that they said train a face detector and not classify. For example, from the same network there was a feature which accurate detected cats yet they didn't explicitly train a cat detector either (see the section "Cat and human body detectors"). As the network represents the content as generic features it is clear that, if it reaches a high enough level, those features are essentially classifications themselves. tldr; High-level features generated by this unsupervised network are so high-level that one of them aligns with "has a face in the image", others with "has cat in image", etc, but these features cannot be used without labelled training.
- reitzensteinm 14y ago
  Actually, what's significant about this work is that labeled training data was not required: "Contrary to what appears to be a widely-held intuition, our experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not."
  - Smerity 14y ago
    I replied by adding to my comment above as it wouldn't allow me to reply earlier. Reference that. tldr; High-level features generated by this unsupervised network are so high-level that one of them aligns with "has a face in the image", another to "has cat in image" (see the section "Cat and human body detectors") and so on. Note however that they select the "best neuron" for face classification -- the only way they can do that is via using labelled data and testing all the neurons (where each neuron's activation is a feature). Thus, these features cannot be used without labelled training.
    - reitzensteinm 14y ago
      But the difference is that you can show it 1 billion unclassified images, then show it 1000 images you know to be faces, analyzing how its neurons respond to the known inputs to use it to classify the rest of the images. Strictly speaking, you do need to have some labeled data at the end in order to determine how the neural net views faces, but I think that obscures what's notable about this system. The amount of human participation involved in training is potentially six or more orders of magnitude less. That's a breakthrough, and a change in kind, not just degree.
      - Smerity 14y ago
        In a more general response: I don't think what I stated obscures what's notable about the system, I feel I stated exactly what was notable and specifically avoided overstating it. Overhyping when it comes to machine learning and AI seems to be the norm and has already hurt AI/ML severely in the past[1]. More specifically: I didn't disagree with anything you've stated, simply pointed out that labeled training data is necessary in response to the statement that it wasn't.The high-level feature extraction the paper discusses is unsupervised but the classifiers it produces are semi-supervised. It's an important distinction. [1]: http://en.wikipedia.org/wiki/AI_winter http://en.wikipedia.org/wiki/AI_winter
        sellsword 14y ago
        Having a bachelor's emphasis in AI, I think you described it perfectly. I was wondering too from their abstract how they were recognizing "faces" entirely without labels, this makes it clearer. As you said, unsupervised they can find extremely high-level categories. That is pretty impressive.
marshallp 14y ago
It does this for 20,000 different objects categories - this is getting close to matching human visual ability (and there are huge societal implications if computer reach that standard). This is the most powerful AI experiment yet conducted (publicly known).
- frisco 14y ago
  > This is the most powerful AI experiment yet conducted (publicly known). That's an ill-defined statement. AI is a vast and diverse field: what makes one demonstration more "powerful" than another? There are definitely other projects that could be viewed as being in the same class of "powerful" as this cluster. This is certainly an interesting paper, but it has to be viewed in the context of a large and active field.
  - marshallp 14y ago
    That's true. Let's say in machine learning then.
    - JohnHaugeland 14y ago
      Let's not, because it's still wrong on the order of thirty counterexamples. Let's say we'll stop making broad proclamations about the global best in a field we know very little about.
      - robrenaud 14y ago
        Andrew Ng has set many state of the art results on various data sets using similar approaches as the one described in the paper. Here is a reasonably approachable talk he gave about it. http://www.youtube.com/watch?v=ZmNOAtZIgIk http://www.youtube.com/watch?v=ZmNOAtZIgIk
        JohnHaugeland 14y ago
        This is a very interesting talk. Thank you for sharing with me. :) In return, I will offer you two interesting non-sequiturs, because I don't have anything topical and a non-sequitur seems like it's worth half what something germane would be. . Bret Victor, "Inventing on Principle." First 5 minutes are terribly boring. Give him a chance; it's 100% worth it. http://vimeo.com/36579366 http://vimeo.com/36579366 . Damian Conway, "Temporally Quaquaversal Virtual Nanomachine." It's as funny as it sounds. http://yow.eventer.com/events/1004/talks/1028 http://yow.eventer.com/events/1004/talks/1028
marshallp 14y ago
I put the singularity bit in to make it relevant for those who are non-technical. This experiment is significant because it shows that large artificial neural networks can be made to work. People have tried and failed at this for decades. This technigue was "discovered" by geoff hinton at the university of toronto in 2005. However, nobody at tried (or maybe got enough funds) to try it this scale. If this continues to work at larger and larger scale, this would be a machine learning technique that can work accurately on tasks that are hugely important to society - accurate speech recognition - human level compuer vision (make human manual labor redundant)
- Chlorus 14y ago
  "I put the singularity bit in to make it relevant for those who are non-technical." Yeah, I'm sure there's a lot of those on HN...I'd expect this kind of crap in something like Wired, but not here.
  - nik61 14y ago
    This is a Troll response, and gratuitously insulting. HN readers may be broadly technical (or not, who knows?) but few of them have the insight into AI provided here, especially if they got their IT experience or education during the AI Winter referenced above.
- iuguy 14y ago
  Even so, the singularity bit is editorializing a link to a white paper on an equally significant scale. Nowhere in the link is the singularity referenced. As for the point about it being for non-technical people, I don't understand where you're coming from. This is hacker news. If people don't understand it and don't upvote it, then that's their problem, not yours.
- coldarchon 14y ago
  You have absolutely no idea I guess. Let's have a look at the human brain. 1.000.000.000 neuron cells in the brain which can have 200.000 connections with 2 different states. The number of different states of the brain exceeds the number of atoms in the universe, we are speaking of 2 to the power of 200.000.000.000.000 compared to 10 to the power of 82. I guess you can calculate this, right? And you compare this to 16.000 cores?
jackfoxy 14y ago
Normally I advocate adherence to posting the original article title on HN, but if that had been the case I doubt this article would have ever got enough attention to be upvoted. Singularity is near is over the top.
Havoc 14y ago
Agreed. My first reaction was "You're half a dozen zeroes short there chief".

Smerity 14y ago

I was going to make the same request. The singularity should be discussed where relevant, not added to everything. This paper is producing high level features from noisy data in an unsupervised fashion -- a human still needs to indicate the task it should be targeted for and a human still needs to provide labelled training data for these high level features to be of use. This work is interesting enough to warrant detailed discussion on the topic at hand, large scale machine learning, rather than just rehashing discussions of the singularity. Added: As I can't reply to the comment below I'll do it here =] The network provides learned representations that are discriminative. The aim of the network is to learn high level features representative of the content. One of the many features it produced was one which accurately indicated the presence of a face in the image. Note that they said train a face detector and not classify. For example, from the same network there was a feature which accurate detected cats yet they didn't explicitly train a cat detector either (see the section "Cat and human body detectors"). As the network represents the content as generic features it is clear that, if it reaches a high enough level, those features are essentially classifications themselves. tldr; High-level features generated by this unsupervised network are so high-level that one of them aligns with "has a face in the image", others with "has cat in image", etc, but these features cannot be used without labelled training.

reitzensteinm 14y ago
Actually, what's significant about this work is that labeled training data was not required: "Contrary to what appears to be a widely-held intuition, our experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not."
- Smerity 14y ago
  I replied by adding to my comment above as it wouldn't allow me to reply earlier. Reference that. tldr; High-level features generated by this unsupervised network are so high-level that one of them aligns with "has a face in the image", another to "has cat in image" (see the section "Cat and human body detectors") and so on. Note however that they select the "best neuron" for face classification -- the only way they can do that is via using labelled data and testing all the neurons (where each neuron's activation is a feature). Thus, these features cannot be used without labelled training.
  - reitzensteinm 14y ago
    But the difference is that you can show it 1 billion unclassified images, then show it 1000 images you know to be faces, analyzing how its neurons respond to the known inputs to use it to classify the rest of the images. Strictly speaking, you do need to have some labeled data at the end in order to determine how the neural net views faces, but I think that obscures what's notable about this system. The amount of human participation involved in training is potentially six or more orders of magnitude less. That's a breakthrough, and a change in kind, not just degree.
    - Smerity 14y ago
      In a more general response: I don't think what I stated obscures what's notable about the system, I feel I stated exactly what was notable and specifically avoided overstating it. Overhyping when it comes to machine learning and AI seems to be the norm and has already hurt AI/ML severely in the past[1]. More specifically: I didn't disagree with anything you've stated, simply pointed out that labeled training data is necessary in response to the statement that it wasn't.The high-level feature extraction the paper discusses is unsupervised but the classifiers it produces are semi-supervised. It's an important distinction. [1]: http://en.wikipedia.org/wiki/AI_winter http://en.wikipedia.org/wiki/AI_winter
      - sellsword 14y ago
        Having a bachelor's emphasis in AI, I think you described it perfectly. I was wondering too from their abstract how they were recognizing "faces" entirely without labels, this makes it clearer. As you said, unsupervised they can find extremely high-level categories. That is pretty impressive.

marshallp 14y ago

It does this for 20,000 different objects categories - this is getting close to matching human visual ability (and there are huge societal implications if computer reach that standard). This is the most powerful AI experiment yet conducted (publicly known).

frisco 14y ago
> This is the most powerful AI experiment yet conducted (publicly known). That's an ill-defined statement. AI is a vast and diverse field: what makes one demonstration more "powerful" than another? There are definitely other projects that could be viewed as being in the same class of "powerful" as this cluster. This is certainly an interesting paper, but it has to be viewed in the context of a large and active field.
- marshallp 14y ago
  That's true. Let's say in machine learning then.
  - JohnHaugeland 14y ago
    Let's not, because it's still wrong on the order of thirty counterexamples. Let's say we'll stop making broad proclamations about the global best in a field we know very little about.
    - robrenaud 14y ago
      Andrew Ng has set many state of the art results on various data sets using similar approaches as the one described in the paper. Here is a reasonably approachable talk he gave about it. http://www.youtube.com/watch?v=ZmNOAtZIgIk http://www.youtube.com/watch?v=ZmNOAtZIgIk
      - JohnHaugeland 14y ago
        This is a very interesting talk. Thank you for sharing with me. :) In return, I will offer you two interesting non-sequiturs, because I don't have anything topical and a non-sequitur seems like it's worth half what something germane would be. . Bret Victor, "Inventing on Principle." First 5 minutes are terribly boring. Give him a chance; it's 100% worth it. http://vimeo.com/36579366 http://vimeo.com/36579366 . Damian Conway, "Temporally Quaquaversal Virtual Nanomachine." It's as funny as it sounds. http://yow.eventer.com/events/1004/talks/1028 http://yow.eventer.com/events/1004/talks/1028

marshallp 14y ago

I put the singularity bit in to make it relevant for those who are non-technical. This experiment is significant because it shows that large artificial neural networks can be made to work. People have tried and failed at this for decades. This technigue was "discovered" by geoff hinton at the university of toronto in 2005. However, nobody at tried (or maybe got enough funds) to try it this scale. If this continues to work at larger and larger scale, this would be a machine learning technique that can work accurately on tasks that are hugely important to society - accurate speech recognition - human level compuer vision (make human manual labor redundant)

Chlorus 14y ago
"I put the singularity bit in to make it relevant for those who are non-technical." Yeah, I'm sure there's a lot of those on HN...I'd expect this kind of crap in something like Wired, but not here.
- nik61 14y ago
  This is a Troll response, and gratuitously insulting. HN readers may be broadly technical (or not, who knows?) but few of them have the insight into AI provided here, especially if they got their IT experience or education during the AI Winter referenced above.
iuguy 14y ago
Even so, the singularity bit is editorializing a link to a white paper on an equally significant scale. Nowhere in the link is the singularity referenced. As for the point about it being for non-technical people, I don't understand where you're coming from. This is hacker news. If people don't understand it and don't upvote it, then that's their problem, not yours.
coldarchon 14y ago
You have absolutely no idea I guess. Let's have a look at the human brain. 1.000.000.000 neuron cells in the brain which can have 200.000 connections with 2 different states. The number of different states of the brain exceeds the number of atoms in the universe, we are speaking of 2 to the power of 200.000.000.000.000 compared to 10 to the power of 82. I guess you can calculate this, right? And you compare this to 16.000 cores?

jackfoxy 14y ago

Normally I advocate adherence to posting the original article title on HN, but if that had been the case I doubt this article would have ever got enough attention to be upvoted. Singularity is near is over the top.

Havoc 14y ago

Agreed. My first reaction was "You're half a dozen zeroes short there chief".