10 ms·

A Revolutionary Technique That Changed Machine Vision

175 points by lelf 12y ago

pavlov 12y ago
It's interesting that, in the image showing the classes easiest and hardest for the algorithm, all the easy ones are animals and all the difficult ones are human-made artifacts. Does nature tend to create forms which are easily detected by relatively simple neural networks? The evolutionary explanation could be that this allows animals with primitive neural systems to more easily distinguish other members of their species.
- Houshalter 12y ago
  Much more training time was spent on animals (10% of the dataset is literally just pictures of dogs) and much more variation among man made objects.
- bane 12y ago
  > The evolutionary explanation could be that this allows animals with primitive neural systems to more easily distinguish other members of their species. It's probably the other way around. Neural systems optimize for natural objects.
  - pavlov 12y ago
    But isn't the neural network (like the algorithm described in the article) a mathematical concept? Are the computer-based neural networks so closely modeled after biological equivalents that they would inherit such an optimization? (As you can tell, I know nearly nothing about neural networks, but curious to learn more...)
    - bane 12y ago
      I have no idea. But I'd wager that cause and effect in natural systems has more to do with visual systems adapting to shapes than shapes of entire animals adapting to other species' visual systems.
    - deleted 12y ago
      [deleted]
      - kastnerkyle 12y ago
        Digital circuits? I mean, it is just some matrix multiplies and a nonlinearity (then stack to the moon). No "circuitry" is really involved until you get into recurrent networks, and even then that is just feedback. Not quite sure what you mean here. There have been experiments trying to encode information the way the brain does, they just haven't worked very well (or at least not as well). There is equivalency with PCA (well ZCA, a modified form of PCA) in cat and monkey brains, and likely others as well. See Sejnowski and Bell in [1] Also, PCA is an affine transform, so there is no reason it couldn't be incorporated/learned by the net itself. In fact, I think most nets these days eschew PCA/ZCA when they have sufficient data support. To clarify for others, this type of neural network has nothing to do with the brain. A neural network is really a "universal function approximator", and I actually prefer to call it as such. Our goal is to learn the best possible mapping of input -> label, through whatever means necessary. It turns out that learning hierarchies of features helps from both a learning aspect and a computational point of view. But a sufficiently wide single layer could do the same thing in theory. [1] http://www.ncbi.nlm.nih.gov/pubmed/9425547 http://www.ncbi.nlm.nih.gov/pubmed/9425547
    - michaelochurch 12y ago
      But isn't the neural network (like the algorithm described in the article) a mathematical concept? It is. It's a family of models that (a) can be expressed compactly using linear algebra and (b) can represent a large class of mathematical functions. (In fact, the family of neural nets can learn all continuous functions.) "Learning" is typically some variety of gradient descent (on the "error surface" defined by the training set) in the space of parameters. Are the computer-based neural networks so closely modeled after biological equivalents that they would inherit such an optimization? In my opinion, no. Biological neural networks are, in fact, a lot more complicated. They have time behavior, changing network topologies, and chemical influences (neurotransmitters) that play a major role. There's still a lot we don't know about them. I disagree with GP's contention. While the biological neural network was an inspiration for this class of mathematical models, and convolutional behavior (in ANNs, a regularization, or mechanism for reducing the dimensionality of the parameter space, sacrificing training-set performance but often improving test-set performance) may be used in our visual cortex, but artificial neural nets are quite different and, mathematically, most varieties are quite simple.
- ma2rten 12y ago
  I think it's because these human made object have no single form. For instance "letteropener" describes the function of the object, not the form. Compare that to "red fox" which is always gonna look more or less the same.
  - kastnerkyle 12y ago
    Yes, but the network is still differentiating between select breeds which means it is learning traits of the animal which are unique to the breed. And training at a higher level is perfectly doable, say "fox" instead of the fox breed. Ultimately, if you are able to look at an image and say "letter opener" there are features which differentiate this from a knife/can opener/whatever - these are the exact things a convolutional neural network should be able to use (in theory) and has nothing to do with the label, which is typically unimportant as long as it is unique and accurate. We could flip all the labels around and still get unique answers - the network is just learning a mapping from input -> some integer, and I would argue the variance in dog breeds and lighting in natural scenes is much trickier than the angle/shape of a letter opener. I still think it comes down to the composition of this particular dataset. Augmenting this with images scraped from online stores would be very interesting as it is fairly trivial to get huge numbers of images for anything that is typically sold online - I think Google is way ahead on this one!
    - bunderbunder 12y ago
      It's impossible to tell from the examples given in the article, but I wouldn't be surprised if the same classifier that gets 100% on "Blenheim Spaniel" and "Flat-coated Retriever" gets less than 100% on "Dog". It's a question of how visually coherent the category you're trying to learn is. From a purely visual perspective, the first two categories are relatively tightly bunched in the state space, whereas "dog" covers a diffuse cloud of appearances whose total range might even encompass the area where many non-dog animals also lie. Humans may rely on some additional semantic knowledge about different kinds of animal to produce an accurate classification. It's not entirely unlike how determining the meaning of the words in the phrase "eats shoots and leaves" can't be done reliably without incorporating contextual clues such as whether we were just talking about pandas or a murder in a restaurant. There may also be issues around how distinct the categories are from each other. A couple years ago yours truly picked up a letter opener off the table and used it to spread butter on his toast, much to the amusement of his hosts.
- theoh 12y ago
  I'd say it's an entropy thing. The principle of "correlation of parts" means images of animals have low entropy relative to, say, machines. Intuitively a machine can contain an almost arbitrary set of shapes and components, whereas animal forms are more constrained.
  - kastnerkyle 12y ago
    See my earlier answer - I think that it is all about the data. And most things humans design have a similar "correlation of parts" due to the human preference for symmetry. Not all the images are face on headshots - lots of running/action, off centered, etc. ImageNet is a very "real" dataset in that sense. Entropy isn't the right term here - low entropy would imply something about the compressibility of each image based only on whether it was a machine or a natural object, which I don't think is case.
    - theoh 12y ago
      Well, if the effect is just experimental error, I agree that no explanation is needed. I was using entropy in the information theoretic sense, as you might assign a measure of entropy in bits to each character in a language. If part of an animal is "less surprising" I'd say it contributed less to the entropy of the whole thing. Maybe that's too woolly. There is a difference between complex mechanical and organic objects, though. See the universal machine tools illustrated on this page: http://www.lathes.co.uk/adcock%26shipleycombination/ http://www.lathes.co.uk/adcock%26shipleycombination/ There is no way you could identify the whole machine based on seeing a small part if an identical part recurs in many different machines. It's not holographic in the way animals are.
- deleted 12y ago
  [deleted]
- possibilistic 12y ago
  > Does nature tend to create forms which > are easily detected by relatively simple > neural networks? Poison markings and mimicry, for sure. Furthermore, large swaths of plant life in various spectra. reply
  - kastnerkyle 12y ago
    Chicken, egg. Maybe these things are easy because they are crucial to survival (and everything that was bad at recognizing these signs died), not due to being easier for simple visual processes.
    - bunderbunder 12y ago
      Both chicken and egg at the same time. Evolution's going to favor the solution that involves the least costly adaptations. That probably means simpler markings will be favored because it's less costly for the prey species to evolve them and it's less costly for the predator species to evolve an instinct to avoid them.
- kastnerkyle 12y ago
  It could also be that there are more labeled examples of animal, or that the translation/rotation of an animal is less disruptive than than the same transform to a car or other object. I know for sure that ImageNet has a huge amount of animals in it, down to very select sub-breeds, while the object categories are usually at higher levels.
deleted 12y ago
[deleted]
oftenwrong 12y ago
Here is the challenge site: http://image-net.org/challenges/LSVRC/ http://image-net.org/challenges/LSVRC/ The datasets can be found by clicking the links for each year's challenge.
drpgq 12y ago
Not to denigrate the techniques used here, but it is interesting as a computer vision researcher (face recognition in my case) how important good labelled training and testing sets are. Some of my successes over the years have come more from figuring out where to get good data than good computer vision techniques.
- trhway 12y ago
  good education does matter :)
- kastnerkyle 12y ago
  Not only large datasets, but the elasticity of a community to revisit old ideas from a new angle. Computer vision has been very good in this regard overall, though there were some dark times (https://www.facebook.com/yann.lecun/posts/10152034328862143 https://www.facebook.com/yann.lecun/posts/10152034328862143).
- tzs 12y ago
  Can you ever generate artifical training and test data? For instance, suppose I would like to be able to take a photo of a chess game and turn that into a diagram of the position. I have no idea where I would get natural photos of thousands or millions of chess games to use for training and testing a chess piece identifying vision system. Could I instead make 3D models of common chess set designs, and then generate and render photorealistic images of chess positions to use for training and test data for the vision system?
  - ertdfgcb 12y ago
    Probably, but it might not be able to work very well with different conditions, like a picture of a chessboard in central park vs a chessboard in a library. This could probably be solved with more renders, and even if it can't, the renders would probably be a good starting point
  - Houshalter 12y ago
    You still need some natural data, otherwise the network will probably overfit on regularities in your CG engine that don't exist in the real world. It might do that anyway, even with natural data.
graycat 12y ago
What I see in the OP, via my Web browser Firefox 27.0.1, is just an ad I can't make go away in the middle of the screen. So, without some special effort, say, grab and parse the HTML, I can't read the content. Anyone else have this issue?
- inetsee 12y ago
  I have Firefox 32 on Kubuntu 14.04, and I had no problem clicking on the close button on the ad.
  - graycat 12y ago
    Thanks. I couldn't see a button for closing the window. Apparently somehow asking for higher magnification of the Firefox window didn't give higher magnification of the ad window; thus I didn't see the close button. Thanks. With your feedback, I tried again and did see the big X close button outside of the main part of the ad window. The thing did close. Read the article. Cute. Thanks.
    - ars 12y ago
      Try ESC next time, it works on many of those.
kastnerkyle 12y ago
I think it is key to remember that the accessibility of huge amounts of labeled data is behind most of this. ImageNet is 1.2TB (~1.2 million images!). Convolutional neural nets have been around for a long, long time (early 90s? or maybe late 80s) and they just needed more data (and a few training tricks :) ). As we enter the world of "big data" more and more companies are locking this level/size (or bigger) data away in closed vaults. This is a competitive edge in business, but also strangles the ability for academics to do "real world" research, which is often a huge criticism of academia at large. Academic research can have a role in tech R&D if given a chance, so I hope that companies will open their data to private researchers to continue this kind of improvement - even if under secrecy clauses or some other safeguard. All that said, the technological improvements have been astounding, and I hope this is only the beginning! There have been some incredible results using related techniques in NLP for machine translation recently, and speech has always been a great playpen. If anyone is interested in playing with these things in Python, myself and another researcher have recently created a library for using these type of neural networks as a black-box WITHOUT ANY TRAINING!, in a similar style to scikit-learn. See [1] for more. We support only a few nets currently, but plan to integrate support for caffe and pylearn2 in the near future. [1] http://sklearn-theano.github.io/ http://sklearn-theano.github.io/
- agibsonccc 12y ago
  Great work guys! Are these going to only be computer vision nets? I think the huge problem with most deep learning frameworks out there are they are either hard to use or limited in scope. There's nothing wrong with that per se, but I'd be curious to see what you guys intend. It's great that you guys are giving an sklearn interface to it. Edit: I should add my criticisms come from a biased perspective: I write http://deeplearning4j.org/ http://deeplearning4j.org/ I love comparing notes with others regardless. Either way: Wish you the best of luck with it.
  - kastnerkyle 12y ago
    Support is planned for audio and hopefully text - I am working on building a million song dataset to recreate the work Sander Dieleman did for Spotify, and have had some possible support in October for the weights of a trained speech network! So yes, feature extraction from other domains should be on the horizon. We are specifically trying to make it easy to say: I want to transform an image (or audio, etc.) using a pretrained net. Download the weights, extract the features for me, and give me the feature vectors so I can do something with those. This seems to be really, really, really hard in all the tools I have used and usually involves training yourself, which is not very useful for things on the level of ImageNet. Of course, having nice examples and good docs is one of the great parts of scikit-learn, and is actually one of the things I have been working on most recently. Our docs aren't to that level yet, but I hope they can be one day. DeCAF (precursor to Caffe) and OverFeat binaries were really kind of the first in this regard (about 1 year ago now), but IMO one of the limitations is that they lose interaction with the rest of the Python ecosystem for data munging, and simple algorithms for exploration. By wrapping the weights, we hope to leverage the support of the Python ML ecosystem easily, while still being able to use the power of these networks. Right now the most compelling use case is as part of a scikit-learn pipeline i.e. make_pipeline(OverfeatTransformer, LinearSVC) or whatever. Feed images in, get predictions out. I am also working on a demo of "writing your own twitter bot" similar to https://twitter.com/id_birds https://twitter.com/id_birds , written by Daniel Nouri . I like sloths, so it will definitely be a slothbot. I also hope to support recurrent architectures from Groundhog (https://github.com/lisa-groundhog/GroundHog https://github.com/lisa-groundhog/GroundHog) as several researchers here at the LISA lab have been using it to get pretty amazing results in NLP and audio, both of which are potential targets in the future. If we can leverage their work, it would be a very nice way for people to immediately play with SOTA architectures in different applications. In any case, just loading in weights and extracting image features easily is nice, and was a benefit for both Michael and myself in a research project this summer.
a_e_k 12y ago
Can any machine vision researchers recommend a survey/overview paper or two that would make a good technical introduction to these particular techniques?
- kastnerkyle 12y ago
  I gave a talk at EuroScipy 2014, in the back part there are a ton of references and resources to get started. See [1] [1] https://speakerdeck.com/kastnerkyle/euroscipy2014 https://speakerdeck.com/kastnerkyle/euroscipy2014
m_ke 12y ago
If you want to see a convolutional net in action checkout out the demo on http://clarifai.com/ http://clarifai.com/
AndrewOMartin 12y ago
There are two irresponsible sentences in the article, "In other words, it is not going to be long before machines significantly outperform humans in image recognition tasks." and "Or put another way, It is only a matter of time before your smartphone is better at recognizing the content of your pictures than you are.". The irresponsibility is in seeing the existence of a technique to solve a more complex version of a toy problem (e.g. find the location in this photo that exactly matches pattern x), and inferring that the same technique, given more power, will exhibit super-human behaviour. It's a reasonable claim that, in the task as described, the technique outperforms humans but that's about as exciting a claim as saying how much quicker the latest supercomputer is at arithmetic than a human. The point being that human object recognition isn't about labelling a scene with nouns, but somehow instinctively knowing the relevant objects for a situation and, if required, the appropriate situation-specific noun. I therefore request the final sentence of the article be rewritten as "Or put another way, it may only be a matter of time, funding and motivation before your smartphone (with equivalent computing resources of the likes of Google Inc. and University of Tokyo) can ascribe one or more nouns from a set of size of order 1000 to regions of a photo, given that huge amounts of pre-processing and man hours have been dedicated to the pre-processing of that exact set of nouns to create a training set, and that you take the photo in similar lighting conditions as the training set, don't apply any filters, and that the objects referred to by the appropriate nouns is neither small nor thin, better than you.".
- blauwbilgorgel 12y ago
  It is just extrapolating the error rate reduction over the last few years. Spam filters have become better than moderators in labeling spam only in the last decade or so. When computers first started to become faster than mathematicians this was really a breakthrough. The same is happening now with object and speech recognition. The computer succesfully completes a task. That it is not how humans intuitively approach these same tasks is irrelevant for this accomplishment. What if the results were only half as good, but the system behaved more like humans, who does this satisfy? The state-of-the-art is capable of detecting far more than 1000 objects, does not need labeled data, is robust to changes in light and does not care about the camera used. No preprocessing the data needed, features are automatically generated (preprocessing the target labels is a bit silly BTW). So yes, in the very near future, algorithms will be better security guards than well... security guards.

175 points by lelf 12y ago

It's interesting that, in the image showing the classes easiest and hardest for the algorithm, all the easy ones are animals and all the difficult ones are human-made artifacts. Does nature tend to create forms which are easily detected by relatively simple neural networks? The evolutionary explanation could be that this allows animals with primitive neural systems to more easily distinguish other members of their species.

Houshalter 12y ago
Much more training time was spent on animals (10% of the dataset is literally just pictures of dogs) and much more variation among man made objects.
bane 12y ago
> The evolutionary explanation could be that this allows animals with primitive neural systems to more easily distinguish other members of their species. It's probably the other way around. Neural systems optimize for natural objects.
- pavlov 12y ago
  But isn't the neural network (like the algorithm described in the article) a mathematical concept? Are the computer-based neural networks so closely modeled after biological equivalents that they would inherit such an optimization? (As you can tell, I know nearly nothing about neural networks, but curious to learn more...)
  - bane 12y ago
    I have no idea. But I'd wager that cause and effect in natural systems has more to do with visual systems adapting to shapes than shapes of entire animals adapting to other species' visual systems.
  - deleted 12y ago
    [deleted]
    - kastnerkyle 12y ago
      Digital circuits? I mean, it is just some matrix multiplies and a nonlinearity (then stack to the moon). No "circuitry" is really involved until you get into recurrent networks, and even then that is just feedback. Not quite sure what you mean here. There have been experiments trying to encode information the way the brain does, they just haven't worked very well (or at least not as well). There is equivalency with PCA (well ZCA, a modified form of PCA) in cat and monkey brains, and likely others as well. See Sejnowski and Bell in [1] Also, PCA is an affine transform, so there is no reason it couldn't be incorporated/learned by the net itself. In fact, I think most nets these days eschew PCA/ZCA when they have sufficient data support. To clarify for others, this type of neural network has nothing to do with the brain. A neural network is really a "universal function approximator", and I actually prefer to call it as such. Our goal is to learn the best possible mapping of input -> label, through whatever means necessary. It turns out that learning hierarchies of features helps from both a learning aspect and a computational point of view. But a sufficiently wide single layer could do the same thing in theory. [1] http://www.ncbi.nlm.nih.gov/pubmed/9425547 http://www.ncbi.nlm.nih.gov/pubmed/9425547
  - michaelochurch 12y ago
    But isn't the neural network (like the algorithm described in the article) a mathematical concept? It is. It's a family of models that (a) can be expressed compactly using linear algebra and (b) can represent a large class of mathematical functions. (In fact, the family of neural nets can learn all continuous functions.) "Learning" is typically some variety of gradient descent (on the "error surface" defined by the training set) in the space of parameters. Are the computer-based neural networks so closely modeled after biological equivalents that they would inherit such an optimization? In my opinion, no. Biological neural networks are, in fact, a lot more complicated. They have time behavior, changing network topologies, and chemical influences (neurotransmitters) that play a major role. There's still a lot we don't know about them. I disagree with GP's contention. While the biological neural network was an inspiration for this class of mathematical models, and convolutional behavior (in ANNs, a regularization, or mechanism for reducing the dimensionality of the parameter space, sacrificing training-set performance but often improving test-set performance) may be used in our visual cortex, but artificial neural nets are quite different and, mathematically, most varieties are quite simple.
ma2rten 12y ago
I think it's because these human made object have no single form. For instance "letteropener" describes the function of the object, not the form. Compare that to "red fox" which is always gonna look more or less the same.
- kastnerkyle 12y ago
  Yes, but the network is still differentiating between select breeds which means it is learning traits of the animal which are unique to the breed. And training at a higher level is perfectly doable, say "fox" instead of the fox breed. Ultimately, if you are able to look at an image and say "letter opener" there are features which differentiate this from a knife/can opener/whatever - these are the exact things a convolutional neural network should be able to use (in theory) and has nothing to do with the label, which is typically unimportant as long as it is unique and accurate. We could flip all the labels around and still get unique answers - the network is just learning a mapping from input -> some integer, and I would argue the variance in dog breeds and lighting in natural scenes is much trickier than the angle/shape of a letter opener. I still think it comes down to the composition of this particular dataset. Augmenting this with images scraped from online stores would be very interesting as it is fairly trivial to get huge numbers of images for anything that is typically sold online - I think Google is way ahead on this one!
  - bunderbunder 12y ago
    It's impossible to tell from the examples given in the article, but I wouldn't be surprised if the same classifier that gets 100% on "Blenheim Spaniel" and "Flat-coated Retriever" gets less than 100% on "Dog". It's a question of how visually coherent the category you're trying to learn is. From a purely visual perspective, the first two categories are relatively tightly bunched in the state space, whereas "dog" covers a diffuse cloud of appearances whose total range might even encompass the area where many non-dog animals also lie. Humans may rely on some additional semantic knowledge about different kinds of animal to produce an accurate classification. It's not entirely unlike how determining the meaning of the words in the phrase "eats shoots and leaves" can't be done reliably without incorporating contextual clues such as whether we were just talking about pandas or a murder in a restaurant. There may also be issues around how distinct the categories are from each other. A couple years ago yours truly picked up a letter opener off the table and used it to spread butter on his toast, much to the amusement of his hosts.
theoh 12y ago
I'd say it's an entropy thing. The principle of "correlation of parts" means images of animals have low entropy relative to, say, machines. Intuitively a machine can contain an almost arbitrary set of shapes and components, whereas animal forms are more constrained.
- kastnerkyle 12y ago
  See my earlier answer - I think that it is all about the data. And most things humans design have a similar "correlation of parts" due to the human preference for symmetry. Not all the images are face on headshots - lots of running/action, off centered, etc. ImageNet is a very "real" dataset in that sense. Entropy isn't the right term here - low entropy would imply something about the compressibility of each image based only on whether it was a machine or a natural object, which I don't think is case.
  - theoh 12y ago
    Well, if the effect is just experimental error, I agree that no explanation is needed. I was using entropy in the information theoretic sense, as you might assign a measure of entropy in bits to each character in a language. If part of an animal is "less surprising" I'd say it contributed less to the entropy of the whole thing. Maybe that's too woolly. There is a difference between complex mechanical and organic objects, though. See the universal machine tools illustrated on this page: http://www.lathes.co.uk/adcock%26shipleycombination/ http://www.lathes.co.uk/adcock%26shipleycombination/ There is no way you could identify the whole machine based on seeing a small part if an identical part recurs in many different machines. It's not holographic in the way animals are.
deleted 12y ago
[deleted]
possibilistic 12y ago
> Does nature tend to create forms which > are easily detected by relatively simple > neural networks? Poison markings and mimicry, for sure. Furthermore, large swaths of plant life in various spectra. reply
- kastnerkyle 12y ago
  Chicken, egg. Maybe these things are easy because they are crucial to survival (and everything that was bad at recognizing these signs died), not due to being easier for simple visual processes.
  - bunderbunder 12y ago
    Both chicken and egg at the same time. Evolution's going to favor the solution that involves the least costly adaptations. That probably means simpler markings will be favored because it's less costly for the prey species to evolve them and it's less costly for the predator species to evolve an instinct to avoid them.
kastnerkyle 12y ago
It could also be that there are more labeled examples of animal, or that the translation/rotation of an animal is less disruptive than than the same transform to a car or other object. I know for sure that ImageNet has a huge amount of animals in it, down to very select sub-breeds, while the object categories are usually at higher levels.

deleted 12y ago

[deleted]

oftenwrong 12y ago

Here is the challenge site: http://image-net.org/challenges/LSVRC/ http://image-net.org/challenges/LSVRC/ The datasets can be found by clicking the links for each year's challenge.

drpgq 12y ago

Not to denigrate the techniques used here, but it is interesting as a computer vision researcher (face recognition in my case) how important good labelled training and testing sets are. Some of my successes over the years have come more from figuring out where to get good data than good computer vision techniques.

trhway 12y ago
good education does matter :)
kastnerkyle 12y ago
Not only large datasets, but the elasticity of a community to revisit old ideas from a new angle. Computer vision has been very good in this regard overall, though there were some dark times (https://www.facebook.com/yann.lecun/posts/10152034328862143 https://www.facebook.com/yann.lecun/posts/10152034328862143).
tzs 12y ago
Can you ever generate artifical training and test data? For instance, suppose I would like to be able to take a photo of a chess game and turn that into a diagram of the position. I have no idea where I would get natural photos of thousands or millions of chess games to use for training and testing a chess piece identifying vision system. Could I instead make 3D models of common chess set designs, and then generate and render photorealistic images of chess positions to use for training and test data for the vision system?
- ertdfgcb 12y ago
  Probably, but it might not be able to work very well with different conditions, like a picture of a chessboard in central park vs a chessboard in a library. This could probably be solved with more renders, and even if it can't, the renders would probably be a good starting point
- Houshalter 12y ago
  You still need some natural data, otherwise the network will probably overfit on regularities in your CG engine that don't exist in the real world. It might do that anyway, even with natural data.

graycat 12y ago

What I see in the OP, via my Web browser Firefox 27.0.1, is just an ad I can't make go away in the middle of the screen. So, without some special effort, say, grab and parse the HTML, I can't read the content. Anyone else have this issue?

inetsee 12y ago
I have Firefox 32 on Kubuntu 14.04, and I had no problem clicking on the close button on the ad.
- graycat 12y ago
  Thanks. I couldn't see a button for closing the window. Apparently somehow asking for higher magnification of the Firefox window didn't give higher magnification of the ad window; thus I didn't see the close button. Thanks. With your feedback, I tried again and did see the big X close button outside of the main part of the ad window. The thing did close. Read the article. Cute. Thanks.
  - ars 12y ago
    Try ESC next time, it works on many of those.

kastnerkyle 12y ago

I think it is key to remember that the accessibility of huge amounts of labeled data is behind most of this. ImageNet is 1.2TB (~1.2 million images!). Convolutional neural nets have been around for a long, long time (early 90s? or maybe late 80s) and they just needed more data (and a few training tricks :) ). As we enter the world of "big data" more and more companies are locking this level/size (or bigger) data away in closed vaults. This is a competitive edge in business, but also strangles the ability for academics to do "real world" research, which is often a huge criticism of academia at large. Academic research can have a role in tech R&D if given a chance, so I hope that companies will open their data to private researchers to continue this kind of improvement - even if under secrecy clauses or some other safeguard. All that said, the technological improvements have been astounding, and I hope this is only the beginning! There have been some incredible results using related techniques in NLP for machine translation recently, and speech has always been a great playpen. If anyone is interested in playing with these things in Python, myself and another researcher have recently created a library for using these type of neural networks as a black-box WITHOUT ANY TRAINING!, in a similar style to scikit-learn. See [1] for more. We support only a few nets currently, but plan to integrate support for caffe and pylearn2 in the near future. [1] http://sklearn-theano.github.io/ http://sklearn-theano.github.io/

agibsonccc 12y ago
Great work guys! Are these going to only be computer vision nets? I think the huge problem with most deep learning frameworks out there are they are either hard to use or limited in scope. There's nothing wrong with that per se, but I'd be curious to see what you guys intend. It's great that you guys are giving an sklearn interface to it. Edit: I should add my criticisms come from a biased perspective: I write http://deeplearning4j.org/ http://deeplearning4j.org/ I love comparing notes with others regardless. Either way: Wish you the best of luck with it.
- kastnerkyle 12y ago
  Support is planned for audio and hopefully text - I am working on building a million song dataset to recreate the work Sander Dieleman did for Spotify, and have had some possible support in October for the weights of a trained speech network! So yes, feature extraction from other domains should be on the horizon. We are specifically trying to make it easy to say: I want to transform an image (or audio, etc.) using a pretrained net. Download the weights, extract the features for me, and give me the feature vectors so I can do something with those. This seems to be really, really, really hard in all the tools I have used and usually involves training yourself, which is not very useful for things on the level of ImageNet. Of course, having nice examples and good docs is one of the great parts of scikit-learn, and is actually one of the things I have been working on most recently. Our docs aren't to that level yet, but I hope they can be one day. DeCAF (precursor to Caffe) and OverFeat binaries were really kind of the first in this regard (about 1 year ago now), but IMO one of the limitations is that they lose interaction with the rest of the Python ecosystem for data munging, and simple algorithms for exploration. By wrapping the weights, we hope to leverage the support of the Python ML ecosystem easily, while still being able to use the power of these networks. Right now the most compelling use case is as part of a scikit-learn pipeline i.e. make_pipeline(OverfeatTransformer, LinearSVC) or whatever. Feed images in, get predictions out. I am also working on a demo of "writing your own twitter bot" similar to https://twitter.com/id_birds https://twitter.com/id_birds , written by Daniel Nouri . I like sloths, so it will definitely be a slothbot. I also hope to support recurrent architectures from Groundhog (https://github.com/lisa-groundhog/GroundHog https://github.com/lisa-groundhog/GroundHog) as several researchers here at the LISA lab have been using it to get pretty amazing results in NLP and audio, both of which are potential targets in the future. If we can leverage their work, it would be a very nice way for people to immediately play with SOTA architectures in different applications. In any case, just loading in weights and extracting image features easily is nice, and was a benefit for both Michael and myself in a research project this summer.

a_e_k 12y ago

Can any machine vision researchers recommend a survey/overview paper or two that would make a good technical introduction to these particular techniques?

kastnerkyle 12y ago
I gave a talk at EuroScipy 2014, in the back part there are a ton of references and resources to get started. See [1] [1] https://speakerdeck.com/kastnerkyle/euroscipy2014 https://speakerdeck.com/kastnerkyle/euroscipy2014

m_ke 12y ago

If you want to see a convolutional net in action checkout out the demo on http://clarifai.com/ http://clarifai.com/

AndrewOMartin 12y ago

There are two irresponsible sentences in the article, "In other words, it is not going to be long before machines significantly outperform humans in image recognition tasks." and "Or put another way, It is only a matter of time before your smartphone is better at recognizing the content of your pictures than you are.". The irresponsibility is in seeing the existence of a technique to solve a more complex version of a toy problem (e.g. find the location in this photo that exactly matches pattern x), and inferring that the same technique, given more power, will exhibit super-human behaviour. It's a reasonable claim that, in the task as described, the technique outperforms humans but that's about as exciting a claim as saying how much quicker the latest supercomputer is at arithmetic than a human. The point being that human object recognition isn't about labelling a scene with nouns, but somehow instinctively knowing the relevant objects for a situation and, if required, the appropriate situation-specific noun. I therefore request the final sentence of the article be rewritten as "Or put another way, it may only be a matter of time, funding and motivation before your smartphone (with equivalent computing resources of the likes of Google Inc. and University of Tokyo) can ascribe one or more nouns from a set of size of order 1000 to regions of a photo, given that huge amounts of pre-processing and man hours have been dedicated to the pre-processing of that exact set of nouns to create a training set, and that you take the photo in similar lighting conditions as the training set, don't apply any filters, and that the objects referred to by the appropriate nouns is neither small nor thin, better than you.".

blauwbilgorgel 12y ago
It is just extrapolating the error rate reduction over the last few years. Spam filters have become better than moderators in labeling spam only in the last decade or so. When computers first started to become faster than mathematicians this was really a breakthrough. The same is happening now with object and speech recognition. The computer succesfully completes a task. That it is not how humans intuitively approach these same tasks is irrelevant for this accomplishment. What if the results were only half as good, but the system behaved more like humans, who does this satisfy? The state-of-the-art is capable of detecting far more than 1000 objects, does not need labeled data, is robust to changes in light and does not care about the camera used. No preprocessing the data needed, features are automatically generated (preprocessing the target labels is a bit silly BTW). So yes, in the very near future, algorithms will be better security guards than well... security guards.