3 ms·

Sure there are some pretty visualizations of what images turn on a neuron or whatever. But no one could hand code an algorithm that does the same job that's eve

Sure there are some pretty visualizations of what images turn on a neuron or whatever. But no one could hand code an algorithm that does the same job that's even half as good. >And that's false too. Hand designed programs to do machine vision can get significant fractions of the accuracy of NNs and sometimes even perform better and/or faster depending on the domain. Source? A few years ago people thought it was amazing an algorithm could recognize cats 12% of the time. And even the best non-neural approaches use some form of machine learning. >a NN might clue into features that you weren't aware even existed. That's the whole point...

jacquesm 9y ago
Your statements are nonsense. Yes, we do have a good idea of what goes on inside a NN trained on images and Yes hand designed programs exist and work and sometimes work very well. It's just that NNs are rapidly moving ahead and maybe some time in the future both your statements will be true but today they are not. Recognizing cats is not the sum total of machine vision problems and those 'pretty visualizations' go a lot further than what you give them credit for. I've been doing nothing but reading up on machine learning and working through piles of example code and models for the last couple of months and I think based on what I've seen so far that we are definitely headed in the direction that you suggest. At the same time we aren't there yet and such hype does not help. Keep in mind that the domain of machine vision is extremely broad and much wider than Kaggle competitions and that saying 'we have no idea' when clearly we do tries to sketch NNs as magic and they're not. If anything the present generation is best characterized as an optimization of what you could have done for many years, the only reason they are the way they are is because we'd like to have our results in a reasonable time. That makes applying NNs to large datasets possible and that in turn is what helps to solve some of these problems.
- Houshalter 9y ago
  >Yes, we do have a good idea of what goes on inside a NN trained on images I'd love a source for this. The last time I checked I didn't find much. Sure we know the first few layers do stuff like edge detection. But the higher layers are much harder. And even if you know images are activating them, writing code that can accomplish the same task is much harder. >It's just that NNs are rapidly moving ahead and maybe some time in the future both your statements will be true but today they are not. The best NNs have already surpassed human accuracy at imagenet. That's insane! No one expected that result a few years ago. Yes object recognition isn't the only task in machine vision, but it's pretty representative.
  - jacquesm 9y ago
    > I'd love a source for this. I will do some digging for you tomorrow, it's 3:30 am and I really should get some sleep. > Sure we know the first few layers do stuff like edge detection. Yes, basic primitives. > But the higher layers are much harder. Well, those higher layers are built up out of the primitives the lower layers contain. So you get things that are conceptually easier to understand rather than harder. > And even if you know images are activating them, writing code that can accomplish the same task is much harder. That I completely agree with, I've basically gone that route in the last couple of months. > The best NNs have already surpassed human accuracy at imagenet. Yep. It's extremely impressive. It's a watershed moment in the history of computing and half of us are still asleep or in denial. Which is fine by me. But please don't hype. It isn't magic and it isn't perfect - yet. edit: ok, found two of the things I've looked at: https://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma https://stats.stackexchange.com/questions/114385/what-is-the... https://stats.stackexchange.com/questions/146413/why-convolutional-neural-networks-belong-to-deep-learning https://stats.stackexchange.com/questions/146413/why-convolu... And this is interesting code: https://github.com/yosinski/deep-visualization-toolbox https://github.com/yosinski/deep-visualization-toolbox If anything I think that NNs as applied to vision problems are somewhat easier to make sense of than doing the same for NLP or audio. Though it definitely feels like cheating compared to actually programming a solution. An alternative name for 'deep learning' would be 'automatic multi-level feature engineering', but that wouldn't be nearly as sexy.
throwaway373811 9y ago
It's a bit more than 10%. Refer to the submissions to ILSVRC .

jacquesm 9y ago

Your statements are nonsense. Yes, we do have a good idea of what goes on inside a NN trained on images and Yes hand designed programs exist and work and sometimes work very well. It's just that NNs are rapidly moving ahead and maybe some time in the future both your statements will be true but today they are not. Recognizing cats is not the sum total of machine vision problems and those 'pretty visualizations' go a lot further than what you give them credit for. I've been doing nothing but reading up on machine learning and working through piles of example code and models for the last couple of months and I think based on what I've seen so far that we are definitely headed in the direction that you suggest. At the same time we aren't there yet and such hype does not help. Keep in mind that the domain of machine vision is extremely broad and much wider than Kaggle competitions and that saying 'we have no idea' when clearly we do tries to sketch NNs as magic and they're not. If anything the present generation is best characterized as an optimization of what you could have done for many years, the only reason they are the way they are is because we'd like to have our results in a reasonable time. That makes applying NNs to large datasets possible and that in turn is what helps to solve some of these problems.

Houshalter 9y ago
>Yes, we do have a good idea of what goes on inside a NN trained on images I'd love a source for this. The last time I checked I didn't find much. Sure we know the first few layers do stuff like edge detection. But the higher layers are much harder. And even if you know images are activating them, writing code that can accomplish the same task is much harder. >It's just that NNs are rapidly moving ahead and maybe some time in the future both your statements will be true but today they are not. The best NNs have already surpassed human accuracy at imagenet. That's insane! No one expected that result a few years ago. Yes object recognition isn't the only task in machine vision, but it's pretty representative.
- jacquesm 9y ago
  > I'd love a source for this. I will do some digging for you tomorrow, it's 3:30 am and I really should get some sleep. > Sure we know the first few layers do stuff like edge detection. Yes, basic primitives. > But the higher layers are much harder. Well, those higher layers are built up out of the primitives the lower layers contain. So you get things that are conceptually easier to understand rather than harder. > And even if you know images are activating them, writing code that can accomplish the same task is much harder. That I completely agree with, I've basically gone that route in the last couple of months. > The best NNs have already surpassed human accuracy at imagenet. Yep. It's extremely impressive. It's a watershed moment in the history of computing and half of us are still asleep or in denial. Which is fine by me. But please don't hype. It isn't magic and it isn't perfect - yet. edit: ok, found two of the things I've looked at: https://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma https://stats.stackexchange.com/questions/114385/what-is-the... https://stats.stackexchange.com/questions/146413/why-convolutional-neural-networks-belong-to-deep-learning https://stats.stackexchange.com/questions/146413/why-convolu... And this is interesting code: https://github.com/yosinski/deep-visualization-toolbox https://github.com/yosinski/deep-visualization-toolbox If anything I think that NNs as applied to vision problems are somewhat easier to make sense of than doing the same for NLP or audio. Though it definitely feels like cheating compared to actually programming a solution. An alternative name for 'deep learning' would be 'automatic multi-level feature engineering', but that wouldn't be nearly as sexy.

throwaway373811 9y ago

It's a bit more than 10%. Refer to the submissions to ILSVRC .