5 ms·

On the one hand I am an experienced software engineer with a computer science degree. On the other hand I was never interested in AI and I basically know "nothi

On the one hand I am an experienced software engineer with a computer science degree. On the other hand I was never interested in AI and I basically know "nothing" about AI, computer vision and artificial neural networks. AI always seemed too far off and bothered my engineering spidey sense for intractable problems. So I may embarrass myself with the following... The more software engineering experience I get the more neural networks bother me. I have not studied them in detail... and yet, they seem like a crappy solution. Throw a lot resources at something, add a bit of math and push a pile of data through. Does that ever lead to result that can't be bested though simpler more traditional methods? For example, the math behind how the network is organized and learns, that same math can be used to write algorithms which reveal patterns in data. Patterns which can then be studied further. Then there is the doing of things. What ever pattern is responsible for doing something in a network, the same function can be described much more simply by an algorithm. It just seems like neural networks would be interesting only if an alien spaceship with stupefying computing resources fell on us. And it fell on us in such a way that we could fully utilize the power of this new super computer while at the same time we could not learn anything new about algorithms or anything else. Only then does it seem like "train the neural network" is a smart strategy. In every other scenario is just seems like a decent if somewhat lazy start to an investigation. Really just a way to kick the tires of a problem. So did I just completely embarrass myself with my lack understanding of ANNs?

kevin1024 14y ago
Check out this (awesome) talk: http://www.youtube.com/watch?v=ZmNOAtZIgIk http://www.youtube.com/watch?v=ZmNOAtZIgIk There are a number of areas in which unsupervised feature learning outperforms the hand-programmed models we have been fine-tuning for decades.
entropy_ 14y ago
I don't understand why this got down-voted. Having read up about AI and looked into things like neural networks I can definitely tell that he's wrong. But surely the questions he asks are pertinent to the discussion, aren't they? For someone foreign to the field they actually make a lot of sense and I would be very hard pressed to tell him why he's wrong given that I only have very little knowledge myself. Surely whoever thought this post was so wrong as to merit down-voting could explain why that is and answer the man's questions?
- adgar 14y ago
  People are dicks on the Internet.
lmkg 14y ago
If you try to design an algorithm, what you get is a model of how you think your system behaves. If you train a neural network, what you get is a model of how your system actually behaves. In other words, a neural network does not embed any assumptions or preconceived notions of how the system ought to work. Intuitively, not incorporating knowledge of how the worlds works ought to be a disadvantage, but in practice it turns out to be not just an advantage, but an embarrassingly large one. Neural Networks aren't the only common approach in modern AI research, but the other techniques (e.g. Support Vector Machines) have similar properties: they're dumb methods of finding correlations in data. The field of AI is slowly coming to grips with the fact that more training data trumps better algorithms. That said, I sympathize with your frustrations. You can solve a lot of problems by throwing more cycles at it, but you don't learn anything, either about the algorithms or about your target domain. For example, a human plays the game of Go differently than a machine. A human has an intuition of what moves are good and what are bad, and only considers a handful; a machine relies less on evaluating board position and more on brute-forcing through possible board configurations. As machines get more powerful, eventually the machines catch up to humans in ability, with that same approach. While their methods will become impressive in their efficacy, we learn nothing about how humans are so much better at evaluating board position and selecting moves to speculate on.
- skarayan 14y ago
  Sensing is easier to model out than intuition. Intuition requires the self component because it comes from feelings and past experiences. The current algorithms are important too, but they do not model the self.
- sanxiyn 14y ago
  What are you doing? I am training a randomly wired neural net to play Tic-Tac-Toe. Why is the net wired randomly? I do not want it to have any preconceptions of how to play. Minsky then shut his eyes. Why do you close your eyes? Sussman asked his teacher. So that the room will be empty. At that moment, Sussman was enlightened. -- Jargon File
- fghh45sdfhr3 14y ago
  If you try to design an algorithm, what you get is a model of how you think your system behaves Actually if you rigorously examine the same data you are feeding your network with well applied statistical methods, that is not true. But in my experience statistic are actually some of the hardest math, and not many people, other than statisticians, are very good at using them correctly.
Retric 14y ago
Let's say you want to build a robot that walks around like a person or even a goat. There are basically two part's of that problem, building the hardware, and building the software. If you use neural networks for the software then it's going to take more resources to run, and take a while to train. However, while people may be able to write more efficient software that requires fewer resources developing that software is going to take longer and cost a lot more money. http://www.youtube.com/watch?v=cNZPRsrwumQ http://www.youtube.com/watch?v=cNZPRsrwumQ The idea that I can create software which works even if I don't understand why it works is a powerful idea. The point of AI as a field is stepping back and saying: "Computers are fast and cheap, people are slow and expensive, can't we automate this?" and continuing to do that not just to secretary's, or even coders, but also designers. As long as you can build a large enough training set and toss enough computing resources you can basically solve any problem. Granted, the need for large training set's and processing power can also be a huge limitations. But, in the end neural nets / AI is just another tool like your computer, Algebra, or a big stick. The it's a cool idea and solves useful problems is enough for people to spend a lot of time and money looking into it.
- fghh45sdfhr3 14y ago
  Granted, the need for large training set's and processing power can also be a huge limitations. I think that is exactly what bothers me about it. It's that everything you say would be 100% correct IF we had far, far, far more computing resources at our disposal. But since we don't, humans solving a problem actually does not take longer or cost more than training a neural net. Not if you want the neural net to be as good as the solution humans come up with. Only if you are satisfied with a very quick and dirty approximation can NNs be faster and cheaper than humans.
euccastro 14y ago
What ever pattern is responsible for doing something in a network, the same function can be described much more simply by an algorithm. Yes, but for that you'd have to find the algorithm, which takes lots of human resources. That's what unsupervised learning saves you. In several specialized fields like computer vision and aural interpretation, a generic neural network trained for hours can outperform algorithms that took decades of research to refine.

kevin1024 14y ago

Check out this (awesome) talk: http://www.youtube.com/watch?v=ZmNOAtZIgIk http://www.youtube.com/watch?v=ZmNOAtZIgIk There are a number of areas in which unsupervised feature learning outperforms the hand-programmed models we have been fine-tuning for decades.

entropy_ 14y ago

I don't understand why this got down-voted. Having read up about AI and looked into things like neural networks I can definitely tell that he's wrong. But surely the questions he asks are pertinent to the discussion, aren't they? For someone foreign to the field they actually make a lot of sense and I would be very hard pressed to tell him why he's wrong given that I only have very little knowledge myself. Surely whoever thought this post was so wrong as to merit down-voting could explain why that is and answer the man's questions?

adgar 14y ago
People are dicks on the Internet.

lmkg 14y ago

If you try to design an algorithm, what you get is a model of how you think your system behaves. If you train a neural network, what you get is a model of how your system actually behaves. In other words, a neural network does not embed any assumptions or preconceived notions of how the system ought to work. Intuitively, not incorporating knowledge of how the worlds works ought to be a disadvantage, but in practice it turns out to be not just an advantage, but an embarrassingly large one. Neural Networks aren't the only common approach in modern AI research, but the other techniques (e.g. Support Vector Machines) have similar properties: they're dumb methods of finding correlations in data. The field of AI is slowly coming to grips with the fact that more training data trumps better algorithms. That said, I sympathize with your frustrations. You can solve a lot of problems by throwing more cycles at it, but you don't learn anything, either about the algorithms or about your target domain. For example, a human plays the game of Go differently than a machine. A human has an intuition of what moves are good and what are bad, and only considers a handful; a machine relies less on evaluating board position and more on brute-forcing through possible board configurations. As machines get more powerful, eventually the machines catch up to humans in ability, with that same approach. While their methods will become impressive in their efficacy, we learn nothing about how humans are so much better at evaluating board position and selecting moves to speculate on.

skarayan 14y ago
Sensing is easier to model out than intuition. Intuition requires the self component because it comes from feelings and past experiences. The current algorithms are important too, but they do not model the self.
sanxiyn 14y ago
What are you doing? I am training a randomly wired neural net to play Tic-Tac-Toe. Why is the net wired randomly? I do not want it to have any preconceptions of how to play. Minsky then shut his eyes. Why do you close your eyes? Sussman asked his teacher. So that the room will be empty. At that moment, Sussman was enlightened. -- Jargon File
fghh45sdfhr3 14y ago
If you try to design an algorithm, what you get is a model of how you think your system behaves Actually if you rigorously examine the same data you are feeding your network with well applied statistical methods, that is not true. But in my experience statistic are actually some of the hardest math, and not many people, other than statisticians, are very good at using them correctly.

Retric 14y ago

Let's say you want to build a robot that walks around like a person or even a goat. There are basically two part's of that problem, building the hardware, and building the software. If you use neural networks for the software then it's going to take more resources to run, and take a while to train. However, while people may be able to write more efficient software that requires fewer resources developing that software is going to take longer and cost a lot more money. http://www.youtube.com/watch?v=cNZPRsrwumQ http://www.youtube.com/watch?v=cNZPRsrwumQ The idea that I can create software which works even if I don't understand why it works is a powerful idea. The point of AI as a field is stepping back and saying: "Computers are fast and cheap, people are slow and expensive, can't we automate this?" and continuing to do that not just to secretary's, or even coders, but also designers. As long as you can build a large enough training set and toss enough computing resources you can basically solve any problem. Granted, the need for large training set's and processing power can also be a huge limitations. But, in the end neural nets / AI is just another tool like your computer, Algebra, or a big stick. The it's a cool idea and solves useful problems is enough for people to spend a lot of time and money looking into it.

fghh45sdfhr3 14y ago
Granted, the need for large training set's and processing power can also be a huge limitations. I think that is exactly what bothers me about it. It's that everything you say would be 100% correct IF we had far, far, far more computing resources at our disposal. But since we don't, humans solving a problem actually does not take longer or cost more than training a neural net. Not if you want the neural net to be as good as the solution humans come up with. Only if you are satisfied with a very quick and dirty approximation can NNs be faster and cheaper than humans.

euccastro 14y ago

What ever pattern is responsible for doing something in a network, the same function can be described much more simply by an algorithm. Yes, but for that you'd have to find the algorithm, which takes lots of human resources. That's what unsupervised learning saves you. In several specialized fields like computer vision and aural interpretation, a generic neural network trained for hours can outperform algorithms that took decades of research to refine.