5 ms·
When I was an AI grad student back in the late 80's and early 90's logic was all the rage. People published papers about things like non-monotonic logics, inte
by lisper 8y ago
When I was an AI grad student back in the late 80's and early 90's logic was all the rage. People published papers about things like non-monotonic logics, intentional vs extensional logics, higher-order logics, the frame problem, circumscription... One thing that always bothered me about the field was that the whole premise behind using logic as the basis for AI seemed to me to be obviously flawed because neither the inputs to nor the outputs from my brain were logical propositions, they were images and sounds on the input side, and sounds and gestures on the output side. Even my internal reality didn't seem to consist of logical propositions. When I went to open the refrigerator to make a sandwich, I didn't feel like I needed to prove any theorems in order to do it. And I think subsequent developments have vindicated my skepticism.
Well, I have a similar objection to neural networks. Human learning is fundamentally qualitatively different from contemporary neural network learning. It is simply not the case that I learn what a cat looks like by being shown a bunch of images of cats and having some meta-input that says "label:cat". Instead, I see a cat -- and a whole bunch of other things, including the environment that the cat happens to be in -- and I hear some sounds, and I somehow manage to make sense of the whole melange and sort out what a cat is and what the sound of my mother's voice is when she says, "That's a cat!, and the fact that the "cat" part of the audio stream is a label, and that it stands for this furry purry thing moving around in front of me, and that by producing some outputs that cause me to produce the same sounds from my own body I get the reward of hearing mommy say, "That's right!"
What we call neural networks today are interesting and useful, but I don't think they are even remotely close to anything interesting that the human brain does, and so quibbling over whether or not they are "essentially polynomial regression" or not is, IMHO, missing the point.
[EDIT: I had closed with "missing the point rather badly" but subsequent comments have convinced me that this extra emphasis was unwarranted.]
- mettamage 8y agoI disagree, fundamentally understanding what neural nets are from any angle might be useful to debug them better. I do agree with that learning happens differently in the human brain. Your comment points out that humans sub-consciously cluster a lot of things, and do a lot of other things at the same time (e.g. you also mention reinforcement training) in order to make sense of the world. If only children between 0 and 3 would be self-aware enough of what they're going through, would've made this question so much more easy to answer. I wonder whether their form of learning and our form of learning is the same, I don't think so (pruning and all that).
- LaserToy 8y agoI would say that we were too eager to jump on "Learning" term. It implies that those things are somehow close to us, humans. In reality NN don't learn anything, it is just a optimization problem on the simplified model. And all the issue with them just proves that we are mostly brute-forcing it.
- dsamarin 8y agoWhile neural networks are responsible for recent breakthroughs in problems like computer vision, machine translation and time series prediction, they can also combine with reinforcement learning algorithms. Reinforcement learning refers to goal-oriented algorithms, which learn how to attain a complex objective (goal) or maximize along a particular dimension over many steps; for example, maximize the points won in a game over many moves. They can start from a blank slate, and under the right conditions they achieve superhuman performance. Like a child reinforced by its mother, these algorithms are penalized when they make the wrong decisions and rewarded when they make the right ones – this is reinforcement. While that may sound trivial, it's a vast improvement over their previous accomplishments, and the state of the art is progressing rapidly. Reinforcement learning solves the difficult problem of correlating immediate actions with the delayed returns they produce. Like humans, reinforcement learning algorithms sometimes have to wait a while to see the fruit of their decisions. They operate in a delayed return environment, where it can be difficult to understand which action leads to which outcome over many time steps. Reinforcement learning algorithms can be expected to perform better and better in more ambiguous, real-life environments while choosing from an arbitrary number of possible actions, rather than from the limited options of a video game. That is, with time we expect them to be valuable to achieve goals in the real world.
- gambler 8y ago> One thing that always bothered me about the field was that the whole premise behind using logic as the basis for AI seemed to me to be obviously flawed because neither the inputs to nor the outputs from my brain were logical propositions Well, calculus is a bunch of logic propositions and it explains physics pretty well. So clearly some good things can be achieved by formal logic alone. Decision trees are very much "logic" and they work pretty well. Logic-based AI is pretty cool and would be great for fixing a lot of current problems in software engineering. Too bad it's mostly abandoned.
- whatshisface 8y ago>Well, calculus is a bunch of logic propositions and it explains physics pretty well. So clearly some good things can be achieved by formal logic alone. The idea to use logical rules to infer things about nature was arrived at through the non-logical "gestures and senses" methods that the parent was talking about. If all you had was logic you'd have no way to choose your axioms, or even your rules of inference. The axioms and rules of inference were, ultimately, selected from the vast sea of alternative options based on our observations, which rest on our senses.
- gambler 8y agoMy point is that abandoning simple algorithms just because they don't (seem to) lead to AGI is a mistake.
- lisper 8y ago> calculus is a bunch of logic propositions and it explains physics pretty well That's the wrong level of abstraction for AI.
- ivalm 8y ago> Well, calculus is a bunch of logic propositions and it explains physics pretty well. I mean, it is true that you can construct a Turing machine using predicate logic, and thus all computer algorithms can be expressed in this way, but what matters is the relevant level of abstraction, and thinking about things using predicate logic is not always the best.
- JulianMorrison 8y agoThe essence of this is that humans aren't blank slates. We have 500 million years of ancestors helping us tune our functioning in a world that contains large moving self willed discrete lifeforms that are pertinent to our survival.
- breck 8y ago> It is simply not the case that I learn what a cat looks like by being shown a bunch of images of cats and having some meta-input that says "label:cat". How do you know this? It seems to me this is likely how it works. If you spend an hour playing with a cat, and you look at the cat with your eyes at 60hz, you see about 5 million labeled augmented images of a cat.
- lisper 8y agoWhere does the meta-input enter my brain? How do I distinguish the meta-input from the training data?
- grumdan 8y agoPossibly when we learn that a cat is called a cat. Without that, we may still learn to distinguish a cat from a dog, but I guess the label would be an internal one.
- dnautics 8y agothat's not how it works. Presuming you don't already know chinese, I can do something like show you a new chinese character that you've never seen, once, and you will be able to pick it out of a group of ten random chinese characters. Almost every time.
- breck 8y agoBut how long do you show it to me for? If you show it to me for 20 seconds, you've actually shown it to me ~1,000 times.
- dnautics 8y agoOk. Presuming your example is correct (it's not, neural impulses travel at roughly half the speed of sound) I could describe a Chinese character for you and you could identify it, so that's zero shot learning. Even so, if you've ever actually trained a neural networks you would know what happens when you feed the same data (n) times
- cr0sh 8y agoWhat you're kinda describing is something that the "godfather" (if you will) of Deep Learning, Geoffrey Hinton, also has suspicions about. https://www.quora.com/Why-is-Geoffrey-Hinton-suspicious-of-backpropagation-and-wants-AI-to-start-over https://www.quora.com/Why-is-Geoffrey-Hinton-suspicious-of-b... Basically, we have yet (from what I know - I am not a researcher, and this field moves fast) to find any form or method by which biological neural networks (aka brains) perform "backpropagation" - ie, neurons seem to be feedforward only systems. Furthermore, we (people) don't need thousands or millions of labeled examples in order to learn something. Now, I know someone else replied to you about how you see millions of cat images based on some flawed reasoning that the eyes are seeing at 60 Hz - which is untrue, because it's a continuous analog system - plus there's the thing of neurons being spiking potential systems (vs what common ANNs model); but the idea is intriguing that we do see much more than one or two "images". But there are other situations where we learn things that don't require this at all, and there aren't such vast amounts of "labeled input" to learn a concept. These two issues are problems from mainly a conceptual point; they obviously don't stop current ANNs from functioning or being useful. But they do call into question how our brains are actually doing things, and especially doing this with such low amounts of power (vs the huge amounts of power required for deep learning systems, for what would be basic tasks for humans). Another wrench that may come into play: I recently learned (I think it was from an HN post) that biologists and others are coming around to the idea that genes don't work like we once thought they actually did, and that cells - internally and externally - have some kind of communications system and other "stuff" going on that makes the whole system much more complex than the simplification most people learn about. I tend to wonder how such new information and realizations and "mechanics" will play into how we understand how neurons actually work and communicate biologically - whether any of that plays into "how brains learn". There's long been questions about chemical pathways and other things that aren't captured by common (and even uncommon) ANN models. Then there's also been questions as to what roles other cells in the brain that aren't neurons (glial cells) in the process, if any.
- jayd16 8y agoSeems to be that logic is about deduction and NN is about inference. Human are capable of both and I think both are required to achieve true ai.
- Isinlor 8y agoNeural Networks seem to be capable of both too. Although I'm just guessing what do you mean precisely by deduction. Would you agree that solving Sudoku requires deduction? "Finally, we show how recurrent relational networks can learn to solve Sudoku puzzles from supervised training data, a challenging task requiring upwards of 64 steps of relational reasoning. We achieve state-of-the-art results amongst comparable methods by solving 96.6% of the hardest Sudoku puzzles." https://arxiv.org/pdf/1711.08028.pdf https://arxiv.org/pdf/1711.08028.pdf "We present NeuroSAT, a message passing neural network that learns to solve SAT problems after only being trained as a classifier to predict satisfiability. Although it is not competitive with state-of-the-art SAT solvers, NeuroSAT can solve problems that are substantially larger and more difficult than it ever saw during training by simply running for more iterations. Moreover, NeuroSAT generalizes to novel distributions; after training only on random SAT problems, at test time it can solve SAT problems encoding graph coloring, clique detection, dominating set, and vertex cover problems, all on a range of distributions over small random graphs." https://arxiv.org/abs/1802.03685 https://arxiv.org/abs/1802.03685