7 ms·
I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about
by tsimionescu 6y ago
I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world, if only we increase the number of model parameters by a few more orders of magnitude (GPT-3 and the likely strategy for GPT-4).
Of course, all this without any model of how likely it is that the knowledge is embedded in the text, such as trying to train simple models on descriptions of a simple world. I think it's very likely a model couldn't even learn simple arithmetic (addition, subtraction multiplication, division on the rational numbers, let's say) given all of human writing on arithmetic, nevermind being able to reason about the entire world.
- Isinlor 6y agoIt does lead to intelligence and knowledge of the world. It leads to somewhat similar understanding of the world as our understanding of 4D, quantum mechanics or environment close to a black hole. Pretty bad one, mostly based on repeating whatever we have read about in popular science literature without any intuitive understanding. GPT-3 understanding of physical world is similarly bad. It's just simply a modality not available to the model. But GPT-3 is able to do few-shots learning. If you show it couple of times how to do something it will try to repeat after you showing some rudimentary understanding of what you are trying to do. It's even able to learn to do simple analogies [0]. GPT-3 is able to do very, very simple arthritics like addition and subtractions with 2-3 numbers. Actually even a lot smaller models trained on arthritics are able to do impressive feats like symbolic integration and solving differential equations. > Deep Learning for Symbolic Mathematics [1]: Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be surprisingly good at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica. What is true is that GPT-3 is not very good at following instructions. It would not be able to execute an algorithm. So, it would not learn arithmetic from instructions if you would not include explicit examples. [0] https://medium.com/@melaniemitchell.me/can-gpt-3-make-analogies-16436605c446 https://medium.com/@melaniemitchell.me/can-gpt-3-make-analog... [1] https://arxiv.org/abs/1912.01412 https://arxiv.org/abs/1912.01412
- mrfusion 6y agoSo you’re saying ai needs to be grounded to the real world to be effective?
- ccozan 6y agoYes. Specifically the way human brain interacts with environment, the same way any AGI will need to sense and react to an environment, the more real the better. An interesting example would be if the NN learns something from a book, also fed it with the all the critique on that book ( the environment reaction ). So it can possibly learn from a human what is good or bad in that book, etc. Then both AI and humans can speak the same language ( on terms.of "feeling"/sensing ) and eventually learn the proper context, so less parameters are needed.
- mikkom 6y ago> An interesting example would be if the NN learns something from a book, also fed it with the all the critique on that book ( the environment reaction ). But isn't that exactly what has been done with GPT-3? AFAIK it has been trained with both facts (wikipedia and such) and web-crawled text content that contains both content and critique of content (book reviews, movie reviews etc.)?
- ccozan 6y agoI know that, but does the network perceive the critique as environmental reaction? It means if it can distinguish between them. Putting them in the same basket not sure if it helps.
- mellosouls 6y agoIt does lead to intelligence and knowledge of the world. It leads to somewhat similar understanding of the world as our understanding of 4D, quantum mechanics or environment close to a black hole. Pretty bad one [...] GPT-3 understanding of physical world is similarly bad. No, GPT-3 has no "understanding" whatsoever - we at least have a conscious engagement with the topics you cite and a genuine understanding that we lack a directly relatable perspective. Our understanding in many circumstances - like those you mention, may be severely constrained, but it exists as part of a sentience that there is no reason to believe GPT-3 or any other current AI tool - possesses.
- valgaze 6y agoThe parameter/dataset arms race producing emergent intelligence rather than just bigger models is a fantasy. However without a good/better explanation in the meantime , really interesting capabilities emerge at higher parameters Specifically to your point of arithmetic— here is GPT3 performing “zero shot” (ie no examples) arithmetic: https://photos.app.goo.gl/pf6fHcrio7QFtU4S8 https://photos.app.goo.gl/pf6fHcrio7QFtU4S8 The “completion” includes an answer to the problem but also provides a natural language summary (GPT3) GPT3 has ~170billion parameters (GPT2 ~1.5B) & somehow is capable of many-digit arithmetic w/o fine-tuning or task-specific training. (Its training data was “messy” from Common Crawl like for search engines) Microsoft’s Turing-NLG has ~17billion parameters & was successful only in 1 of every 2 attempts of 2-digit arithmetic and only successful less than 10% of the time for higher numbers of digits.
- nl 6y agoThat's pretty impressive. Google's Pegusus summarizer learnt to count to 5: https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html https://ai.googleblog.com/2020/06/pegasus-state-of-art-model...
- esc_colon_q 6y agoIt is very true that GPT-XYZ is fundamentally limited, like all transformer models, but don't misattribute the root cause to the fact that it's just processing text. The real limitation is that these are feedforward networks that just do perception without any processing of what they've perceived. You can try to hide that fact for a while by increasing the depth of the perception network, basically hard-coding some processing into the single pass, but you're still not capturing any of the "absolutely requires self-feedback" behavior that we care about that takes a human more than a split second to do (aka almost all actual thought). A statistical model that made good use of what was coming in could absolutely learn (to take your example) math at every level based on nothing but text, transformers are just nowhere close to having that capability because of their design limitations.
- tsimionescu 6y agoBut the fact that it 'lives' in a world of text is much more fundamental. There is simply not enough information in text to draw a working model of the world. We presuppose such a model in our words and in our communication, so fundamentally any algorithm that is solely trained on text can't learn a model of the world from it.
- hackinthebochs 6y agoIt shouldn't be that hard to imagine. If the objective is to predict human text, at some point the best way to predict text is to capture the kinds of knowledge that humans know and use in text generation. Human knowledge is a strong prior for human text generation, and so it shouldn't be surprising that for a large model its parameters would end up in a portion of parameter space that encodes some human knowledge.
- tsimionescu 6y agoBy this token, the best way to learn Mandarin would be to start looking at text in Mandarin and trying to assign it meaning. While it theoretically can work, it is vastly more difficult than trying to relate it to a previously learned model. In fact, it is so difficult that we have never successfully understood a lost human language this way (though with the caveat that we don't have the huge corpus of text that GPT-3 was trained on in any of the lost languages we tried it on).
- hackinthebochs 6y agoNo, I said the best way to learn to predict human text is to capture human knowledge. I did not say how best to capture human knowledge. Obviously doing gradient descent over a huge text corpus is massively inefficient for a human. Whether its the best way for a machine to learn it is another matter. After all, the amount of data that humans experience through their senses during formative years, as well as the "computational work" that went into evolving human brain architectures surely dwarfs GPT-3's training regime.
- contravariant 6y agoThat said the interesting thing about language AIs is that they are by design able to communicate, which leads to interesting results, even though it is increasingly obvious that they don't really have any 'awareness' as such.
- m0zg 6y agoI don't see why you're "amazed". Neural networks are _provably_ an universal approximator. As such, they can approximate _any_ function to an arbitrary degree of precision. It just might require an impractically large/complex neural net to do so, and you might not be able to train it yet, but it's _provably_ doable in the limit, if you remove size/training constraints. I'm a practitioner in the field, and an AI skeptic because of it. I think perceptual tasks work reasonably well, NLP barely works for anything, and any sort of cognition doesn't work at all, as of August 2020. I will be the first to admit that we don't have real "AI" yet, and we won't have it in the foreseeable future. But to say that real intelligence is not possible with the current tool set is simply incorrect. It may be impractical. It may take 100+ years to figure out how to build nets this large and train them successfully before sun burns out, but it is _theoretically_ provably doable. In a way it's kind of like fusion: you have a proof that it can work right above your head during the day. We just don't know how to do it at our "human" scale with our knowledge and capabilities. Our knowledge and capabilities, however, are not a constant - they improve exponentially over time, although most people don't perceive this process as exponential.
- deleted 6y ago[deleted]
- ckcheng 6y agoFor clarity, Universal approximation doesn't mean ANNs can approximate just absolutely any function... Depending on which proof you're looking at, ANNs can provably approximate any continuous function, or any Lebesgue integrable function, or any real vector-valued continuous function on a compact subset of R^n space [1]. That's certainly a very large class of functions, but it's not just any function. Also, those proofs only work if you allow ANNs with arbitrary number of layers or layer width. Perhaps the required ANN size to approximate the AI / AGI function is so huge that it's beyond the number of particles on Earth. At what point is it beyond impractical? What if the ANN size (number of layers or neurons) is beyond the number of particles in the galaxy? Maybe it's just the wrong approach. I haven't studied those proofs in detail, but as existence proofs, they just show that some ANN exists that can approximate a given function (of a certain class). There's no guarantee that there's a general algorithm that can learn the required ANN, is there? Maybe you just have to be really lucky to randomly initialize onto the correct approximating ANN... Also, I'm not aware of any proof that human or bio NNs implement a Lebesgue integrable function, or etc. It's entirely possible bio NNs uses some weird (quantum?) physical process, and implements a function outside the scope of any ANN universal approximation theorem. I'm just a little more cautious around those universal approximation theorems. I'm more or less in agreement with your comment otherwise! [1]: https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- LdSGSgvupDV 6y agoIt seems they are on the right direction. But, the number of parameters hit the physical barrier of silicon-based device. To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world. I personally got a feeling that this is bound to how nature calculate the physical phenomenon. For simulation, there are lots of "for loop" in our code/machine. But, in real world, we could not observe any "for loop". It is fast and continuous in the macro scale. In order to reach strong AI, it is time to focus on quantum computing.
- sudosysgen 6y agoFor loops in AI are actually only there to implement continuous mathematical operations such as matrix multiplications in a single core. Modern AI chips and implementations use very little to no for loops.
- LdSGSgvupDV 6y agoSorry, maybe it is a wrong analogy. What I want to emphasize is that there is no calculating time in nature. At least, we could not feel "lag" in our "Nature Server". For the part of chips, did you mean that it runs the "for loop" as parallel in each micro-chip (or gate or whatever tiny structure it is)?
- sudosysgen 6y ago>Sorry, maybe it is a wrong analogy. What I want to emphasize is that there is no calculating time in nature. At least, we could not feel "lag" in our "Nature Server". Yes, but the reality is that there is lag in our projection of what nature is and that we just ignore it. Different parts of our perception actually don't exist at the same instant. >For the part of chips, did you mean that it runs the "for loop" as parallel in each micro-chip (or gate or whatever tiny structure it is)? Pretty much, the for-loop is unfolded and ran in parallel.
- i-am-curious 6y ago
- IfOnlyYouKnew 6y agoIt amazes me people think their brain works differently. It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud. Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network. On the other side of the equation, the emergent behaviour of deep networks make them fundamentally different from the sort of statistics that came before. So even if the building blocks are old and boring, the larger system isn't.
- visarga 6y agoWhat we need here, I think, is a GAN-like approach. Use a generator + a discriminator. The generator could be like GPT-3, but it needs another model to filter out the nonsense. We work the same way - come up with crazy ideas, then check them out to see if they hold ... 'I had an idea, but nah, it wouldn't work'. It's a process supported by different skills.
- snowwrestler 6y agoThe issue isn't whether neural networks work, the issue is the dataset. Look again at what the parent wrote: > I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world Emphasis mine. The human brain does not develop what we know as human intelligence by analyzing text.
- tsimionescu 6y agoAs another poster wrote, the main problem is not the neural network itself, it is the training set, and very likely the training algorithm as well. My claim is that working knowledge of the world is simply not encoded in human text. All human text (and speech) presupposes a certain model of the world that is learned from experiencing the world, likely based on a model that itself is partially pre-trained by evolutionary processes.
- hackinthebochs 6y ago