4 ms·

>This would only be true if you believe in a singular definition of intelligence where you believe there is only one right way to get an answer. The whole idea

>This would only be true if you believe in a singular definition of intelligence where you believe there is only one right way to get an answer. The whole idea of the first principles movement was that it was the "right way" anchored by how humans supposedly approached problems and many of these brilliant minds thought intelligence could not exist without "symbolic manipulation". Well it wasn't "the right way" because it mostly failed and even the little niches it did work out produced systems that didn't behave anything like we did. Anything other than clear definitions and unambiguous axioms (aka the real world) and logic systems fall apart. I don't need to believe there is only one path to intelligence (I don't) to see the first principle way just doesn't cut it. General Intelligence will never happen this way. Narrow real world intelligence will never happen this way. Decades of the brightest minds toiling and some people still don't seem to get it. Logic scales poorly if your problem space isn't exactly defined. >There are lots of reasons we don't train AI from first principles, one of the biggest being that neural networks are just flat-out easier to train. The single biggest reason is that it doesn't work for most aspects of intelligence we observe in the real world. >Training from first principles is annoying and time consuming, but it hasn't failed as a way for accomplishing tasks Yes it has. >it was ill-suited for an extremely specific field. Ah yes, the very specific fields of language, vision and speech synthesis and understanding, reinforcement learning, OCR and many more. These were all things attempted with GOFAI systems prior to NNs. Not only that, these systems had decades of a headstart. Symbolic systems were not the underdog here. They were discarded because they struggled to succeed. Symbolic NLP had gone by the wayside long before even transformers and GPT arrived. That's how poor they were. Like Frederick Jelineck famously said, "Every time I fire a linguist, the performance of the system goes up." First principles are an idea that never actually left the realm of fiction. >Humans observably do not learn the same way as GPT does, we do not start out by learning language and then logic, we do the opposite. This doesn't actually matter. A language model is trying to predict the next token. It is trying to reverse engineer the computation that could have led to an utterance. It is trying to model the casual processes that led to the text. It's not trying to draw shadows. It's trying to build the object whose shadow is a result. I'm not saying this means GPT is a human or is doing everything the human way(whatever that may be) but gradient descent to predict a language corpus and evolution after millions of years(which is itself just another dumb optimizer not first principling anything) can converge on the same solution. Babies will communicate coherently much earlier in a non speaking language so vocal production is obviously a huge barrier that has nothing to do with language understanding itself. Humans not formally trained in logic are typically very bad at it. We're obviously not logic creatures. This is exactly my point. How humans think they work are how they actually work are very different things entirely.

danShumway 3y ago
> Well it wasn't "the right way" because it mostly failed "Failed" and "we weren't able to replicate it in a simulator" are two entirely different things. > [...] Yes it has. Based on what? If your only criteria here is what is useful for AI, then you're taking an extremely reductionist stance on what intelligence is. > Ah yes, the very specific fields of language, vision and speech synthesis and understanding, reinforcement learning, OCR and many more. No, the very specific field of AI. Holy crud. Even within computing, first-principle approaches are wildly useful as soon as you step outside of the field of AI, first-principle systems produce better and more accurate simulations, are better equipped to handle emergent systems, they're better equipped for complicated tasks like physical rendering. Outside of AI, nobody doubts that efficacy of first-principle reasoning about the world. As a way to model the world, first principle reasoning and simulations work great. And if your only definition of what does and doesn't work as a way to reason about the world is "what works with AI" then... sure, whatever. But the world is bigger than AI. > The single biggest reason is that it doesn't work for most aspects of intelligence we observe in the real world. You're making an argument here about whether or not symbolic representations and first-principle reasoning is useful for AI. That... isn't what I'm talking about. I can't stress enough how reductive and narrow it is to approach a conversation about methods of reasoning through the lens of "if it's not useful for AI, then it's not real." And even if first-principle reasoning was completely useless, that has nothing to do with whether or not GPT works the same way as a human being. This misses the broader point that however you think the human brain works, it very much appears to be modeling the world and learning about the world differently than GPT. First-principle reasoning isn't the only way that humans learn of course, but we do obviously teach people information about the world based on first principles. There is going to be a lot of that in any school setting. But whatever, let's throw that all out the window; fine. Let's say that humans don't learn anything at all through examining and building on first-principles. However it is that humans do learn, the end result is different from GPT in observable ways. Chess is a good example of that. ---- > This doesn't actually matter. I mean... it doesn't matter unless you want to be able to make accurate predictions about what the system will and won't excel at, or want to engage with the nuances of how the system works beyond the broad strokes. Are you seriously suggesting that it's of no importance that we figure out how large neural networks like GPT work internally? I don't think any serious AI researcher would agree with that. > This doesn't actually matter. A language model [not] is trying to predict the next token. It is trying to reverse engineer the computation that could have led to an utterance. I don't think OpenAI researchers would agree with you on this. Yes, GPT is building heuristics and models that allow it to better predict an utterance. It is not clear that it has any motivation to accurately model the underlying reality that led to that utterance. To be clear, we don't know how GPT models the world internally or what it models. It is a leap of faith to say that it is optimizing its internal models for accuracy rather than usefulness or general applicability. That is expressing a lot of confidence that I have not seen shared within general AI research or writing. > I'm not saying this means GPT is a human or is doing everything the human way (whatever that may be) Okay, we agree! That's literally all anyone is saying here, GPT does not appear to approach chess the same way that a human does and doesn't think about the game the same way that a human does. It does not "understand" the game in an identical way to how humans understand the game. That doesn't mean it's not doing anything, it just means it's not thinking using the same systems we use. If you view that as an insult to GPT or like it's some kind of contest, that's on you. The only thing that's actually being said is that humans and GPT think about and understand chess differently. ---- > but gradient descent to predict a language corpus and evolution after millions of years(which is itself just another dumb optimizer not first principling anything) can converge on the same solution. Citation needed. I believe it can result in very powerful solutions, I don't see any reason to believe that pure language prediction and evolution, two algorithms that are fed different inputs and optimize in different ways would produce identical results. Evolution itself doesn't produce identical results when different inputs and environments are supplied -- even within the exact same system with the exact same training strategy, you get resulting intelligences that are wildly different. Again, since I have to clarify this, I'm not bashing LLMs when I say that. I'm saying that evolutionary pressures based on real-world inputs are different from fitness functions. I don't really see how that would be a controversial thing to say. > Babies will communicate coherently much earlier in a non speaking language so vocal production is obviously a huge barrier that has nothing to do with language understanding itself. You're not going to be able to get around this. Babies exhibit logical inferences about the world before they exhibit the ability to understand language. This is true if you teach a baby sign language, it's true if you try to judge its auditory or verbal vocabulary. In addition, when a child gets to be 5 years old, they exhibit logical behavior beyond their linguistic abilities. Their logical reasoning progresses faster than their language skills. Now, contrast that with a small LLM. A 7B LLM possesses almost perfect language skills, but very limited reasoning ability. The conclusion? LLMs learn language faster than reasoning, their reasoning capabilities are an emergent property from their language skills, not the other way around. But humans do not learn the same way that an LLM learns. We do not primarily learn by predicting language tokens. Predicting language tokens is a strategy for a kind of learning, but it's not the one that humans use. In fact, if you try to teach a human how to read though token prediction, they become illiterate. There was just a giant scandal about this a while ago where reading scores dropped dramatically when teachers moved away from first-principles phonics-style teaching to a predictive model where they asked kids "what do you think the next word might be?" We are doing something different. Whether AI can replicate that... who cares? It genuinely does not matter if GPT works the same way as a human being does. Lots of tools work differently from humans and are still useful. But it's different, and understanding that it's different can help us use the tool more effectively.

danShumway 3y ago

> Well it wasn't "the right way" because it mostly failed "Failed" and "we weren't able to replicate it in a simulator" are two entirely different things. > [...] Yes it has. Based on what? If your only criteria here is what is useful for AI, then you're taking an extremely reductionist stance on what intelligence is. > Ah yes, the very specific fields of language, vision and speech synthesis and understanding, reinforcement learning, OCR and many more. No, the very specific field of AI. Holy crud. Even within computing, first-principle approaches are wildly useful as soon as you step outside of the field of AI, first-principle systems produce better and more accurate simulations, are better equipped to handle emergent systems, they're better equipped for complicated tasks like physical rendering. Outside of AI, nobody doubts that efficacy of first-principle reasoning about the world. As a way to model the world, first principle reasoning and simulations work great. And if your only definition of what does and doesn't work as a way to reason about the world is "what works with AI" then... sure, whatever. But the world is bigger than AI. > The single biggest reason is that it doesn't work for most aspects of intelligence we observe in the real world. You're making an argument here about whether or not symbolic representations and first-principle reasoning is useful for AI. That... isn't what I'm talking about. I can't stress enough how reductive and narrow it is to approach a conversation about methods of reasoning through the lens of "if it's not useful for AI, then it's not real." And even if first-principle reasoning was completely useless, that has nothing to do with whether or not GPT works the same way as a human being. This misses the broader point that however you think the human brain works, it very much appears to be modeling the world and learning about the world differently than GPT. First-principle reasoning isn't the only way that humans learn of course, but we do obviously teach people information about the world based on first principles. There is going to be a lot of that in any school setting. But whatever, let's throw that all out the window; fine. Let's say that humans don't learn anything at all through examining and building on first-principles. However it is that humans do learn, the end result is different from GPT in observable ways. Chess is a good example of that. ---- > This doesn't actually matter. I mean... it doesn't matter unless you want to be able to make accurate predictions about what the system will and won't excel at, or want to engage with the nuances of how the system works beyond the broad strokes. Are you seriously suggesting that it's of no importance that we figure out how large neural networks like GPT work internally? I don't think any serious AI researcher would agree with that. > This doesn't actually matter. A language model [not] is trying to predict the next token. It is trying to reverse engineer the computation that could have led to an utterance. I don't think OpenAI researchers would agree with you on this. Yes, GPT is building heuristics and models that allow it to better predict an utterance. It is not clear that it has any motivation to accurately model the underlying reality that led to that utterance. To be clear, we don't know how GPT models the world internally or what it models. It is a leap of faith to say that it is optimizing its internal models for accuracy rather than usefulness or general applicability. That is expressing a lot of confidence that I have not seen shared within general AI research or writing. > I'm not saying this means GPT is a human or is doing everything the human way (whatever that may be) Okay, we agree! That's literally all anyone is saying here, GPT does not appear to approach chess the same way that a human does and doesn't think about the game the same way that a human does. It does not "understand" the game in an identical way to how humans understand the game. That doesn't mean it's not doing anything, it just means it's not thinking using the same systems we use. If you view that as an insult to GPT or like it's some kind of contest, that's on you. The only thing that's actually being said is that humans and GPT think about and understand chess differently. ---- > but gradient descent to predict a language corpus and evolution after millions of years(which is itself just another dumb optimizer not first principling anything) can converge on the same solution. Citation needed. I believe it can result in very powerful solutions, I don't see any reason to believe that pure language prediction and evolution, two algorithms that are fed different inputs and optimize in different ways would produce identical results. Evolution itself doesn't produce identical results when different inputs and environments are supplied -- even within the exact same system with the exact same training strategy, you get resulting intelligences that are wildly different. Again, since I have to clarify this, I'm not bashing LLMs when I say that. I'm saying that evolutionary pressures based on real-world inputs are different from fitness functions. I don't really see how that would be a controversial thing to say. > Babies will communicate coherently much earlier in a non speaking language so vocal production is obviously a huge barrier that has nothing to do with language understanding itself. You're not going to be able to get around this. Babies exhibit logical inferences about the world before they exhibit the ability to understand language. This is true if you teach a baby sign language, it's true if you try to judge its auditory or verbal vocabulary. In addition, when a child gets to be 5 years old, they exhibit logical behavior beyond their linguistic abilities. Their logical reasoning progresses faster than their language skills. Now, contrast that with a small LLM. A 7B LLM possesses almost perfect language skills, but very limited reasoning ability. The conclusion? LLMs learn language faster than reasoning, their reasoning capabilities are an emergent property from their language skills, not the other way around. But humans do not learn the same way that an LLM learns. We do not primarily learn by predicting language tokens. Predicting language tokens is a strategy for a kind of learning, but it's not the one that humans use. In fact, if you try to teach a human how to read though token prediction, they become illiterate. There was just a giant scandal about this a while ago where reading scores dropped dramatically when teachers moved away from first-principles phonics-style teaching to a predictive model where they asked kids "what do you think the next word might be?" We are doing something different. Whether AI can replicate that... who cares? It genuinely does not matter if GPT works the same way as a human being does. Lots of tools work differently from humans and are still useful. But it's different, and understanding that it's different can help us use the tool more effectively.