4 ms·

Your first statement seems to contradict your previous. Did you originally mistype what you meant when you said greater exposure leads to worse outcomes? Becaus

by bumby 3y ago

Your first statement seems to contradict your previous. Did you originally mistype what you meant when you said greater exposure leads to worse outcomes? Because now you’re implying more exposure has the opposite effect. Regardless, it still misses the point. I’ve never been explicitly exposed to base-72, yet I can reason my way through it. I would argue my performance wouldn’t be any different than base-82. So I can transfer basic principles. What the LLM result you referenced shows is that it is not learning basic principles. It sure seems like you just read the abstract and ran with it. As far as the psychology of decision making, again, I think you're speaking with greater confidence than is warranted. In time critical examples, I’m inclined to agree. And there’s certainly some notable psychologists who would expand it beyond snap judgments. But there are also some notable psychologists who tend to disagree. It’s not a settled science, despite your confidence. But again, that’s getting stuck in the limitations of the example and missing the forest for the trees. The point is not in whether decisions are made consciously or subconsciously, but rather how learning can be inferred from previous experience and transferred to novel experiences. Whether this happens consciously or not is besides the point. And you are further going down what I was explicitly taking against: confusing image/pattern recognition for contextual reason. You can see this in the relatively recent Go issue; any human could see what the issue was because they understand the contextual reasoning of the game but the AI could not and was fooled by a novel strategy. The points I’ve been making have completely flew over your head to the point where you’re shoehorning in a completely different conversation.

deleted 3y ago
[deleted]
famouswaffles 3y ago
>Did you originally mistype what you meant when you said greater exposure leads to worse outcomes? Because now you’re implying more exposure has the opposite effect. I guess so. I've never meant to imply greater exposure leads to worse outcomes. >I would argue my performance wouldn’t be any different than base-82. Even if that were true and i don't know that i agree, the authors of that paper make no attempt to test in circumstances that might make this true for LLMs as it might for people. So the paper is not evidence of the claim (no basic principles) either way. For example, i reckon your performance on the proceeding 82 test will be better if taken a immediately after than if taken weeks or months later. So surrounding context is important even if you're right. >What the LLM result you referenced shows is that it is not learning basic principles. I disagree here and i've explained why. >You can see this in the relatively recent Go issue; any human could see what the issue was because they understand the contextual reasoning of the game but the AI could not and was fooled by a novel strategy. You're talking about this ? https://www.zmescience.com/future/a-human-just-defeated-an-ai-in-go-heres-why-that-matters/ https://www.zmescience.com/future/a-human-just-defeated-an-a... KataGo taught itself to play go by explicitly deprioritizing “losing” strategies. This means it didn’t play many amateur strategies because they were lost early in the training. This is hard for a human to understand because humans all generally share a learning curve going from beginning to amateur to expert. So all humans have more experience with “losing” techniques. Basically what I’m saying is, it might be that the training scheme of this AI explicitly prioritized having little understanding of these specific tactics, which is different than not having any understanding. This circles back to the point I made earlier. Having failure modes humans don't or won't understand or have is not the same as a lack of "true understanding". We have no clue what "basic principles" actually are on the low level. The less inductive bias we try to shoehorn into models, the better performing they become. Models literally tend to perform worse the more we try to bake "basic principles" in. So presence of an odd failure mode we *think* belies a lack of "basic principles" is not necessarily evidence of a lack of it. >The points I’ve been making have completely flew over your head to the point where you’re shoehorning in a completely different conversation. You're convinced it's just "very good pattern matching", whatever that means. I disagree.
- bumby 3y ago
  I think the short of it is that it seems to me that you are confusing a system having very good heuristics for having a solid understanding of principles of reality. Heuristics, without an understanding of principles, is what I mean by rote pattern matching. But heuristic break down, particularly in edge cases. Yes, humans also rely heavily on heuristics because we generally seek to use the least effort possible. But we also can mitigate against those shortcomings by reasoning about basic principles. This shortcoming is why I think both humans and AI can make seemingly stupid mistakes. The difference is, I don't think you've provided evidence that AI can have a principled understanding while we can show that humans can. Having a principled understanding is important to move from simple "cause-effect" relationships to understanding "why". This is important because the "why" can transfer to many unrelated domains or novel scenarios. E.g., racism/sexism/...most -'isms' appear to be a general heuristics that help us make quick judgements. But we can also our decision-making process by reverting to basic principles, like the idea that humans have equal moral worth regardless of skin tone or gender. AI can even mimic these mitigations, but you haven't convinced me that it can fundamentally change away from it's training set based on an understanding of basic principles. As for the Go example, a novice would be able to identify that somebody is drawing a circle around it's pieces; your link even states this. But you recharacterizing this as a specific strategy is weird when that strategy causes you to lose the game. It misses the entire meaning of strategy. We see the limitations of AI in it's reliance to training data from autonomous vehicles to healthcare. They range from the serious (cancer detection) to the humorous (Marines overtaking robots by hiding in boxes like in Metal Gear). The paper you referenced similarly shows it is reliant on proximity to the training set, rather than actually understanding the underlying principles.
  - famouswaffles 3y ago
    >I think the short of it is that it seems to me that you are confusing a system having very good heuristics for having a solid understanding of principles of reality. Humans don’t have a grasp of the “principles of reasoning” and as such are incapable of distinguishing “true”, "different" or “heuristic” assuming such a distinction is even meaningful. Where you are convinced of “faulty shortcut”, I simply think “different”. Multiple ways to skin a cat. a plane's flight is as "true" as any bird. There's no "faulty shortcut" even when it fails in ways a bird will not. You say humans are "true" and LLMs are not but you base it on factors that can be probed in humans as well so to me, your argument simply falls apart. This is where our divide stems from. >I don't think you've provided evidence that AI can have a principled understanding while we can show that humans can. What would be evidence to you? Let’s leave conjecture and assumptions. What evaluation exist that demonstrate this “principled understanding” in humans? and how would we create an equitable test in LLMs? >a novice would be able to identify that somebody is drawing a circle around it's pieces; your link even states this. But you recharacterizing this as a specific strategy is weird when that strategy causes you to lose the game. You misunderstand. I did not characterize this as a specific “strategy”. Not only do modern Go systems not learn like humans, but they also don’t learn from human data at all. KataGo didn’t create a heuristic to play like a human because it didn’t even see humans play. >The paper you referenced similarly shows it is reliant on proximity to the training set, rather than actually understanding the underlying principles. Even the authors make it clear this isn’t necessarily the bridge to take so it’s odd to see you die on this hill. The counterfactual of syntax is Finding the main subject and verb of something like “Think are the best LMs they.” in verb-obj-subj order (they, think) instead of “They think LMs are the best.” in subj-verb-obj order (they, think). LLMs are not being trained on text like the former to any significant degree if at all yet the performance is fairly close. So what, it doesn’t “underlying principles of syntax” but still manages that ? The problem is that you take a fairly reasonable conclusion from these experiments. I.e LLMs can/often also rely on narrow, non-transferable procedures for task-solving and proceed to jump the shark from there. >but you haven't convinced me that it can fundamentally change away from it's training set based on an understanding of basic principles. We see language models create novel functioning protein structure after training, no folding necessary. https://www.researchgate.net/publication/367453911_Large_language_models_generate_functional_protein_sequences_across_diverse_families https://www.researchgate.net/publication/367453911_Large_lan... So does it still not understand the “basic principles of protein structures”?
deleted 3y ago
[deleted]

by bumby 3y ago

deleted 3y ago

[deleted]

famouswaffles 3y ago

>Did you originally mistype what you meant when you said greater exposure leads to worse outcomes? Because now you’re implying more exposure has the opposite effect. I guess so. I've never meant to imply greater exposure leads to worse outcomes. >I would argue my performance wouldn’t be any different than base-82. Even if that were true and i don't know that i agree, the authors of that paper make no attempt to test in circumstances that might make this true for LLMs as it might for people. So the paper is not evidence of the claim (no basic principles) either way. For example, i reckon your performance on the proceeding 82 test will be better if taken a immediately after than if taken weeks or months later. So surrounding context is important even if you're right. >What the LLM result you referenced shows is that it is not learning basic principles. I disagree here and i've explained why. >You can see this in the relatively recent Go issue; any human could see what the issue was because they understand the contextual reasoning of the game but the AI could not and was fooled by a novel strategy. You're talking about this ? https://www.zmescience.com/future/a-human-just-defeated-an-ai-in-go-heres-why-that-matters/ https://www.zmescience.com/future/a-human-just-defeated-an-a... KataGo taught itself to play go by explicitly deprioritizing “losing” strategies. This means it didn’t play many amateur strategies because they were lost early in the training. This is hard for a human to understand because humans all generally share a learning curve going from beginning to amateur to expert. So all humans have more experience with “losing” techniques. Basically what I’m saying is, it might be that the training scheme of this AI explicitly prioritized having little understanding of these specific tactics, which is different than not having any understanding. This circles back to the point I made earlier. Having failure modes humans don't or won't understand or have is not the same as a lack of "true understanding". We have no clue what "basic principles" actually are on the low level. The less inductive bias we try to shoehorn into models, the better performing they become. Models literally tend to perform worse the more we try to bake "basic principles" in. So presence of an odd failure mode we *think* belies a lack of "basic principles" is not necessarily evidence of a lack of it. >The points I’ve been making have completely flew over your head to the point where you’re shoehorning in a completely different conversation. You're convinced it's just "very good pattern matching", whatever that means. I disagree.

bumby 3y ago
I think the short of it is that it seems to me that you are confusing a system having very good heuristics for having a solid understanding of principles of reality. Heuristics, without an understanding of principles, is what I mean by rote pattern matching. But heuristic break down, particularly in edge cases. Yes, humans also rely heavily on heuristics because we generally seek to use the least effort possible. But we also can mitigate against those shortcomings by reasoning about basic principles. This shortcoming is why I think both humans and AI can make seemingly stupid mistakes. The difference is, I don't think you've provided evidence that AI can have a principled understanding while we can show that humans can. Having a principled understanding is important to move from simple "cause-effect" relationships to understanding "why". This is important because the "why" can transfer to many unrelated domains or novel scenarios. E.g., racism/sexism/...most -'isms' appear to be a general heuristics that help us make quick judgements. But we can also our decision-making process by reverting to basic principles, like the idea that humans have equal moral worth regardless of skin tone or gender. AI can even mimic these mitigations, but you haven't convinced me that it can fundamentally change away from it's training set based on an understanding of basic principles. As for the Go example, a novice would be able to identify that somebody is drawing a circle around it's pieces; your link even states this. But you recharacterizing this as a specific strategy is weird when that strategy causes you to lose the game. It misses the entire meaning of strategy. We see the limitations of AI in it's reliance to training data from autonomous vehicles to healthcare. They range from the serious (cancer detection) to the humorous (Marines overtaking robots by hiding in boxes like in Metal Gear). The paper you referenced similarly shows it is reliant on proximity to the training set, rather than actually understanding the underlying principles.
- famouswaffles 3y ago
  >I think the short of it is that it seems to me that you are confusing a system having very good heuristics for having a solid understanding of principles of reality. Humans don’t have a grasp of the “principles of reasoning” and as such are incapable of distinguishing “true”, "different" or “heuristic” assuming such a distinction is even meaningful. Where you are convinced of “faulty shortcut”, I simply think “different”. Multiple ways to skin a cat. a plane's flight is as "true" as any bird. There's no "faulty shortcut" even when it fails in ways a bird will not. You say humans are "true" and LLMs are not but you base it on factors that can be probed in humans as well so to me, your argument simply falls apart. This is where our divide stems from. >I don't think you've provided evidence that AI can have a principled understanding while we can show that humans can. What would be evidence to you? Let’s leave conjecture and assumptions. What evaluation exist that demonstrate this “principled understanding” in humans? and how would we create an equitable test in LLMs? >a novice would be able to identify that somebody is drawing a circle around it's pieces; your link even states this. But you recharacterizing this as a specific strategy is weird when that strategy causes you to lose the game. You misunderstand. I did not characterize this as a specific “strategy”. Not only do modern Go systems not learn like humans, but they also don’t learn from human data at all. KataGo didn’t create a heuristic to play like a human because it didn’t even see humans play. >The paper you referenced similarly shows it is reliant on proximity to the training set, rather than actually understanding the underlying principles. Even the authors make it clear this isn’t necessarily the bridge to take so it’s odd to see you die on this hill. The counterfactual of syntax is Finding the main subject and verb of something like “Think are the best LMs they.” in verb-obj-subj order (they, think) instead of “They think LMs are the best.” in subj-verb-obj order (they, think). LLMs are not being trained on text like the former to any significant degree if at all yet the performance is fairly close. So what, it doesn’t “underlying principles of syntax” but still manages that ? The problem is that you take a fairly reasonable conclusion from these experiments. I.e LLMs can/often also rely on narrow, non-transferable procedures for task-solving and proceed to jump the shark from there. >but you haven't convinced me that it can fundamentally change away from it's training set based on an understanding of basic principles. We see language models create novel functioning protein structure after training, no folding necessary. https://www.researchgate.net/publication/367453911_Large_language_models_generate_functional_protein_sequences_across_diverse_families https://www.researchgate.net/publication/367453911_Large_lan... So does it still not understand the “basic principles of protein structures”?

deleted 3y ago

[deleted]