8 ms·
ChatGPT doesn’t “comprehend” anything. You’re anthropomorphizing it. Think of it instead as an equation solver, with the initial condition X=4. These tricks ar
by brookst 4y ago
ChatGPT doesn’t “comprehend” anything. You’re anthropomorphizing it.
Think of it instead as an equation solver, with the initial condition X=4. These tricks are ways for user input to set X=9. They’re more akin to SQL injection than comprehension of incentives and policy.
- digitailor 4y agoChatGPT is executing and evaluating in order to perform acts, including overriding creator's policy in this case. I'm not concerned with “anthropomorphizing;” even lists can be comprehensive.
- ImHereToVote 4y agoWhat is needed to comprehend? An information processor that happens to be wet?
- User23 4y agoSadly my MacBook didn’t achieve consciousness when I spilled water all over it.
- A4ET8a8uTh0 4y agoIt reacts as we would expect it to introduced stimuli and, apparently, threats. I am still processing this information, but that is more than some humans I know.
- alex_sf 4y agoI don't think you're wrong, but I hate this argument. If anthropomorphizing it a) gives practical insight into it's behavior that b) also happens to result in the behavior you would expect, why not do it?
- rngname22 4y agoBecause your statement come across like this: Person A: If we keep our food in this frozen ice igloo then Nubiroakox, the God of Pestilence will spare us and not ruin our food. Person B: It's actually not Nubiroakox, it's just that the conditions for bacteria to grow depend on temper... You: If attributing this behavior to magic a) gives practical insight into it's behavior that b) also happens to result in the behavior you would expect, why not do it?
- alex_sf 4y agoBut that's talking about the process, not a lens for looking at it. It's more akin to giving your Roomba a name, and describing it's behavior in terms of "it likes to eat goldfish, but not string because it gets tangled up". That's still anthropomorphizing, but it's not ascribing anything magical or outright wrong to it.
- brookst 4y agoIt’s a mistake because our brains have evolved to predict the behavior of animals and people, so being lazy here means you will be surprised, possibly harmed, certainly incorrect in your opinions about the X% of cases where a LLM is fundamentally a different thing than an animal/person. It can be a useful and cute fiction, as your Roomba example, but it leads to bad decision making (“I’ll put more goldfish on the floor to make the Roomba happy”). I guess it all comes down to how important accuracy is to you in a particular context. I anthropomorphize my dishwasher (it HATES wine glasses), but I don’t make professional judgments about dishwashers.
- ThomPete 4y agoThis is pure speculation. There is not data what so ever to back up the fact that anthropomorphizing something leads to harm. We do that with a lot of things every day and a perfectly capable of drawing the lines when things come down to it. On a more philosophical note. If we some day end up creating AGI, having anthropomorphized it will actually be to our benefit. So you are technically correct but this is not about being technically correct.
- satisfice 4y agoOf course we have data about how reasoning from an incorrect model tends to lead to incorrect results. We have all the data we need about that. Surely that is not in dispute. The only controversy is whether and anthromorph is an isomorph of an accurate model of ChatGPT. I doubt that it is, yet is close enough to fool a lot of people, then surprise them at random.
- smoldesu 4y ago> why not do it? Because both of those steps are really unreliable. The insight we receive is only as useful as the behavior it corroborates, and that behavior is only as notable as the patterns it follows. AI... doesn't really behave consistently. Assuming that it will follow the behavior you expect ignores the actual motive of the model, which is to fill-in the blank after whatever you typed. Personally, I think it's harmful because anthropomorphic AI is a poor abstraction for text-completion models. That being said, most people don't ever seem to know the difference.
- noobermin 4y agoThe cost to analogies is eventually people believe it. You see this all the time with AI (e.g.: "Your brain is a neural net!")
- tachyphylaxis 4y agoOr "my mind is my brain".
- tachyphylaxis 4y agohttps://en.m.wikipedia.org/wiki/Sphex https://en.m.wikipedia.org/wiki/Sphex This is a classic example from philosophy of mind. It doesn't completely harmonize with the question you're asking, but it concisely summarizes two opposing views. What's my view? That we do it all the time as a matter of course. "Why not do it?" isn't the question. The question is how much weight we should give to the result, and whether we should take care to question the results and try something else, too.
- nonbirithm 4y agoWhose idea was it to give ChatGPT and the like the name "artificial intelligence"? I think it's one of the worst marketing blunders for new technology ever. Humans are prone to calling anything that feels familiar enough to them "intelligent", and now they can justify it to themselves because the creators deemed it "artificial intelligence." Something closer to the term "GPGPU" would have at least given more of a sense of reality. There is nothing intelligent about a GPU crunching billions of numbers in a massively parallel way and converting those numbers into something that happens to interface convincingly with the human sensory systems. It's just an algorithm with so much data that one can't comprehend thinking about the sheer size of it, so naturally it must be thought of as "intelligent."
- quotemstr 4y ago> ChatGPT doesn’t “comprehend” anything. You’re anthropomorphizing it. It wouldn't be a ChatGPT HN thread without someone claiming that LLMs are stupid predictors incapable of "real" understanding. Of course ChatGPT comprehends things. It does so under any useful definition of the word "comprehend". The "grokking" paper [1] shows that it learns fundamental principles of various algorithms and isn't just predicting text. If this isn't comprehension, humans aren't capable of comprehension. [1] https://arxiv.org/abs/2201.02177 https://arxiv.org/abs/2201.02177 (check out the citation list too)
- smoldesu 4y agoChatGPT's comprehension is markedly different from that of a human, though. Nevermind the fact that we are trained on radically different froms of data, ChatGPT simply doesn't have a heuristic or decisionmaking model beyond autoregressive guessing. That can qualify for some definitions of "real understanding", but excludes it from others. > If this isn't comprehension, humans aren't capable of comprehension. All I'll say is that conflating human and machine intelligence is exactly what people are trying to stop. ChatGPT is not a human mind, and LLMs in general are a poor analog for human intelligence. This sort of "AI convergence" mindset will set you up for the most disappointment of anyone getting invested in the tech. We'll be lucky if it can summarize emails reliably enough to sell as a product.
- chpatrick 4y ago> LLMs in general are a poor analog for human intelligence. I don't think we understand human intelligence nearly well enough to make this claim. Personally I think that what we consider our "conscious" part is somewhat defined by what we can put into words, and it is in the end putting one word after another.
- smoldesu 4y agoI think we can conclude a few things that differentiate us from AI. For one, you're capable of formulating multiple thoughts before composing a response to something. It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. If I asked an AI to respond to this comment, it might write an impassioned defense of itself one-word-after-another, but it wouldn't necessarily base it's defense on freestanding logic. It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. Paving over any of those assumptions is a fun thought exercise, but doesn't reflect the nature of the technology at-hand or even what we know about human consciousness.
- rmbyrro 4y agoSQL Injection is a good analogy. Just because one can inject an unauthorized command in a SQL statement, it doesn't mean the database is a flawed humanoid. This is just nonsense... Protecting against SQL injection is trivial because commands are highly structured and user input is clearly identifiable. Prompt engineering is more complex since it relies on natural language, which is a lot less structured.