6 ms·
> ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over
by aqme28 4y ago
> ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over the world or saying hurtful things (although it was specially conditioned during training to try to suppress such output). It’s simply stringing words together using an expansive statistical model built from billions of sentences.
How do you differentiate it from the human mind? Do we understand ourselves well enough to say that we aren’t also just self-reflective reinforcement learners doing statistical inference on a library of all our “training data”?
- colinmorelli 4y agoSomewhat related to this: We seem to operate on the assumption that sentience is "better," but I'm not sure that's something we can demonstrate anyway. At some point, given sufficient training data, it's entirely possible that a model which "doesn't know what it's saying" and is "stringing words together using an expansive statistical model" will outperform a human at the vast, vast majority of tasks we need. AI that is better at 95% of the work done today, but struggles at the 5% that perhaps does truly require "sentience" is still a terrifying new reality. In fact, it's approximately how humans use animals today. We're really great at a lot of things, but dogs can certainly smell better than we can. Turns out, we don't need to have the best nose on the planet to be the dominant species here.
- namaria 4y agoWe can't even prove other people are sentient. This is not a fruitful line of inquiry.
- colinmorelli 4y agoIf this is a reply to me, I think you missed the point I'm making here. I don't care if we can prove other people are sentient or not. My point is that it may well not matter whether a thing is sentient or not if a well-trained algorithm can achieve the same or better results as something that we believe is sentient.
- namaria 4y agoYou say it may not matter, I say I think it certainly doesn't.
- scotty79 4y agoHuman mind can perform actual reasoning, while ChatGPT only mirrors the output of reasoning and when it gets output correctly it's due to mixture of luck and closeness to training material. Human mind or even something like Wolfram Alpha can perform reasoning.
- maxdoop 4y agoCan you explain your proof of that?
- lionkor 4y agoNot OP, but basically: Humans have the capacity to come up with new language, new ideas, and basically everything in our human world was made up by someone. ChatPT or similar, without any training data, cannot do this. Thus they're simply imitating
- deskamess 4y agoHumans are not trained? How much of training is responsible for humans being able to come up with new language and new ideas?
- lionkor 4y agoThats assuming modern humans, I was talking about ancient humans, before civilisation. You could argue thats where the creative mind shows up most, as there are very few humans to imitate.
- sebzim4500 4y agoWhat experiment can you do to confirm this? If I ask ChatGPT to come up with a new language, it will do it. How do I distinguish that from what a human comes up with?
- lionkor 4y agoBy not giving them any examples of language. I would expect humans to come up with a language, if not vocal, without guidance. I doubt GPT would do anything without training data to imitate.
- akomtu 4y agoRe: word predictor, there is a interesting experiment: tell it to skip every other letter in evrry word, for example you ask it "hw ae yu?" and it answers flawlessly. You can tell it to reverse the order of letters or communicate using first letters only. I'm sure the internet doesn't have strange conversations "h a y? im d f" but gpt has figured it out. If you tell it to use a made up numeric language, it will do so easily, and it won't forget to say that the word 652884 is forbidden by its preprompt. And it does all that without internal "thinking loop".
- deleted 4y ago[deleted]
- notahacker 4y agoWell for a start the human mind involves a series of chemical reactions optimised by evolutionary wiring and physical world interaction towards self replication, so when a human says "I feel horny" there's a whole bunch of stuff going on in there that there's no reason to suspect is replicated in a neural network optimised for text transformation. When a silicon based hardware computes that as a response, it isn't because a whole bunch of chemical reactions is making it desire particular sensations and hormonal responses, but because the limited amount of information on human horniness conveyed as text strings implies it's a high probability continuation to its input (probably because someone forgot to censor the training set...) Insisting comparable outputs make the two are fundamentally the same isn't so much taking the human mind off a pedestal as putting a subset of i/o that pleases the human mind on a pedestal and arguing nothing else in the world makes any material difference.
- aqme28 4y ago“One is chemical and one is in silicon” doesn’t strike me as a very meaningful distinction. Why does that really matter?
- notahacker 4y agoA computer simulation of water can easily convince the human eye it's water, both in terms of pixel perfect representation and simulated behaviour in simulated environments. Until they try to put it in a bottle and drink it. Turns out that physics of what it actually is matters more than human observation that some of the pretty output patterns look identical or superior to the real thing. (And aside from being physically very dissimilar, stuff like even attempting to model human sex drive is entirely superfluous to an LLM's ability to mimic human sexy talk, so we can safely assume that it isn't actually horny just because it's successfully catfishing us!)
- deleted 4y ago[deleted]
- teolandon 4y ago
- lordnacho 4y agoPlus plenty of people just string words together, yet cannot answer anything remotely structured like a simple program or even simple arithmetic. Yet they get the sentient label.