6 ms·
That's the difference though. I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model
by lsy 3y ago
That's the difference though. I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins. To a neural network there is no distinction. The closest the entire dynamic comes is the very basic annotation of RLHF which itself is done by an external human who is providing the value judgment, but even that is absent once training is over.
Despite not having the bird's sense for electromagnetic waves, I have an understanding that they are there, because humans saw behavior they couldn't describe and investigated, in a back-and-forth with a world that has some capacity to disprove hypotheses.
Additional modalities are really just reducible to more kinds of text. That still doesn't exhaust the world, and unless a machine has some ability to integrate new data in real time alongside a meaningful commitment and accountability to the world as a world, it won't be able to cope with the real world in a way that would constitute genuine intelligence.
- famouswaffles 3y ago>I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins. Yeah this isn't really true. There's not how humans work. For a variety of reasons, Plenty stick with their incorrect model despite the world indicating otherwise. In fact, this seems to be normal enough human behaviour. Everyone does it, for something or the other. You are no exception. And yes LLMs can in fact tell truth from fiction. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 https://arxiv.org/abs/2310.06824 Your argument seems to boil down to "they can't perform experiments" but that isn't true either.
- user_named 3y agoIt is a very basic fact that LLMs have no concept of true or false, it only has an ability to look up what text data it has seen before. If you do not understand this you are in no position to discuss LLMs.
- ewild 3y ago[flagged]
- deleted 3y ago[deleted]
- SpicyLemonZest 3y agoI really don't know what people mean when they say this. We routinely instruct computer chips to evaluate whether some condition is true and take action on that basis, even though the chip is "just" a selectively doped rock. Why would the details of an LLM's underlying architecture mean that it can't have a concept of true or false?
- xcv123 3y agoOne of the most ridiculous comments I have read about LLMs here. The ~100 layer deep neural networks infer many levels of features over the text, including the concept of true and false. That is trivial for an LLM. Are you completely unaware these are based on deep neural networks? Convolutional Neural Networks don't operate by "look up" of text data.
- user_named 3y agoOkay, so then tell me how does it decide whether it is true or false that Biden is the POTUS? It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.
- jeffparsons 3y ago
- astrange 3y agoThere is no such thing as a world model, and you don't have one of them. This is a leftover bad psychological concept from the 70s AI researchers who never got anywhere. People and other creatures do very little modeling things, they mostly just do stuff.
- xcv123 3y agoWorld model means inner representation of the external world. Any organism with a functioning brain has a world model. That's what brains do. If you don't have a world model then you are a vegetable and could not be replying on HN.
- astrange 3y agoIf you close your eyes, how long can you navigate in the environment without hitting something? Not long, because you didn't model it. If you're taking out the recycling, do you take the time to identify (model) each piece of it first? No, because that's not necessary.
- xcv123 3y agoWait, you actually think we are talking about modelling as a conscious deliberate process in active working memory? Well there's your fundamental mistake. That is not what we are discussing, not even remotely. The vast model in your brain is learned and generated unconsciously without your direct awareness.
- jofla_net 3y agoI do agree, but more importantly love this part of the argument! Its when all the personality differences become too much to bear and suddenly people are accused of not even knowing themselves. Been there before, what a wild ride!
- Dylan16807 3y ago> suddenly people are accused of not even knowing themselves It's not some desperate retort. People don't know themselves very well. Look at the research into confabulation, it seems to be standard operating procedure for human brains.
- mattigames 3y agoIn all these arguments its implied that this "genuine intelligence" is something humans all have, and nothing could be farther from the truth, that is why we have flat earthers or religious people and many other people beliving for decades easily refutable lies.