9 ms·
A Multi-Level View of LLM Intentionality
- famouswaffles 3y ago>Unless you think that there is some fundamental reason why LLMs will never be able to play chess competently, and I doubt there is, then it seems that we could with the right prompts implement some sort of chess AI using an LLM. You can play a good game of chess (or poker for that matter) with GPT. https://twitter.com/kenshinsamurai9/status/1662510532585291779 https://twitter.com/kenshinsamurai9/status/16625105325852917... https://arxiv.org/abs/2308.12466 https://arxiv.org/abs/2308.12466 There's also some work going on in the eleuther ai discord training LLMs specifically for chess to see how they shape up. They're using the pythia models. so far: Pythia 70M, est ELO 1050 Pythia 160M, est ELO 1370
- lawlessone 3y agoI've found they fall apart after a couple of moves and lose track of the game. Edit: This might not be the case anymore it seems, my below point doesn't actually contradict you, seems it matters a lot how you tell the model your moves. Also saying things like "move my rightmost pawn" completely confuses them.
- famouswaffles 3y agoNot had it lose track with the format in the first link (GPT-4, not really tried 3.5)
- lawlessone 3y agoYeah i was wrong. I think it has gotten better since i tried this.
- pixl97 3y agoThe token model of LLMs doesn't map well into how human experience the world of informational glyphs. Left and right is a intrinsic quality of our vision system. An LLM has to map the idea of left and right into symbols via text and line breaks. I do think it will be interesting as visual input and internal graphical output is integrated with text based LLMs as that should help correct their internal experience to be based closer to what we as humans experience.
- lawlessone 3y ago" An LLM has to map the idea of left and right into symbols via text and line breaks." Oh yeah that's i suggested it :) I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better
- famouswaffles 3y ago>I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better GPT-4's spatial position understanding is actually really good all things considered. By the end, 4 was able to construct an accurate maze just from feedback about the current position and possible next moves after each move by GPT-4. https://ekzhu.medium.com/gpt-4s-maze-navigation-a-deep-dive-into-react-agent-and-llm-s-thoughts-b1823fb266ee https://ekzhu.medium.com/gpt-4s-maze-navigation-a-deep-dive-... I think we just don't write much about moving through space and that is why reasoning about it is more limited.
- astrange 3y agoA funny thing GPT-4 is unusually good at is giving driving directions. This shouldn't work, and of course isn't 100% right, but… it's kind of right. Bard can answer questions like this, but I think it actually uses the Maps API. (It certainly says that's what it's doing.) On the other hand, every chatbot including GPT-4 is both unable to do ASCII art and unable to tell it can't do it. (Bard always shows you `cowsay` and tells you it's what you asked for, no matter what it was supposed to be.)
- bt1a 3y agoI tried so hard to make ascii art with GPT-4 api :(
- walnutclosefarm 3y agoProbably. But what seems much more interesting is to have a spatial model pre-seeded in the LLM, so that it "attaches" language to that as part of its training. Ditto for other models of the world we want the language module to be able to draw on and reason with.
- labrador 3y agoThe authors of the text the model was trained on certainly had intentions. Many of those are going to be preserved in the output.
- passion__desire 3y agoCan we say ChatGPT or its future versions would be like an instantiation of the Boltzmann Brain concept if it has internal qualia? The "brain" comes alive with the rich structure only to disappear after the chat session is over.
- SanJoseEngineer 3y ago[dead]
- jedharris 3y agoCool way of putting it. Let's run with that. A good actor can be seen as instantiating a Boltzmann Brain while on stage -- especially when improvising (as always may be needed). Maybe each of us is instantiating some superposition of Boltzmann Brains in everyday life as we wend our way through various social roles... From now on I'll listen for the subtle popping sounds as these BBs get instantiated and de-instantiated all around me... Of course a philosopher can object that (1) these BBs are on a substrate that's richer than they are so aren't "really" BBs and (2) they often leave traces that are available to them in later instantiations which again classical BBs can't. So maybe make up another name -- but a great way to think.
- passion__desire 3y agoIn a sense, we are facilitating BBs generation by providing it with rich mathematical space. The rich structure makes BBs more probable than normal randomness allows for.
- ftxbro 3y agoare our real brains boltzmann brains that have internal qualia and come alive with the rich structure only to lose it when we die
- lukev 3y agoI'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And yet, I think there's room to argue that LLMs (as currently implemented) cannot have intentions. Not because of their capabilities or behaviors, but because we know how they work (mechanically at least) and it is incompatible with useful definitions of the word "intent." Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess. Otherwise, we are in the somewhat absurd position of needing to argue that all mathematical functions "intend" to yield their result. Maybe you could go there, but it seems to be torturing language a bit, just like people who advocate definitions of "consciousness" wherein even rocks are a "little bit conscious."
- Icko 3y ago> Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess. I have two arguments against. One, you could argue that state is transferred between the layers. It may be inelegant for each chain of state transitions to be the same length, but it seems to work. Two, it may not have "states", but if the end result is the same, does it matter?
- bt1a 3y agoThat's a great way of looking at it. Comparing model weights to our brains and how we process input, you could imagine model weights as a brain frozen at time t=0. The prompt tokens are the sensory input, and the generation parameters are like twists to how the neurons pass information to each other. The token context window is like the capacity of one's working memory. At the conclusion of the last layer of processing, the output tokens are like one's subjective experience. At the least it's made me think for a moment about `stateless` and its meaning
- deleted 3y ago[deleted]
- intended 3y agoAfter having spent a ridiculous amount of effort to get LLMs to work, I am certain they are simply predicting the next token. If LLMs actually could reason, there is a much much wider set of applications where they would be actively used. The term “hallucination” does us all an injustice by propagating the idea of an anthropomorphized LLMs. Everything an LLM does is a hallucination. You and me can make out valid patterns from invalid patterns, because we have an idea of some reality. (Incidentally there are some very weird implications/ perspectives deriving from these 2 positions. Eg - If you had infinite data, would a LLM ever need to calculate?) Point being - the more intimate the use with an LLM, the more its emergent properties are non-emergent.
- dhoe 3y agoAll the whole damn universe does is move from the configuration at time t to the one at t+1. You cannot deduce from this whether the universe contains reasoning or not. We know from experience that it does, but a universe that doesn't seems possible.
- corethree 3y agoWow, where do I even start? The statement boils down a fascinating, nuanced field into an oversimplification that doesn't do justice to the complexities of machine learning, natural language processing, or, you know, human cognition for that matter! Let's talk about "predicting the next token." Sure, that's the technical framework, but what happens within that prediction is an intricate dance of probabilities, patterns, and weighted connections that come together to form something that can assist, inform, and sometimes even entertain. There's a vast landscape of difference between a machine that predicts the next word in a sentence and a machine that can draft an entire poem, answer a complicated question, or simulate conversation in a way that can sometimes pass for human thought. Is it reasoning in the way humans do? No. But to say that LLMs are "simply" predicting the next token is like saying a car is "simply" a collection of nuts and bolts that move in a certain way. It's true, but it's missing the whole picture. Just think about the implications! If this was as trivial as "predicting the next token," then why aren't we seeing this level of application everywhere? As for the term "hallucination," I get it. It's a bit anthropomorphic, sure, but language always is. We use human-centric language to describe lots of things that aren't human. We say economies are "healthy" or "sick," we say a defense in football is "stalwart." Is it perfect? No. But it gives people a way to discuss and think about complex topics, including this one. And guess what, complex discussions are how progress happens! The point about infinite data is intriguing, but let's not go off the rails here. The question isn't whether an LLM would ever "need" to calculate; it's whether the way it calculates could ever truly mimic human thought or reasoning. That's a long road we're still traveling down. But here's the kicker: just because we're not there yet doesn't mean the work that's been done is insignificant or simplistic. Emergent properties becoming "non-emergent" the more you interact with an LLM? That's the point! The more you use these systems, the more you understand their capabilities and limitations, and the better you can leverage them for tasks that are useful, interesting, or revealing. So, yeah, let's not box in what is one of the most dynamic, evolving fields right now with a one-liner that's as limiting as it is dismissive.
- prvc 3y agoA Multi-Level View of a True Scotsman.
- mgraczyk 3y agoThe reason why it isn't useful to ascribe intentions without a mechanistic explanation of intentionality is because you will incorrectly predict what the model will do in surprising ways. I think it's true that current generation LLMs could, in principle, have intentionality in the way described in the article. But they would have to be trained on many orders of magnitude more data than current models. Also AutoGPT does not work. I encourage the author to play around with it and try to get it to do something useful with high success probability.
- gwd 3y agoMy analogy for GPT-4 is this: GPT-4 is writing a novel, in which a human talks to a very smart AI. This helps me contextualize its hallucinations: if I were writing such a novel and I knew the answer to something, I would put in the correct answer; if I were writing such a novel and I didn't know the answer to something (and had no way to look it up), I would make up something plausible. From that perspective, I think multi-intentionality also works. If I write a story about Bob, then Bob (in the story) has intentions, although he's just a figment of my imagination; and when we read characters in novels, we use the imputed intentions of the characters to understand their behavior, although we know they're fictional and don't actually exist. So yes; on one level, I want to write an exciting story; on a second level, I'm simulating Bob in my head, who wants to execute the perfect robbery. On one level, GPT-4 wants to write a story about a smart AI; on a second level, the smart AI in GPT-4's story wants to win the chess game by moving the queen to put the king in check.