7 ms·
We need a more sophisticated debate, but we're not going to get it from the Financial Times. Large language models are alien. They are a new thing, not artific
by Animats 4y ago
We need a more sophisticated debate, but we're not going to get it from the Financial Times.
Large language models are alien. They are a new thing, not artificial general human-like intelligence. They sort of think, but not at all like humans, and not at all like previous concepts of artificial intelligence from science fiction.
We don't really know how they work inside. They show indications of developing some kind of internal world model. They can play chess and Othello. But we don't know what form that world model has. They do not, in a fundamental sense, know what they are doing, and are prone to serious errors because of that.
Despite this, they're really good at language-oriented tasks. This is both useful and upsetting. Humans have now had shoved in their face that many tasks considered "intellectual" can be done by systems that internally are simple, but very large. It also makes humans realize that much human intellectual activity is just autocomplete. This includes tasks previously considered high-value.
This is terrifying to humans who thought they were on top and fear becoming obsolete.
On top of this, they're at least as good at emotion, convincing, and lying as humans. Better on those than on accuracy, in fact. Nobody expected that. The major companies in the field are frantically clamping down on the emotion and lying part. The convincing part will get monetized.
If regulation is needed, it should be related to what companies are allowed to use this for. The EU does not allow computer systems to make un-appealable negative decisions about humans. (Does that still apply to the UK, or did the UK "deregulate" after Brexit?) That's a good starting point. A requirement that computer systems which interact with humans for commercial or political purposes must be identified as automated systems is a good next step. That's a starting point.
- otabdeveloper4 4y ago> They can play chess and Othello They memorized chess notation as found in chess books. (If you've ever seen those, they are just pages and pages of chess notation and nothing else.)
- consumer451 4y agoI keep reading this correct sounding idea that ChatGPT is no more than text prediction. Yesterday I put that to the test, I think. prompt: Let's play a game. Here are the rules: There are 3 boxes, red, green, blue. Red is for prime numbers. If the number is not a prime, then green for odd numbers, and blue for even numbers. I tried a dozen or so various numbers and it worked perfectly, returning the correct colored box. I then asked it to output my game as javascript, and the code ran perfectly. There seem to be two options: 1. my made up game exists in the training corpus 2. ChatGPT 3.5 is able to understand basic logic Is there a third option?
- mitthrowaway2 4y agoIt seems clear that the space of all possible texts to predict is so vast, that the only way to do effective prediction like that is to do actual "understanding". This makes sense if you think about it from a Kolmogorov complexity point of view. A program that outputs correct colors for all of those boxes, and does all the other things, based on memorization alone, will end up needing a hopelessly gigantic "chinese room" dictionary for every combinatorial situation. Even with all the parameters in these models, it would not be enough. On the other hand, a program that simply does the logic and returns the logically- correct result would be much shorter. Seems obvious so I'm not sure why this confused argument continues.
- consumer451 4y agoThanks for putting that much more intelligently than I could have. Tangent: I snooped in your profile and found the Eliezer Yudkowsky interview. I just re-posted it in hopes of further discussion and to raise my one point. https://news.ycombinator.com/item?id=35443581 https://news.ycombinator.com/item?id=35443581
- Xelynega 4y agoDoesn't your argument assume that there are no patterns inherent to language itself that can't be explained by intelligence? As an extreme example, your argument could be used to support the idea that statistical fitting is impossible. Since any process that outputs the correct answer based on a process of memorization(e.g. fitting the data) would require a hopelessly gigantic "chinese dictionary" for all the possible input:output combinations. Compression and entropy are super interesting, I don't understand this need to explain them away instead of trying to understand more about why this form of compression is so effective for language models.
- mitthrowaway2 4y agoCompression and entropy are super interesting! I'm not explaining them away; I'm pointing out that effective compression is likely to end up emulating some of the training data's generative process itself -- the patterns of thought that created text like this -- because it becomes harder and harder to "memorize" as the problem space grows. If you try to fit samples from the function 5.2*sin(17.3 x + 0.25) using a bunch of piecewise-linear lookup tables as your basis, you'll need a very large table to get good accuracy over any range! A much more effective compression of that data is the function itself. And so if your basis includes sine and cosine functions, you'll get a very accurate fit with those, very quickly and compactly. Claude Shannon built an early language model which was doing "just" statistics, which he describes in his famous paper A Mathematical Theory of Communication. Compression was exactly his goal. He builds up a Markov model to draws letters from English, including ever-longer correlations. A first order approximation draws random letters according to their frequency of occurrence in English: OCRO HLI RGWR NMIELWIS... A second order approximation is based on the probability of transition from one character to the next, for example Q will always be followed by U: ON IE ANTSOUTINYS ARE T INCTORE... The next more refined model includes trigram probabilities, for example TH will usually be followed by O or E: IN NO IST LAT WHEY CRATICT... It gradually gets more English-like, yes, but you are never going to get GPT-like performance from extending this method to character n-grams of n = 1,000,000. At this point in the paper Shannon starts over, using words and word transition probabilities, to get: THE HEAD AND IN FRONTAL ATTACK ON AN ENGLISH WRITER THAT... But even then, just extending this Markov model to account for the transition probabilities of n-grams of words won't get you to GPT. And obviously there's no interesting thought happening inside there. That's the point I'm trying to make here; when some people say it's "just statistics", they seem to be imagining a Markov model extended to be very large, which knows which words follow other words which follow other words, with various transition probabilities... But that wouldn't work well enough to correctly play spontaneously-invented logic puzzles. The problem space grows too fast for that to work. Try to create a chess program that uses a Markov model. You can do it in theory, but you'll effectively need to fit the (10^120)-gram of transition probabilities (Shannon's number). Now compare that to a chess program that does an alpha-beta search. Even monkeys on typewriters are more likely recreate Deep Blue's programming than a decent Markov-based chess program.
- JoshuaDavid 4y agoHave a look at this research paper[1] and follow-up post[2]. The upshot is that the researchers trained a model of the same shape that the main language models are to play Othello. The model started with no knowledge of the game, and was fed a bunch of move sequences (e.g. "c4 c3 d3 e3"). The researchers were then able to take that model, and figure out, by looking only at the activations, what the board state was. Changing the activations so that they modeled a different board state caused the model to play moves that made sense with the new board state, and did not make sense with the original board state. And then the follow-up research[2] established that, not only can you determine the board state by looking at the activations, but that you can trivially do so. Specifically, if you look at the residual stream after layer 4, you can build a linear classifier for whether each of the 64 squares is empty, and whether each of the 64 squares contains a token owned by the player whose turn it is. If you bump the values in the direction implied by each of those classifiers, you can make the model output moves in the same way as if the tokens on those squares had the opposite color. There's even a colab notebook[3] you can play with. That research was on Othello, not chess, but I'm pretty sure that "LLMs are able to develop models of the world if doing so helps them predict the next token better" generalizes to chess too. [1] https://arxiv.org/pdf/2210.13382.pdf https://arxiv.org/pdf/2210.13382.pdf [2] https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... [3] https://colab.research.google.com/github/likenneth/othello_world/blob/master/Othello_GPT_Circuits.ipynb https://colab.research.google.com/github/likenneth/othello_w...
- zvmaz 4y ago> Humans have now had shoved in their face that many tasks considered "intellectual" can be done by systems that internally are simple, but very large. It also makes humans realize that much human intellectual activity is just autocomplete. Could you give an example? I had a conversation with it about the computational model of the mind, and at one moment, I asked it to tell me if Chomsky thinks that the model is complete (i.e., describes all aspects of the mind), and provide citations. It did, with citations and references; I checked each one of them: ALL of them were hallucinations. I asked it what was the percentage of surgeries done under laparoscopy in France, with references. It sure gave me the percentage and the references; I checked: ALL of them were not correct. I don't know if it makes humans realize that much human intellectual activity is just autocomplete; it certainly did not make me think so.
- ShamelessC 4y ago> ALL of them were hallucinations This is a known issue. How about asking it to generate code? I don’t really trust that you won’t just deliberately find an example that doesn’t work and say “see?! It’s so dumb!!!” And further this type of reaction reeks of the sort of insecurity that the parent comment was suggesting. You _need _ these things to be unintelligent. It threatened your individual skill set, so you prompted it like a human rather than deliberately leaning into its strengths and “assuming good faith” (whatever that means for an LLM). Fwiw, plugins will likely solve this issue. As well more advanced models.
- zvmaz 4y agoI asked the author of the comment to give me an example of the many "intellectual" tasks that can be done by "systems" that are "internally simple" (what does "simple" mean?) but "large" (with or without human intervention?). I then pointed to the known (and serious) issue of hallucinations; how does this all fit with the author's "reductionist" model of much human thought being "just autocomplete"? I don't feel threatened as my previous comment suggests; I just think that human thought is not "just autocomplete".
- 4y ago
- famouswaffles 4y agoThere's nothing simple about prediction. Stop and think about what it would take to be able to predict what an intelligent being would say/react to any arbitrary input. Predicting the next token only sounds simple. It says nothing about what the model has learnt in the process.
- fnovd 4y ago>We don't really know how they work inside. I keep reading this, and my response is so what? Do you have a ball sitting handy somewhere? Pick it up, throw it in the air, and then catch it. How did you do that? Can you regurgitate all of the equations used to define how objects move, how you determined the approximate mass of the ball, how you calibrated your muscles to apply a specific amount of force and then how you orchestrated the whole thing? Of course not, that's ridiculous. You don't need to know physics or even be literate in order to throw and catch a ball. Your brain "knows" how to do it but that knowledge is opaque and the part of your brain that talks to people can't actually communicate any of the nitty-gritty. So why does it matter that an AI can't tell us how it does what it does? We're the smartest things we know and we can't even do it. We can give rationalized mockups of a lot, sure, but the AI can, too. Why do we think introspective clairvoyance is a requirement of AGI?
- joenot443 4y agoThe reason LLMs are novel in this regard are because they are a software model which can't interrupted, inspected, and understood during its execution. Any other software system running on your machine has a state which at any level of abstraction, from an HTML checkbox to a single LED pixel, can be understood and reasoned with. It is a system we can step through in a discrete and understandable way because it's a system we created. My understanding is that we cannot do this with an LLM. There isn't a callstack which can be followed to trace why we returned 'foo' instead of 'bar', just the oft-mentioned matrix of floats corresponding to tokens. Perhaps not everyone sees it this way, but I think this separation, of a system which we can conceptualize to one we cannot, is a big deal.
- fnovd 4y agoIt's a big deal as far as impact, sure, but I think it's also OK for us to abandon the need for complete control. Because, really, that's what this is about: we're worried that if we can't understand what's happening, we can't control it. Personally I think that's just fine. It's a different class of thing but that's also OK. Do we even really know all that we should about how to manufacture chips? My understanding is that we rely heavily on experiments and heuristics. I think with complexity that's just how things are, sometimes. And again, going to the human-throwing-a-ball metaphor, maybe generalized intelligence actually requires introspective opaqueness. Maybe it's some cosmic law we don't understand: the better a system is at handling open-ended issues the less it can be systematically understood. I just think the utility of application is so, so much more important than our inability to know down to the last bit how a given LLM works. If it works, and delivers value, then we just don't need the why. We can and should try to understand, but we can also accept that we won't always get all of the answers we want. I mean, why does anyone do anything? Ask someone to explain why they did everything they did today and I'm sure a lot of what they'll tell you is made up or just plain wrong. Humanity seems to be just fine despite all that. Why do we expect our apprentices to be different?
- IshKebab 4y ago> We need a more sophisticated debate, but we're not going to get it from the Financial Times. > > Large language models are alien. They are a new thing, not artificial general human-like intelligence. They sort of think, but not at all like humans, and not at all like previous concepts of artificial intelligence from science fiction. This is more or less exactly what the FT article said so I think it's a bit unfair to criticise them as you did: > The truth is that machine learning systems are nothing like us in the way they are engineered, but they are no less significant for it. To take just one example: the fact that non-human AI systems, perhaps with faces and voices, will soon be able to participate in political debate in a sophisticated way is likely to be more important for the future of democracy than the fact that they do not “think” like humans. Indeed, asking whether a machine learning system can “think” like a human is often as useful as asking whether a car can gallop as fast as a horse. Honestly that's a lot more sophisticated than most of the debate on HN has been. There are a disappointing number of people here who think that it's "just predicting words" or that it can't "think" because it's a machine and thinking is somehow reserved for people with squashy brains (and souls presumably).
- __loam 4y agoIt's pretty generous to say that the models are thinking, and to denigrate brains as merely squishy. During every period of technological progress, people have used the most complicated technology of the day to try and describe brains, like clockwork or hydraulics. This is no different. Brains are special. We know some of their principles but we still don't understand how cognition occurs given those principles. They are, of course, physical machines. They must operate on physical principles. But comparing what is basically a reflex response from an unthinking ml model to them? Come on.
- IshKebab 4y ago> It's pretty generous to say that the models are thinking Why? Only if you pick an especially narrow definition of "think". > Brains are special. Only in that their "algorithm" is better than current ML. As far as we know there's no "soul" or physical process that a computer couldn't theoretically also perform (please don't link that quantum microtubule nonsense). > to denigrate brains as merely squishy They are merely squishy computers. Very very complex and big computers, sure. But check out OpenWorm for example. Same thing, just not squishy.
- YeGoblynQueenne 4y ago>> We don't really know how they work inside. Yes, we do. We know their design principles, their operation principles, the designs of specific instances, and everything else that has anything to do with "how they work inside". We know how they work inside. We know how they work inside and out. What we can't do is predict their output, given some input (i.e. a prompt). That's because Large Language Models are so, well, large, and complex, that it's impossible to reproduce their calculations without ... a Large Language Model. In the same way, we can't predict the output of a Random Number Generator. Very much like a large language model, we know how an RNG works, but if we could predict its behaviour, then it wouldn't be working correctly. But why is the inability to predict the behaviour of a complex system reason to imagine all sorts of mysterious and fantastical things happening inside the system, such as it "thinking", or "developing some kind of internal world model"? Those are not things that language models, small or large, are designed to do, and there is no obvious reason why they should be able to do them. There is no reason to assume they can do them either: the fact that they are models of language, trained to reproduce language, and nothing else, suffices to explain their observed behaviour. There are no mysteries of intelligence, or artificial intelligence, to uncover in the study of large language models' behaviour. I am perfectly well aware that there are many people who absolutely want there to be such mysteries, there, and that will overlook everything we know about how language models actually work, to be able to fantasise about those mysteries. But those are fantasies. They are superstitions. Superstitious beliefs about an artifact of technology, that we have created ourselves, and that some of us have now chosen to revere and wonder at as if it was an artifact of an alien civilisation fallen from the sky. Also, on a personal note: not you too, Animats... come on, you're a knowledgeable one. Don't just get swept up in the crowd's madness like that.
- auggierose 4y agoJust because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that. Yes, we know the definition of GPT. But that it is so good at doing what it does, with such a freakingly simple method, one word after another, is emergent behaviour, and wasn't known to us before seeing it with our own eyes. We still don't know WHY it is so good at it. If we did, maybe we could construct one a hundred times smaller.
- __loam 4y agoFT is one of the best publications out there.