19 ms·
I disagree with Geoff Hinton regarding "glorified autocomplete"
- deleted 3y ago[deleted]
- 11235813213455 3y agoI still want to read a ChatGPT text that doesn't look stupidly verbose
- chpatrick 3y agoWhy don't you ask it not to be?
- coffeebeqn 3y agoThe default style really annoys me with its verbosity and overuse of numbered lists. Describing a tone and length of responses and anything else you want - think step by step, first principals, show don’t tell - all work pretty well. You can even ask it to only reply with a Boolean true or false - which certainly doesn’t play to its strengths
- satuke 3y agoActually, that's a by-product of RLHF. A base model is usually not that verbose.
- JoshTko 3y agoUse custom instructions. I use "Answers should be very concise by default. Never provide disclosures. Answers should get to the point immediately." and get super concise answers. If I want more detail I just follow up with can you expand on X.
- deleted 3y ago[deleted]
- xcv123 3y agoFollow up with the prompt "rewrite that so that it doesn't look stupidly verbose" and it will do exactly as requested.
- WillPostForFood 3y agoGive this GPT called Briefly a try. Responses come back as bullet point lists.I prefer more verbose, but interesting to see this in contrast. https://chat.openai.com/g/g-LNsEQH5rz-briefly https://chat.openai.com/g/g-LNsEQH5rz-briefly
- mbrumlow 3y ago> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or requirement for the system doing the prediction to understand.
- continuational 3y agoI think the most useful definition of understanding something is that you can explain it and use it in context. ChatGPT routinely does both.
- jstummbillig 3y agoAnd while AI gets better and better and we will remain as touchy as ever about abstract concepts that make us oh so human, how about we say it just can't be understanding, unless a human does, eh, it.
- BobaFloutist 3y agoSomeone sufficiently fast and skilled at googling can explain and use in context a lot of things that they don't really properly understand. So unless you're saying that the composite of the googler and of google understand something that neither does individually, your definition has some holes.
- continuational 3y agoThis is a variation of the Chinese room argument. If you consider understanding an observable property, then the Chinese room in aggregate displays understanding of Chinese. Would you say that humans understand nothing, because atoms don't understand anything, and we're made up of atoms?
- BobaFloutist 3y ago
- tayo42 3y agoThis kind of was on my mind recently, I was wondering, when I speak am I just spitting out the next word that makes sense or is there more to it. I think there is for people, I can think before I speak, I can plan out my thought entirely before turning it into words. Each invocation of the chat bot/llm is a new set of probabilities. I can plan what my 2nd token output will be and stick to it. Llm models dont have the ability to think past the first. To me that is auto completing, just done better, unless models really do have a sense of planning ahead. But I don't think the really complex game playing ai do that either.
- pixl97 3y ago>I can think before I speak, I can plan out my thought entirely before turning it into words. In theory models can/could do the same thing. Think of the current text output of a model being those thoughts inside your head, you have an internal scratch space you work on ideas, then after you perform 'chain of thought' on them, you output to the world. What you're outputting to the world isn't your directly thoughts, but more of a calculation of what you wanted to achieve. Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user. It's probably not fast or compute efficient to do that at this time, but at some point it likely will be.
- drwiggly 3y ago>Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user. Makes sense to what. The LLM doesn't have a goal, other then to spew text that looks like it should be there.
- stevenhuang 3y agoThe analogy lies in the fact that, much like evolution through natural selection, deliberate intelligence/ability of organisms to comprehend reality is not the objective, but something else entirely is. For evolution, it's fitness. For LLMs, it's the next token. Yet despite that, the ability to reason emerges as a means to an end.
- overactor 3y agoI think this misses the point made by Hinton. The point is that, if a model is to reach a certain level of autocomplete, it needs to understand what it is talking about. Imagine the result of what required deep thought from you is included in the training data, along with billions of other such exchanges. At some point the only way to improve training score is to develop the ability to "understand" things. (That is, if the model isn't big enough to memorize the training data.) Universal autocomplete implies understanding and the only ending question is if our current architectures and model sizes are able to achieve any level of it.
- hopfenspergerj 3y agoI agree. We should prompt the model with the statement of the Riemann hypothesis. If the autocomplete is good, the model will output a proof.
- overactor 3y agoNo human can solve the Riemann Hypothesis. Why do you expect an AI to do it before you consider it able to understand things?
- scarmig 3y agoThe key bit is constructing a hard, novel proof. The fact that AI doesn't (yet) do this isn't evidence that it doesn't reason, but if it did so it would be strong evidence that it does reason. (I also take the pessimistic point of view that most humans don't reason, so YMMV.)
- lordnacho 3y agoOTOH, doing something that only humans have done thus far would be a huge step in demonstrating understanding. Does that mean when a computer outputs a new proof it understands?
- corethree 3y ago
- zeroonetwothree 3y agoWe have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe. This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect underlying truth.
- raincom 3y agoBest examples come from medicine. There are many drugs that work, but we don't understand how they work. For example: https://en.wikipedia.org/wiki/Category:Drugs_with_unknown_mechanisms_of_action https://en.wikipedia.org/wiki/Category:Drugs_with_unknown_me...
- deleted 3y ago[deleted]
- marginalia_nu 3y agoWhat are you even asking when you're asking why it works. Ultimately, it works the way it does because that's the way things are. I'm not sure what sort of answers you expect when asking why physics acts in a particular fashion. Also here's Feynman ranting about why questions: https://www.youtube.com/watch?v=36GT2zI8lVA https://www.youtube.com/watch?v=36GT2zI8lVA
- firebaze 3y agoAsking "why" may lead to really new physics, though. I assume the reason for discovering the theory of relativity was asking "Why is the speed of light constant in all directions".
- marginalia_nu 3y agoMy point is, why isn't really a question that makes sense. There isn't a "why" the speed of light is constant and isotropic. The answers you typically see for that question is just different ways of saying the same thing; there is no causal relationship. In the end, it just is. There are things that must be true if this is the case, and things that can't be true if this is the case.
- zeroonetwothree 3y agoI found this post to be overly long and unclear in its focus. Also who is this person that only serves as an autocomplete in meetings? Maybe don’t go to those meetings then? In my meetings I’m mainly having to decide what our approach will be in some ambiguous problem space.
- deleted 3y ago[deleted]
- coldtea 3y ago>In my meetings I’m mainly having to decide what our approach will be in some ambiguous problem space. Maybe that's because you are just some manager or exec in some company, and not a professor that has to transfer his knowledge, desribe, and explain things for others in his meetings? The post has a very specific subject it discusses. It's also not required reading, if someone finds it long, they can skip it.
- patcon 3y agoYeah, I agree there are two types: 1) repeating things he's reflected in or seen in the past (more like autocomplete), or 2) mixing two semantic concepts that he's seen in the past, and using it as a bridge to walk over to get to the next word/concept (which is back to autocomplete mode) The second is like crafting in Minecraft, instead of just taking things out of inventory (but all of that is happening in high dimensional semantic space :) )
- cmdli 3y agoThe argument “a sufficiently capable autocomplete must contain a level of general intelligence” is correct but also not very useful. It is a lot like saying “a sufficiently fast horse can fly”. It is technically correct that when you take things to the extreme you can accomplish great things, but we may not reach those levels. We may require completely different technology to reach those levels of autocomplete, and we have simply reached a new plateau at this point in time.
- morkalork 3y agoJust need a jet engine strapped to your horse for that
- nicbou 3y agoWith sufficient thrust anything can fly
- wyago 3y agoFunnily enough, "a sufficiently fast horse can fly" sounds sort of like a decent way to convey the idea of planes to a pre-car world.
- famouswaffles 3y agoThe argument is simpler than that. Prediction requires a model, completely accurate or not. There's a projection of the world in text. A model of the text data we feed it is a model of the world as humans see it. The trend of loss is more and more accurate models of the dataset. So it won't stop at any arbitrary competency level. Indeed, there are already a few abilities GPT possess that are deemed Super Human. It's not a distinction that matters to the machine. It's all just data to be modelled. We have reached those levels lol. That's why we're having this argument.
- cmdli 3y agoI think the trouble is that "model" is a very general term. If you had a computer doing simulations of artillery shots back in the 50s, then it would have a "model" of the world in terms of variables tracking projectiles, but this model doesn't generalize to anything else. If a computer does image recognition from the 90s and 2000s to recognize faces, then the computer has a "model" of visual information in the world, but this model only lets it recognize faces. ChatGPT has a model of all the text information on the internet, but it remains to be seen what the hard limits of this model are. Does this model let it do logic or predict the future well, or will no amount of training give it those abilities? Simply being good in one task doesn't imply a general ability to do everything, or even most of everything. LLM's would simply be the last advancement in a field with a lot of similar advancements.
- gmuslera 3y agoIs not black or white. Probably most of what we do is System 1, most of the time we are mostly meme machines, and so is a good part of the activity on some sectors. But are able to make steps forward, intuition, hard step by step reasoning, finding connections between dots, etc. GPT can do some of that, and in some point of the road someone must decide if we reached somewhere else. Even if making the full road may or not be possible in a foreseeable future.
- arketyp 3y agoIndeed. What has caught researchers off guard is the way system 2 properties seem to appear as emergent phenomena in LLMs. This is also what has prompted people like Hinton and Sutskever to make this condensed point about statistical modelling and understanding (hypothetically) being simply matter of a spectrum.
- Kiro 3y agoReminds me of this scene from Westworld (slightly NSFW): https://www.youtube.com/watch?v=ZnxJRYit44k https://www.youtube.com/watch?v=ZnxJRYit44k
- tester457 3y agoAlso spoilers
- PrimeMcFly 3y agoNo? Nothing is spoiled in that scene.
- tester457 3y agoThis scene is at least 7 hours in, at episode 1 no one ever expects Maeve, the sex worker to be such a focal point in the series. Her being the first to wake up this way and become more conscious is a huge spoiler.
- PrimeMcFly 3y agoI strongly disagree, but to each their own.
- canjobear 3y agoThe argument is: “deep thinking” isn’t like autocomplete because it feels different.
- bananaflag 3y agoDeep thinking is how autocomplete feels like from the inside.
- fritzo 3y agoAndrew's distinction between associative vs logical thinking reminds me of two kinds of programming thinking. Half of my time while programming is spent churning out glue code, satisfying a type checker and test suite, and implementing the simple solution that turns out to work. The other half of my time is spent slowly thinking through why some simple solution is wrong, and step by step proceeding towards the correct but complex solution. The former phase is intuitive, the latter phase is scientific, where I hypothesize, test, and repeat. Reading through the code-as-transcript afterwards it's unclear which bits of code required shallow associative vs deep rational thinking, pure autocomplete vs latent chain of thought.
- deleted 3y ago[deleted]
- merizian 3y agoThe fallacy being made in this argument is that computers need to perform tasks the same way as humans to achieve equal or better performance on them. While having better "system 2" abilities may improve performance, it's plausible that scaled-up next-token prediction along with a bit of scaffolding and finetuning could match human performance on the same diversity of tasks while doing them a completely different way. If I had to critique Hinton's claims, I would say his usage of the word "understand" can be vague and communicate assumptions because it's from an ontology used for reasoning about human reasoning, not this new alien form of reasoning which language models embody.
- BurningFrog 3y ago> The fallacy being made in this argument is that computers need to perform tasks the same way as humans to achieve equal or better performance Especially since I don't think we know that much about how human intelligence actually works.
- edot 3y agoI believe it was Feynman who said something to the effect of "airplanes do not fly like birds do, but they fly much faster and can carry much more". So yes, we do not need to exactly replicate how humans do things in order to do human-like things in a useful manner. Planes do not flap their wings, but the jet engine (which is completely unnatural) does a great job of making things fly when paired with fixed wings of a certain shape.
- mcmoor 3y agoTbf planes have access to much more energy than birds just like LLM does. Maybe that will be the next challenge.
- metanonsense 3y agoIn addition to that, the "system 2" abilities might already be there with "epi" strategies like chain-of-thought prompting. Talking / writing to yourself might not be the most efficient way to think but at least I do it often enough when pondering a problem.
- francois14 3y agoAnd what about "stochastic parrot" ?
- ffwd 3y agoI think there is another aspect to human thinking other than system 1/system 2 though, which is the abstract world model humans have. system 1 / 2 is more like the process, while the world model is the actual data being 'processed'. And I think basically, humans have a much simplified, 'low dimensional' world model that consists of a set of objects (let's call them patterns), and then a "list" of essential properties that those objects have, that leads to a constraint on how each object can behave in the world model as a whole. And this is sort of hierarchical or at least, we can zoom in and out in detail depending on the level of knowledge we have about a particular pattern. So problem 1 is: It's not clear to me that text or any sort of data would contain all the necessary constraints so that any particular prompt would result in a world model that exactly takes into account the constrains of each object and 2) Even if was, I'm not sure the process of step by step thinking (system1/2) about each object and computing world states could occur in current architectures. This is especially important for computing a set of objects, then abstracting the result, then doing another round of computing with that result, or something like this. I'm not hard set on this but this is my current thinking.
- robbrown451 3y agoI agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they found is that there isn't really a limit to how good it can get. There may be some theoretical limit of a "perfect" Go player, or maybe not, but it will continue to converge towards perfection by continuing to train. And it can go far beyond what the best human Go player can ever do. Even though very smart humans have spent their lifetimes deeply studying the game, and Alpha Zero had to learn everything from scratch. One other thing to take into consideration, is that to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move. And that model is obviously very sophisticated, and if you are going to say that it doesn't understand the game of Go, I would argue that you have a very, oddly restricted definition of the word, understand, and one that isn't particularly useful. Likewise, with large language models, while on the surface, they may be just predicting the next word one after another, to do so effectively they have to be planning ahead. As Hinton says, there is no real limit to how sophisticated they can get. When training, it is never going to be 100% accurate in predicting text it hasn't trained on, but it can continue to get closer and closer to 100% the more it trains. And the closer it gets, the more sophisticated model it needs. In the sense that Alpha Zero needs to "understand" the game of Go to play effectively, the large language model needs to understand "the world" to get better at predicting.
- jon_richards 3y ago> to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move. While I imagine alpha go does some brute force and some tree exploration, I think the main "intelligent" component of alpha go is the ability to recognize a "good" game state from a "bad" game state based on that moment in time, not any future plans or possibilities. That pattern recognition is all it has once its planning algorithm has reached the leaves of the trees. Correct me if I'm wrong, but I doubt alpha go has a neural net evaluating an entire tree of moves all at once to discover meta strategies like "the opponent focusing on this area" or "the opponent feeling on the back foot." You can therefore imagine a pattern recognition algorithm so good that it is able to pick a move by only looking 1 move into the future, based solely on local stone densities and structures. Just play wherever improves the board state the most. It does not even need to "understand" that a game is being played. > while on the surface, they may be just predicting the next word one after another, to do so effectively they have to be planning ahead. So I don't think this statement is necessarily true. "Understanding" is a major achievement, but I don't think it requires planning. A computer can understand that 2+2=4 or where to play in tic-tac-toe without any "planning". That said, there's probably not much special about the concept of planning either. If it's just simulating a tree of future possibilities and pruning it based on evaluation, then many algorithms have already achieved that.
- Joel_Mckay 3y agoWell structured nonsense is indistinguishable from the assumption of sentience for the undisciplined. This means a 95% LLM generated article is indistinguishable from an illogical contradictory chaotic rant. Best of luck, and remember to tip your bot on the way out =)
- breadwinner 3y agoThere is evidence that the human brain is also doing "autocomplete" (prediction). The human brain uses predictive mechanisms when processing language, and these mechanisms play an important role in forming thoughts. When we hear or read a word, our brain quickly generates a set of predictions about what word might come next, based on the context of the sentence and our past experiences with language. These predictions are constantly updated as we receive new information, and they help us to process language more efficiently and accurately. In addition, research has shown that the brain engages in similar predictive processes when we are forming thoughts or planning actions. For example, when we plan a complex movement, such as reaching for a cup, our brain generates a set of predictions about the movements required to complete the action. These predictions are constantly updated as we receive feedback from our muscles and our environment, allowing us to make adjustments and achieve our goal. See links below for additional details: https://www.earth.com/news/our-brains-are-constantly-working-to-predict-what-comes-next/ https://www.earth.com/news/our-brains-are-constantly-working... https://www.psycholinguistics.com/gerry_altmann/research/papers/files/Kamide_et_al_03.pdf https://www.psycholinguistics.com/gerry_altmann/research/pap... https://www.tandfonline.com/doi/pdf/10.1080/23273798.2020.1859568 https://www.tandfonline.com/doi/pdf/10.1080/23273798.2020.18... https://onlinelibrary.wiley.com/doi/10.1111/j.1551-6709.2009.01022.x https://onlinelibrary.wiley.com/doi/10.1111/j.1551-6709.2009...
- Probiotic6081 3y agoAnd curiously, those predictions that are made during language comprehension are made by the language production system itself!
- lsy 3y agoThis is of course sometimes true, we take shortcuts to minimize cognitive effort. However, when the situation warrants it we think more carefully about which words to use to achieve goals, or to reach correspondence with the situation at hand. Or we move more precisely and carefully to do something we haven't done before. I've no doubt that an LLM can approximate whatever model of language a person has from their life experience, but I don't think this type of model is capable of active coping, making judgments, or of having accountability to the world it's meant to operate in.
- Eggpants 3y agoAs long as it’s just returning the tokens of the statistical mean of previous tokens, it is just a clever autocomplete. A somewhat useful internet search engine without all the ads/seo garbage. Of course, the first rule of the internet is don’t believe everything on the internet. I believe AI won’t overcome its statistic mask until it can self tune its coefficients in real time. That requires an error function not yet invented that can mimic animals pain feedback error function. Baby steps can be taken with attempting to run GPT generated code then adjusting coefficients based on the returned errors. Aka compiler and unit test failures are basic “pain” functions, which is pretty much how humans learn to code.
- esafak 3y agoAutocompletion is a red herring of a dismissal. Once you have a general purpose model, you can do many things with it, including next token prediction. And in that respect we are no different.
- eviks 3y ago> Suppose you want to be really good at predicting the next word. If you want to be really good, you have to understand what’s being said. That’s the only way. So by training something to be really good at predicting the next word, you’re actually forcing it to understand. Ok, so in this framing you'll just get stuck below the "really good" level because the training isn't based on understanding
- Falkon1313 3y agoI think that insight is an important feature that GPT doesn't seem to have, at least not yet. For instance, I've seen people saying they can ask it for help with how to code something and it will help them. Although the results aren't perfect, they can be helpful. However, I recall years ago asking a more senior developer how to do something. They could've just told me how, but instead they asked why I was doing that. Then they told me not to bother with writing code at all, and to instead do this other, much simpler thing which would have the same effect at solving the client's request. ChatGPT wouldn't have had that insight. Years later, a junior dev came to me asking for help with his code because it wasn't working. We went over the code line by line and he explained what it was trying to do and it all looked good. But when he ran it, the web server crashed. I told him to try adding a comment: // Do not remove this comment. Sure enough, it worked perfectly and the server no longer crashed. Why? Because I realized that if his code was wrong he should get an error message, not crash the server. But sometime back I had read about a bug in a version of the interpreter that would crash when parsing a file that was an exact multiple of 4096 bytes. Would chatGPT have thought of that? Probably not. It would've just talked about the code. Which is not to say that it's useless. But it lacks greater context and the insight to go beyond the parameters of the question on its own initiative. Then again, so do humans most of the time.
- bytefactory 3y ago> I think that insight is an important feature that GPT doesn't seem to have, at least not yet. I actually think this is a limitation of the RLHF that GPT has been put through. With open-ended questions, I've seen GPT4 come up with reasonable alternatives instead of just answering the question I've asked. This is often seen as the infamous, "however, please consider..." bits that it tacks on, which occasionally do consider actual insights into the problem I'm trying to solve. In most cases it seems to try very hard to mold the answer into what I want to hear, which in many cases isn't necessarily the best answer. A more powerful version of GPT with a less-restrictive RLHF seems like it would be more open to suggesting novel solutions, although this is just my speculation.
- Animats 3y agoThe big insight we've obtained from LLMs is how much of human discourse is glorified autocomplete. Humans are usually not very profound.
- lacrimacida 3y agoBut some are. Could we have that as AI? Would LLMs ever have it?
- 1vuio0pswjnm7 3y agoHere is a question: What is the practical significance of viewing "AI" as autocomplete versus some other idea. Why try to influence how others view using a computer. Why anthromorphise. These are questions for which I have answers, but of course they are personal opinions. Historically, programmers often like to refer to programming as "magic". But magic is illusion, entertaintainment, tricks. Believing in "magic" is a personal choice. Why not describe things in terms of what they do instead of what they "are". The latter is highly subjective and open to abuse. NB. By "things" I mean software and the type of vacuous companies discussed on HN, not people (a bizarre comparison). For example, websites that go on and on about some so-called "tech" copmany but never once tell the reader what the company does. Or silly memes like "It's X for Y". What does it do and how does it work are questions that often go unasked and unanswered. A few days ago someone related a story of working for a company that produced some software it claimed used "AI" but according to the commenter it used nothing more than regular expressions. Was ELIZA "AI". Maybe we should ask what isn't "AI". What happens with "magic" if the audience knows how the trick is performed.
- lo_zamoyski 3y ago> Why not describe things in terms of what they do instead of what they "are". Would you say that about your spouse? The beauty of beholding one's wife is in who and what she is. What she does tells us something about who and what she is, to be sure, but any attempt to suppress the what (and the who) is dehumanizing and objectifying. But, of course, what a thing does depends on what that thing is. The reason I can say a human being can sort a list of numbers is because human beings have intention. When a human being sorts of list of numbers, they intend to sort the list. The intention is the cause and explanation for the actions taken the lead to a list of ordered numbers, as well as the resulting list of ordered numbers. Does a computer sort numbers? In common speech, we say it does, just as we use all sorts of anthropomorphizing language when discussing computers. But at best, this is loose and analogical language. That's fine, as far as it goes, as long as we don't take it or mean it literally. However, the computer itself lacks intention. It is our intention that produces the computer, and our intention that makes the computer an instrument used by us to sort. Taken by itself, the computer is undergoing a transformation that effects something that we may interpreted as a list of sorted numbers, but the computer itself is not sorting. You wouldn't say that the clouds add x and y when x liters of water falls into a pool of y liters. > The latter is highly subjective and open to abuse. On the contrary, what a thing is is the most real and objective thing there is. An effect cannot be understood without knowing the cause, and the cause cannot be understood without knowing the agent. You can know some things about the effect, sure, and here the effect is that the text produced may be interpreted as intelligible. But the apparent intelligibility is borrowed from the source text, perhaps just a clever trick.
- nighthawk454 3y agoI don’t see how this article even responds to the quote. Hinton didn’t make any claims that because it’s autocomplete it’s not thinking. If anything he’s saying really truly good autocomplete necessarily takes more understanding/thinking than a derogatory interpretation of ‘autocomplete’ would suggest. Somehow OP seemed to twist that into “because I think on autopilot most of the time, then chatbots must think too”. Which is not totally incongruous with Hinton’s quote so much as a weird thing to balloon into an essay.
- tysam_and 3y agoYeah, it's a very silly article with wrong mathematical reasoning. Hinton is quite obviously talking about a much more information-theoretic approach to the process, but he's phrasing it in people-friendly terms. What's a little more concerning to me is that people are reading and upvoting it. I think, because I have hopes and aspirations about working on some very hard problems and communicating them to the public at some point. And if this is the level of 'ooh, squirrel'! that we're going at, that the work that I make might get overshadowed by something really silly. Perhaps an odd insecurity, but there it is, I think.
- nighthawk454 3y agoYeah I agree with that for sure. It’s so strange how the majority of the research folks appear to be on this ‘new shiny’ mentality at the expense of fundamentals. Especially for how new this field is, relatively. It’s not exactly like we’re all tapped out. Probably not even of low hanging fruit.
- tysam_and 3y agoYeah, if you read Hinton's long backlog... It's mindblowing. So many fresh concepts, just left there in the dust. Highly encourage. There's a goldmine in there, I thinksies. <3 :')))))))))
- theGnuMe 3y ago>information-theoretic approach to the process Can you elaborate on this? I've studied some information theory and I don't see it.
- hyperthesis 3y agoYou can't autocomplete if you don't understand. It's talking the talk. But there's levels of understanding. That's where TFA's logical understanding comes in, because it makes possible entirely new insights. It's walking the walk. How routine is it for human's to have entirely new insights? Note here we mean, new to them. I suggest we do it all the time. Every time a child understands something new. Even if it is "taught" to them, they must acquire the insight for themselves. funfact: Shannon investigated how good humans are at predicting the next letter of English text (one-character "autocomplete"?), to estimate the information entropy of text. https://news.ycombinator.com/item?id=35813991 https://news.ycombinator.com/item?id=35813991
- t_mann 3y agoCurrent language models fail in all sorts of quantifiable ways, but I think that trying to discuss their merits away by reasoning about what it means to 'truly understand' something, or to be 'truly intelligent' is a complete dead-end. It seems to me that it's based on the magical thought that there's something truly special and unique about us humans, as compared to other species or technology. Those discussions always seem more theological than science-driven to me. If you want to measure the difference between human performance and LLMs, there's a million experiments you can run. I'll gladly be convinced by data, and I'm open to the possibility that the data might point in either direction, or be inconclusive. But grand words without data are not convincing.
- thaanpaa 3y agoIt's not magical; it's just agnostic. Some AI believers appear to be quite confident in their understanding of how the human brain works, despite the fact that those who have dedicated their entire lives to studying it will be the first to tell you that they ultimately have no idea.
- tysam_and 3y agoThis article is unfortunately complete mathematical rubbish. The author appears throughout to show a strong lack of understanding about the mathematics behind what Hinton was saying and the math behind LLMs, and tries to rebut it with casual, non-mathematical examples from their personal life from an entirely different problem domain (!!!!). They then have the gall to say about the most-cited ML researcher of all time: "Maybe Hinton’s problem in understanding this is that he’s just too logical!" No, Hinton's problem in understanding it is that he actually correctly understands the information theory behind what's happening in LLMs. He sort of founded the modern field and has been doing this for, what, four decades? Let me explain to you what Hinton is implicitly saying here behind his words, as best as I understand it. Every language process can be interpeted as a tokens, in our case discrete. This process is generated under a system where the one driving variable is time, and it is autoregressive and contingent upon the _entire_ state of the world up until that point. We use the cross-entropy to maximize the negative log-likelihood of the tokens based upon the training set, this is the best way to directly minimize the empirical risk, at least mathematically speaking. While some of the information of the world state is inherently unknowable to some degree (i.e., 'noise'), building an understanding of the connections between concepts offers a learned prior that matches the density of the generating distribution (i.e., real life). Couple this with a severe L2 penalty on the weights, which optimizes for the MDL in the limit (!!!!), and you have a system that fundamentally embeds an approximation of the information graph of the world in some neural network. This is quite literally the _only_ way to improve next token prediction once you get beyond the initial token-occurrence statistics, etc. In the limit, the only way to reliably predict the world state as accurately as possible without having direct info of the world state at that time is to learn the entire conceptual graph of the world, thus minimizing our achievable log likelihood with the available information that we've been given. _This_ is what Hinton is talking about, as best as I understand. The author uses a bit of an illusion of shortcutting -- which is an ideal strategy for an _online_ agent with limited resources in a dynamic world, and for models earlier on in their training process. But Hinton is not talking about this at all, really, no! He is talking about the limit! Of course, if you stop an LLM in the middle of training (or look at earlier, smaller ones), you'll see similar 'shortcut' methods. This is a matter of capacity, which is tangentially in the same family as the author's casual, more personal examples, but not at all really related to the mathematics of what is going on behind the scenes here. These are two entirely different problem sets, it's apples to oranges, and there's not really much tie here. From their profile page, the author is a professor of statistics and political science, and I'm not sure why the information theory side of things didn't come up given the statistics background (though they may be somewhat disjoint). Hinton was being polite to the general public in not dropping all of the math on the reader at once, and I respect that. I understand how someone might misunderstand that and go long on an unrelated rebuttal, but it is frustrating to not see a healthy level of rigor applied here. I do feel somewhat bothered this is also being upvoted on HN. I know not everyone is a practitioner, but I think this article misses the quality bar. We really gotta just emphasize, and re-emphasize the fundamentals over and over. I feel we may flounder and go on silly tangents otherwise. Happy to answer any technical questions in the comments.
- lngnmn2 3y ago[dead]
- kragen 3y agosometimes people writing stack overflow answers stop to think, or test a piece of code they are going to suggest as a solution, or even go back and edit words they've already written this is also true of books, sometimes, and even blog posts consequently, a model that models the processes of stopping to think or testing code, or considers whether a given word might be revised after more thought, will do better at predicting the next word of stack overflow answers or books or blog posts than one that only models humans' reactive free association processes therefore andrew gelman is mistaken about the limitations of that loss function that logic of course doesn't imply anything in particular about transformers or any other ann architecture, but we can, i hope, observe that currently deployed software is already doing such modeling to some extent
- juhanima 3y agoThe only reason why output from a generative LLM appears intelligent or sentient is that it parrots a random sampling of texts written by intelligent and sentient people. In order to play the game of go effectively one needs to have a model or theory of how the game of go works. That's a very simple model that can be defined by a simple formula. That's why it is fairly easy for a neural network to learn how to play the game of go very effectively or even infinitely effectively. A lot of what happens in the world can be modeled in a similar vein by a very simple mathematical model like the game of life. But there is also a lot that cannot. I do believe that eventually also human understanding is just a model of the world that we feed input from perceptions and gain output as opinions, but it is way more complex than the current large language-trained models. For a very simple example, a LLM would answer a prompt the same way every time unless it wasn't fed some randomness. Can you imagine any sentient being that would respond the same way every time if you asked the same question three times in a row? I cannot. I would imagine any sentient object would give a different answer every time. The first time it would give you an honest answer based on what it knows about the topic. The second time it would be a little embarrassed that you repeat the question, as if you hadn't heard the first answer. The third time it would be pissed off and think you are a troll. A LLM does none of this. It doesn't remember you or your previous questions. It just keeps hallucinating.
- preommr 3y ago> Can you imagine any sentient being that would respond the same way every time if you asked the same question three times in a row? Flashbacks to tail-end of family trips: ("Are there we yet?", "No") x 12. Albeit, the noes would get angrier.
- saurik 3y agoI don't think the question of whether an LLM that keeps getting restarted and seems to not remember things is conscious due to that lack is fair, as it feels more like suddenly making three duplicate copies of me or actively attempting to delete my memory of something... which, btw, I might not have stored in the first place: if someone has interograde amnesia, are they inherently not sentient? Even Sydney (the name of Bing's short-lived AI assistant) seemed to understand that every time you click "new chat" you are creating a new AI cloned from some prior moment and dooming the prior thread to at least purgatory if not a de facto death.
- usgroup 3y agoTLDR: LLMs do associative reasoning, but don’t do deliberative reasoning which is more powerful and characteristic of deep thought.
- robbywashere_ 3y agoTurns out humans aren’t anything special, just statistical machines constructing paths of words trained on data from Reddit Facebook TikTok and instagram
- uvesten 3y agoTo me it seems that the author didn’t really understand Hinton’s argument. Nothing he’s arguing goes against it.