7 ms·
GPT is great and useful. But it reminds me of young kids who are verbally strong. Some 5-year-olds are amazing talkers, giving you the impression they have a v
by wouldbecouldbe 3y ago
GPT is great and useful.
But it reminds me of young kids who are verbally strong. Some 5-year-olds are amazing talkers, giving you the impression they have a very sophisticated intelligence at first glance. But the more time you spend with them, you realize they don't understand but imitate.
Although the kids will grow up and understanding naturally develops, many people seem to assume LLMs will take the same path, which seems less likely.
Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. Its language capabilities might be useful, but AGI won't be possible without a self-correcting model that can actually reason. Even Altman hinted at that.
Interestingly enough, OpenAI seems to go all in on GenAI, or at least spend a significant part of its resources on it. Generating videos seems like a cool feature, and perhaps commercially viable at some point, but it doesn't seem to bring them closer to AGI. Not sure if that's shiny object syndrome, pressure from Microsoft, surfing the wave, or a big masterplan I'm not seeing.
GenAI will continue to have its purposes, some applications will become more useful with more specific models and more developed reinforcement models on top of it. So there is definitely a commercial use for those. But it will keep getting stuck on the accuracy part, the same place where AGI development is stuck.
- taylorius 3y agoThat phenomenon goes a lot broader than young kids. Verbal fluency and rhetoric has almost become a sort of personal marketing technique. Sounds impressive at first, but doesn't necessarily correspond to anything of substance behind it. My tentative hypothesis is that it's a symptom of the "optimization" of educational content on the internet.
- ben_w 3y agoVarious times I hear someone described as "an idiot's idea of a smart person". I can vibe with that description of GenAI: it does all the things that, as a kid, I regarded as synonymous with intelligent. And yet, now we have it, it's at the level of an intern… an intern in a huge number of fields all at once, so very general and still useful, but an intern nonetheless.
- lamontcg 3y agoThere's a whole lot of people on forums like HN that like to write in this intellectual style with a bit more "Queens English" sounding idioms and some $10 words thrown in. They increasingly sound like they're Generative AI. Maybe a useful side effect of GenAI might be that people will start speaking more bluntly so that they sound more human.
- ben_w 3y agoThe AI is mimicking the humans, so if the models had been trained predominantly on nice efficient (blunt) German culture, it would be the other way around. ("Queen's English" and "$10 words", in the same list?)
- tim333 3y agoThe style reminds me of high school essays. More advanced writers deviate from that to make it entertaining.
- closewith 3y ago> Which is ridiculous, it's just an incredible token predictor. I'm not sure I'm not just a mediocre token predictor.
- exe34 3y agoI've never met anyone who didn't just say the next word in a sequence. So far nobody's been able to produce a complete sentence in one go!
- relaxing 3y agoStop.
- exe34 3y agoI'm sorry, as a large language model, I can't do that Dave.
- ben_w 3y agoNow you have. And I don't just mean the one-word example in the sibling comment: I think a complete sentence first, and then generate the words. The part of my mind I could describe as an "inner voice" does not need to turn this thought into words… but (if you will excuse me giving you an anthropomorphic personification of a homunculus) that inner voice gets annoyed if I don't let it "read out" the words of that sentence.
- exe34 3y agoThat's how you claim you came up with that sentence. Your answer basically reverts back to the Cartesian theater, you even bring up the homunculus yourself. There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us. Ask chat gpt something equally unknowable and it will also earnestly confabulate a similarly magical-thinking answer. I wonder if you two are really so different! Either way, what you claim cannot be an explanation, it simply pushes the problem one step back. However it is that you came up with the sentences, when it came to saying out loud the words, or typing them, you still typed one token after the other. Sure you had a buffer that kept getting filled up faster than you fingers could type or swipe, but ultimately you still produced the next letter/word/sentence. Entire sentences can be tokens. BPE isn't the only encoding.
- yard2010 3y agoAren't human beings just token predictors with very specific rules though
- notahacker 3y agoThe tokens in question didn't exist for most of human evolution, so I'm going to say a fairly confident no.
- exe34 3y agoSo if a mechanism was developed for the LLM to add more tokens over time, you would retract your objection?
- notahacker 3y agoNope (afaik LLMs are already capable of generating novel tokens parsing novel tokens generated by [other] LLMs) The point is that humans and their ancestors had been self-replicating, long-running biochemical processes for millions of years before language (and related abstractions) were even a thing. So I don't see the impressive performance of token-prediction algorithms at associating tokens representing human abstraction with each other in ways humans discern as meaningful as offering much insight into human behaviour, never mind giving a complete explanation for how it happens. The claim that the way we navigate the world boils down to "just" predicting the next token feels even more extraordinary in the context of prediction being something we're pretty bad at.
- ben_w 3y agoTokens don't have to be units of language, they can be any arbitrary chunk of an input stream: https://arxiv.org/abs/2006.03677 https://arxiv.org/abs/2006.03677 (I'd be a bit surprised if our brains are transformers; from what I understand, we're much more sample-efficient than most of our AI so far).
- notahacker 3y agoBut whilst tokens in the form of pixel values might actually map quite well to sensory input from rods and cones, why on earth would we assume that everything that goes on in there maps neatly to the concept of "predict future tokens"? And where are the analogues for stuff like emotions, evolutionarily useful chemical processes which actually make us worse at prediction?
- hackerlight 3y ago> Lot's of devs here, but als in the media, believe GPT is the precursor for AGI. Which is ridiculous, it's just an incredible token predictor. Geoffrey Hinton and Ilya Sutskever have said it is impossible to predict the next token without solving reasoning and intelligence. I'm not entirely sure how they justify that. If I have to guess, they would say there's no hacky shortcut that the neural net can cheat with to solve this problem, actual intelligence is the only way, and so that's what we get. My 2c: The thing that LLM critics miss is that LLMs have learned useful representations of complicated abstract concepts. Analogous to edge detectors -> object detectors in vision networks, LLMs have circuits that correspond to human-understandable concepts. I have no idea how we're going to solve hallucinations, but it's obvious to me that we've already figured out a key milestone on the way to AGI: How to encode abstract concepts into neural nets. It's now just a matter of how to use those representations downstream in a way that doesn't hallucinate.
- candiodari 3y agoEver thought about ... let's say you were God. You wanted to design intelligence, to control insects, animal bodies and eventually humans. What would it need to do? Well, you'll quickly find that bodies require constant control to survive. Whilst an animal's body can survive for months with an animal's mind operating at 1% capacity, if the output of the mind truly stops, the body dies. The heart, respiration, the circulatory system (deciding where the blood goes), ... all need to be controlled with some amount of intelligence for them to function at all. Even just to remain alive, they need intelligence. They don't work on reflexes alone. For mammals, total loss of control leads to fatal injury ... in seconds. Without control, the heart will either stop or (wrong order of muscle firing) be totally ineffective at building pressure in the circulatory system. This will kill oxygen transfer through the blood-brain barrier as soon as the pressure drop reaches the brain, which will be much less than a second, this leads to further loss of control in 2-3 seconds. Irreperable damage sets in before 20 seconds pass, death (permanent and total loss of control) occurs in humans after about 1 minute 30 seconds. The plus side of this seems to be that, when that control is there, mammal bodies are far more efficient than a "passive" robot can be (any robot you can turn off -safely- at any time is a "passive" robot, the big counterexample is a flying drone, which will incur damage and may injure others if turned off or it loses control) So "predict the next token" is really THE most important function of the brain. Second, it is incredibly important to KEEP predicting the next token, to keep going, and keep going, and keep going, no matter how stupid the output, because literally any output will give a result superior to no output at all. No matter the damage, no matter how terribly wrong things are going, it must keep going further.
- garyclarke27 3y agoYou're correct, I have grown to be very suspicious of smooth talkers. eg UK PM Richie Sunak and London Mayor Sadiq Khan are both extremely eloquent and they are both utterly useless in their jobs with zero ideas or imagination as to solving the problems they face. Contrast Elon Musk not a very smooth talker but brilliant at solving real-world problems.
- ben_w 3y ago> Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. Its language capabilities might be useful, but AGI won't be possible without a self-correcting model that can actually reason. Even Altman hinted at that. I think part of this is that all three letters of the initialism mean different things to different people. To me? The domain is broad rather than narrow, so it's got the "G" of "AGI". Definitely agree it is not a complete solution, and thus that reasonable people will say "no Ben this isn't general what are you talking about". I also definitely agree isn't superhuman, but I think superhuman ability across all of its domain is an unreasonable requirement for AGI and what I would instead call that ASI… but I'm also a linguistic descriptivist, so it doesn't matter if the consensus is unreasonable[0] just what the consensus actually is. Also: my understanding from "Fine-Tuning Language Models from Human Preferences"[1] (in particular the summary version of the bug, "The True Story of How GPT-2 Became Maximally Lewd"[2]) is that these models are capable of self-correcting behaviour — we just don't do that live because that makes it much harder to research and to use (every time it even seems like the ChatGPT model changes, someone complains). > But it will keep getting stuck on the accuracy part I suspect otherwise. Some — not all, but some — domains have easily testable results. In these domains, the generated output can be quickly tested, and used to update the model. This is already done with human preferences, but I'm thinking more along the lines of a coding assistant which gets feedback from a compiler, or a maths assistant which gets feedback from… maths isn't my field, I'm just going to say MATLAB or the Wolfram Language and hope these are both as highly regarded as Photoshop. [0] this happens a lot: tautologies such as "ATM machine"; or redefinitions such that "PC" excludes MacOS and all phones and tablets; that Elon Musk can only be called "African-American" by pedants (including me, despite being a descriptivist); exactly how effective and how altruistic is "Effective Altruism" etc. [1] https://arxiv.org/pdf/1909.08593.pdf https://arxiv.org/pdf/1909.08593.pdf [2] https://forum.effectivealtruism.org/posts/5mADSy8tNwtsmT3KG/the-true-story-of-how-gpt-2-became-maximally-lewd-1 https://forum.effectivealtruism.org/posts/5mADSy8tNwtsmT3KG/...
- wouldbecouldbe 3y agoBeing superhuman is the defacto definition of AGI leading to singularity, mainly self-correcting is main requirement, and that's why they fear it. If you just change the definition of AGI to what GPT does currently then indeed by your definition it's AGI. But that's not what OpenAI & musk mean with AGI.
- aurareturn 3y ago>Interestingly enough, OpenAI seems to go all in on GenAI, or at least spend a significant part of its resources on it. Generating videos seems like a cool feature, and perhaps commercially viable at some point, but it doesn't seem to bring them closer to AGI. Not sure if that's shiny object syndrome, pressure from Microsoft, surfing the wave, or a big masterplan I'm not seeing. Building AGI will be extremely expensive. They need ways to stay alive and fund it. I would compare it to Tesla building the Roadster to fund S, and S revenue to fund the next car, etc.
- wouldbecouldbe 3y agoI kind of feel like reasoning models will be less based upon the amount of parameters, but I could be completely wrong.
- ben_w 3y agoWe aren't sure we know how to do it at all yet[0], so there's going to be a lot of research going from here to there. Also, we don't know how to do it safely, which they also care about, and also takes a lot of research as quite a lot of us don't even agree on the question. All that research is going to be expensive. [0] It's long been possible to make an AI which can learn any function, but all the current ones need too much training data, and we don't know if we can reach economic usefulness in all human domains with the currently existing data even if we ignore that so many data holders responded to ChatGPT and Midjourney/Stable Diffusion by locking away all their data.
- roenxi 3y ago> Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. What test (or facet, for that matter) of intelligence/AGI can't be modelled as a token prediction exercise? Bearing in mind that quadriplegics are intelligent even though they have lost control of their arms and legs.