6 ms·
Did we ever think LLMs were a path to AGI...? AGI is friggin hard, I don't know why folks keep getting fooled whenever a bot writes a coherent sentence.
by erhaetherth 3y ago
Did we ever think LLMs were a path to AGI...? AGI is friggin hard, I don't know why folks keep getting fooled whenever a bot writes a coherent sentence.
- RuggedPineapple 3y agoIt's mostly a thing among the youngs I feel. Anybody old enough to remember the same 'OMG its going to change the world' cycles around AI every two or three decades knows better. The field is not actually advancing. It still wrestles with the same fundamental problems they were doing in the early 60s. The only change is external, where computer power gains and data set size increases allow brute forcing problems.
- deleted 3y ago[deleted]
- Eji1700 3y agoI'd say the biggest change is the quantity of available CATEGORIZED data. Tagged images and what not has done a ton to help the field. Further there are some hybrid chips which might help increase computing power specifically for the matrix math that all these systems work on. But yeah, none of this is making what people talk about when they say AGI. Just like how some tech cult people felt that Level 5 self driving was around the corner, even with all the evidence to the contrary. The self driving we have (or really, assisted cruise control) IS impressive, and leagues ahead of what we could do even a decade or two ago, but the gulf between that, and the goal, is similar to GPT and AGI in my eyes. There are a lot of fundamental problems we still don't have answers to. We've just gotten a lot better at doing what we already did, and getting more conformity on how.
- concordDance 3y ago> The field is not actually advancing. Uh, what do you mean by this? Are you trying to draw a fundamental science vs engineering distinction here? Because today's LLMs definitely have capabilities we previously didn't have.
- oska 3y agoThey don't have 'artificial intelligence' capabilities (and never will). But it is an interesting technology.
- concordDance 3y agoThey can be the core part of a system that can do a junior dev's job. Are you defining "artificial intelligence" is some unusual way?
- oska 3y agoI'm defining intelligence in the usual way and intelligence requires understanding which is not possible without consciousness I follow Roger Penrose's thinking here. [1] [1] https://www.youtube.com/watch?v=2aiGybCeqgI&t=721s https://www.youtube.com/watch?v=2aiGybCeqgI&t=721s
- concordDance 3y ago> intelligence requires understanding which is not possible without consciousness How are you defining "consciousness" and "understanding" here? Because a feedback loop into an LLM would meet the most common definition of consciousness (possessing a phonetic loop). And having an accurate internal predictive model of a system is the normal definition of understanding and a good LLM has that too.
- FeepingCreature 3y agoNo, you're not supposed to actually have an empirical model of consciousness. "Consciousness" is just "that thing that computers don't have".
- wildermuthn 3y agoIt’s cool to see people recognizing this basic fact — consciousness is a prerequisite for intelligence. GPT is a philosophical zombie.
- Adunai 3y agoAs an outsider, I can talk to AI and get more coherent responses than from humans (flawed, but it's getting better). That's tangible, that's an improvement. I for one don't even consider the Internet to be as revolutionary as the steam engine or freight trains. But AI is actually modifying my own life already - and that's far from the end. P.S. I've just created this account here on Hacker News because Altman is one of the talking heads I've been listening to. Not too sure what to make of this. I'm an accelerationist, so my biggest fear is America stifling its research the same way it buried space exploration and human gene editing in the past. All hope is for China - but then again, the CCP might be even more fearful of non-human entities than the West. Stormy times indeed.
- torginus 3y agoLLMs have changed the world more profoundly than any technology in the past 2 decades, I'd argue. The fact that we can communicate with computers using just natural language, and can query data, use powerful and complex tools just by describing what we want is an incredible breakthrough, and that's a very conservative use of the technology.
- foldr 3y agoI don't actually see anything changing, though. There are cool demos, and LLMs can work effectively to enhance productivity for some tasks, but nothing feels fundamentally different. If LLMs were suddenly taken away I wouldn't particularly care. If the clock were turned back two decades, I'd miss wifi (only barely available in 2003) and smartphones with GPS.
- peignoir 3y agoYou need time for inertia to happen, I’m working on some mvps now and it takes time to test what works what s possible what does not…
- FabHK 3y agoIndeed. The "Clamshell" iBook G3 [0] (aka Barbie's toilet seat), introduced 1999, had WiFi capabilities (as demonstrated by Phil Schiller jumping down onto the stage while online [1]), but IIRC, you had to pay extra for the optional Wifi card. [0] https://en.wikipedia.org/wiki/IBook#iBook_G3_(%22Clamshell%22) https://en.wikipedia.org/wiki/IBook#iBook_G3_(%22Clamshell%2... [1] https://www.youtube.com/watch?v=1MR4R5LdrJw https://www.youtube.com/watch?v=1MR4R5LdrJw
- qeternity 3y agoI am massively bullish LLMs but this is hyperbole. Smartphones changed day to day human life more profoundly than anything since the steam engine.
- torginus 3y agoI'm kinda curious as to why you think that's the case. I mean, smartphones are nice, and having a browser, chat client, camera etc. in my pocket is nice, but maybe I have been terminally screen-bound all my life, but I could do almost all those things on my PC before, and I could always call folks when on the go. I've never experienced the massively life changing effects of having a smartphone, and (thankfully) none of my friends seem to be those people who are always looking at their phones.
- hyperthesis 3y agoDeep learning was an advance. I think the fundamental achievement is a way to use all that parallel processing power and data. Inconceivable amounts of data can give seemingly magical results. Yes, overfitting and generalizing are still problems. I basically agree with you about the 20 year hype-cycle, and but when compute power reaches parity with human brain hardware (Kurzweil predicts by about 2029), one barrier is removed.
- somewhereoutth 3y agoHuman and computer hardware are not comparable, after all even with the latest chips the computer is just (many) von Neumann machine(s) operating on a very big (shared) tape. To model the human brain in such a machine would require the human brain to be discretizable, which, given its essentially biochemical nature, is not possible - certainly not by 2029.
- hyperthesis 3y agoIt depends on the resolution of discretization required. Kurzweil's prediction is premised on his opinion of this. Note that engineering fluid simulation (cfd) makes these choices in discretization of pde's all the time, based on application requirements.
- fsloth 3y agoThis time around they’ve actually come up with a real productizable piece of tech, though. I don’t care what it’s called, but I enjoy better automation to automate as much of the boring shit away. And chip in in coding when it’s bloody obvious from the context what the few lines of code will be. So not an ”AI”, but closer to ”universal adaptor” or ”smart automation”. Pretty nice in any case. And if true AI is possible, the automations enabled by this will probably be part of the narrative how we reach it (just like mundane things like standardized screws were part of the narrative of Apollo mission).
- antifa 3y ago> Anybody old enough to remember the same 'OMG its going to change the world' cycles around AI every two or three decades Hype and announcements, sure, but this is the first time there's actually a product.
- dragonwriter 3y ago> Hype and announcements, sure, but this is the first time there's actually a product. No, its not. Its just once the hype cycle dies down, we tend to stop calling the products of the last AI hype cycle "AI", we call them after the name of the more specific implementation technology (rules engines/expert systems being one of the older ones, for instance.) And if this cycle hits a wall, maybe in 20 years we'll have LLMs and diffusion models, etc., embedded lots of places, but no one will call them alone "AI", and then the next hype cycle will have some new technology and we'll call that "AI" while the cycle is active...
- Closi 3y agoMainly because LLMs have so far basically passed every formal test of ‘AGI’ including totally smashing the Turing test. Now we are just reliant on ‘I’ll know it when I see it’. LLMs as AGI isn’t about looking at the mechanics and trying to see if we think that could cause AGI - it’s looking at the tremendous results and success.
- gardenhedge 3y agoSince ChatGPT is not indistinguishable from a human during a chat, is it fair to say it smashes the Turing test? Or do you mean something different?
- rayeigenfeldt 3y agoDid you perhaps mean to say not distinguishable?
- NoOn3 3y agoChatGPT is distinguishable from a human, because ChatGPT never responds "I don't know.", at least not yet. :)
- ben_w 3y agoIt can do: https://chat.openai.com/share/f1c0726f-294d-447d-a3b3-f664dcc74de4 https://chat.openai.com/share/f1c0726f-294d-447d-a3b3-f664dc... IMO the main reason it's distinguishable is because it keeps explicitly telling you it's an AI.
- rezonant 3y agoThis isn't the same thing. This is a commanded recital of a lack of capability, not that its confidence in it's answer is low. For a type of question the GPT _could_ answer, most of the time it _will_ answer, regardless of accuracy
- NoOn3 3y agoI just noticed that when I ask really difficult technical questions, but for which there is an exact answer, It often tries to answer plausibly, but incorrectly instead of answering "I don't know". But over time, It becomes smarter and there are fewer and fewer such questions...
- mjan22640 3y ago> Estimated on the basis of five subtests, the Verbal IQ of the ChatGPT was 155
- andrepd 3y agoAre you kidding? Have you seen the reactions since ChatGPT was released, including in this very website? You'd think The Singularity is just around the corner!
- concordDance 3y agoLLMs definitely aren't a path to ASI, but I'm a bit more optimistic than I was that they're the hardest component in an AGI.
- golol 3y agoLLMs are the first instance of us having created some sort of general AI. I don't mean AGI, but general AI as in not specific AI. Before LLMs the problem eith AI was always that it "can only do one thing well". Now we have something on the other side: AI that can do anything but nothing specific particularly well. This is a fundamental advancement which makes AGI actually imaginable. Before LLMs there was literally no realistic plan how to build general intelligence.
- stuaxo 3y agoLLMs are not any kind of intelligence, but it can work to augment intelligence.
- darkerside 3y agoHow smart would any human be without training and source material?
- knicholes 3y agoI think the boy of Aveyron answers that question pretty well.
- darkerside 3y agoThanks for the reference. My takeaway from reading up on him is, not very smart at all.
- Jensson 3y agoSmart enough to make weapons, tame dogs, start fires and cultivate plants. Humans managed to do that even when most of their time was spent gathering food or starving.
- darkerside 3y agoNobody cares about making an AI with basic human survival skills. We could probably have a certified genius level AI that still couldn't do any of that because it lacks a meaningful physical body. If we wanted to make that the goal instead of actual meaningful contributions to human society, we could probably achieve it, and it would be a big waste of time imo.
- heavyset_go 3y agoRead the original ChatGPT threads here on HN, a lot of people thought that this was it.
- discordance 3y agoHow do you know AGI is hard?
- howrar 3y agoEverything is hard until you solve it. Some things continue to be hard after they're solved. AGI is not solved, therefore it's hard.