9 ms·
Scaling will never get us to AGI
- jqpabc123 3y agoThis is why driverless cars are still just demos, and why LLMs will never be reliable. It hard to build something that is yet to be even logically defined. Hoping that AGI will somehow just "emerge" from an inert box of silicone switches (aka a "computer" as we currently know it) is the stuff of movie fantasy ... or perhaps religion.
- tivert 3y ago> ...is the stuff of movie fantasy ... or perhaps religion. IMHO, something like "movie fantasy" constitutes the core belief of a great many software engineers and other so-called rational people in the tech space.
- galangalalgol 3y agoDriverless cars are here, without liability issues they would already be widespread. But scaling isn't what got them there. It was tons of labeled data, both from rules based implementations and shadowing human drivers. The ability to generalize completely outside training data isn't that common among humans IME. That is a high bar. How many of us have done so without at least drawing an analogy to some other experience we have had? Truly unique thinkers aren't that commonplace. I myself could have probably just asked an LLM what I would say in this post and gotten pretty close...
- rchaud 3y ago> Driverless cars are here, without liability issues they would already be widespread In other words, they're in a lab somewhere undergoing perpetual test runs so they don't kill people, so not generally available. It is like saying that email was available in the '80s, you just couldn't message anybody outside of the university network.
- naasking 3y agoYou can order driverless taxis right now in some cities: https://www.axios.com/2023/08/29/cities-testing-self-driving-driverless-taxis-robotaxi-waymo https://www.axios.com/2023/08/29/cities-testing-self-driving...
- MatthiasPortzel 3y agoThat article is discussing Cruise. Since it was published, a Cruise car dragged a pedestrian 20 ft, it came out that they need remote intervention every few miles, the CEO resigned, and they stopped doing rides anywhere. Waymo remains available to my knowledge.
- loire280 3y agoHowever, they require constant human monitoring and interventions from those humans every few miles to work properly - and it's not clear that the need for those interventions will be solved with more training data. https://www.theverge.com/23948708/cruise-robotaxi-suspension-trust-remote-assist https://www.theverge.com/23948708/cruise-robotaxi-suspension...
- tivert 3y ago> However, they require constant human monitoring and interventions from those humans every few miles to work properly It's a little-known fact that AI means "Actually, Indians." https://boingboing.net/2024/04/03/amazons-ai-powered-just-walk-outcheckout-option-turns-out-to-be-1000-workers-watching-you-shop.html https://boingboing.net/2024/04/03/amazons-ai-powered-just-wa...
- minitoar 3y agoNo, I think he meant they are literally here. I see one everyday.
- parpfish 3y ago> But scaling isn't what got them there. It was tons of labeled data, both from rules based implementations and shadowing human drivers. Isn't the second half of that saying that scaling up the dataset was what got them where they are? When I hear about "scaling up" models, I think there are two parts of it: 1) use the same architecture, but bigger 2) make the training data bigger to make use of all those new parameters. So when I hear about something that's not scaling, it would require some sort of fundamental change to architecture or algorithms.
- staticman2 3y ago> Driverless cars are here, without liability issues they would already be widespread. In the sense that if you were allowed to run over anyone without legal consequences lots of tech enthusiasts would sign up for this experience?
- deleted 3y ago[deleted]
- iwsk 3y agoGeneral intelligence somehow just emerged from dirt and water + @.
- jqpabc123 3y agoAnd the Sun just somehow emerged from a cloud of hydrogen gas. The obvious flaw in this line of thinking (as related to AGI) is that we/humans can create/engineer the same or similar without any real understanding of the underlying mechanisms involved.
- the8472 3y agoEvolution is an incredibly dumb, very parallel search over long timescales and that managed to turn blobs of carbon soup into brains because that's a useful (but not necessary) tool for its primary optimization goal. But that doesn't make carbon soup magical. There's no fundamental physics that privileges information-processing on carbon atoms. We don't have the time-scales of evolution, but we can optimize so much harder on a single goal than it can. So I don't see how it's a movie fantasy any more than bottling up stars is (well, ICBMs do deliver bottled sunshine... uh, the analogy is going too far here). Anyway, the point is that while brains and intelligence are complicated systems there isn't anything at all known that says it's fundamentally impossible to replicate their functionality or something in the general category. And scaling will be a necessary but perhaps not sufficient component of that, just because they're going to be complex systems.
- Anotheroneagain 3y agoThe question is if other animals even do have the kind of intelligence that most people call AGI. It seems that this kind of adaptive intelligence may be unique to mammals (and perhaps a few other outliers, like the octopus), with the vast majority of life being limited to inborn reflexes, immitating their conspecifics, and trial and error.
- RohMin 3y agoThis is actually a great perspective to have. The idea that there is no fundamental law of physics that should prevent us from replicating or exceeding the functionality of the human brain through artificial means makes this a question of when, not if.
- jqpabc123 3y agoThe idea that there is no fundamental law of physics that should prevent us ... Maybe ... someday. But right now, we don't even fully understand the physics that makes a human brain work. Every time someone starts investigating, they uncover surprising new complexity. https://theness.com/neurologicablog/is-the-brain-analog-or-digital/ https://theness.com/neurologicablog/is-the-brain-analog-or-d...
- myrmidon 3y agoDo you agree that AGI can emerge from a bunch of haphazardly connected Neurons, using about 700MiB of procedurally generated initialization data (human DNA) and employing 1-2 decades of low-bandwidth, low-power (~25W) training? You might even agree that not a single directed thought went into the design of that whole system. Having established that, it seems laughable to me that it would be an "impossible fantasy" to replicate this with purpose-built silicon. Sure, our current whole-system understanding could still be drastically improved, and the sheer scale of the reference implementation (human connectome) is still intimidating even compared to our most advanced chips, but I have absolutely zero doubt that AGI is just a matter of time (and continuous gradual progress). If a lack of understanding did not stop nature, why should it stop us? :P To me, all the arguments against AGI appear unconvincing, motivated by religion/faith, or based on definitional sophistry. But I'm very open to have that view changed...
- spangry 3y agoWell put. I’ve wonder if some of those on the other side of the argument are actually, perhaps on some subconscious level, proponents of mind-body dualism: https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism
- jqpabc123 3y agoTo me, all the arguments against AGI appear unconvincing, motivated by religion/faith, or based on definitional sophistry. Religion is belief in the unknown without reason or logic. Do you know of any inanimate object that is truly "intelligent"? Without ever knowing or seeing a single, real working example, you "believe" and have "faith" that it is possible. By so doing, you are practicing religion --- not science.
- the8472 3y agoThat's like complaining about artificial, non-organic flight being a fantasy before the wright brothers. Which would not have been entirely wrong, but hardly a knockout argument as we now know with certainty. There is a vast difference between "engineering challenges have not yet been proven moot by a working prototype" and "precluded by physics or mathematics". The (in)animate distinction is more akin to net-positive fusion in a man-made vessel vs. in a gravity well rather than obeys the 2nd law of thermodynamics vs. perpetuum mobile.
- bionhoward 3y agoIt’s not a hope to expect machines to match or exceed the human level of intelligence, because we demonstrate human level intelligent machines by virtue of existence, and the physical limits of computation far exceed those of our specific biology. AKA, we know it’s possible for atomic systems to be as intelligent as we are, because we are. We suspect the limit is far beyond us because computers have dramatically faster processing, essentially perfect memory systems, etc
- infecto 3y agoI like how he references someone who references back to himself.
- glitchc 3y agoCitogenesis. It's a hell of a drug.
- bobosha 3y agoI encourage Gary Marcus and scientists with similar views to create a functioning model as proof of their theories. Given his advocacy for "symbolic AI" it seems more valuable for them to demonstrate their concepts in practice rather than producing countless articles on the subject.
- jerpint 3y agoTo cite results of an arxiv preprint as “proof” that none of these LLM technologies will ever work is so disingenuous. Of course these beasts are data inefficient, but to imply that this means all of the progress is smoke and mirrors is such a stretch. I have a hard time taking any of Marcus’ arguments seriously
- HlessClaudesman 3y ago[flagged]
- osigurdson 3y agoIt is funny when you know neither A or B in A is the B of C claims.
- nick7376182 3y agoI've also never seen a cut snake spit venom!
- adamgordonbell 3y agoThese posts from him are so tiring. If ChatGPT-5 was a high fidelity 100% accurate simulation of a team of top tier grad students, he'd be writing that we will soon run out of grad student brains to scan and results will taper off. It would an powerful tool, changing the world but the post would be the same 'nothing to see here folks' post as this.
- vouaobrasil 3y ago> It would an amazing tool, changing the world but the post would be the same 'nothing to see here folks' post as this. It will change the world for the worse. We don't need more progress and more technology. What's the point of living if we're going to give all the funnest work to AI? Idiocy.
- adamgordonbell 3y agoMaybe amazing was the wrong word. I just meant regardless of whether AI tools are getting better or worse - regardless of whether things are looking to go on improving forever or tapering off - he will write the same post. That makes the posts devoid of meaning. It's gist is unaffected by the things happening in the real world. Will always be "I told you this wouldn't work".
- 3y ago
- CuriouslyC 3y agoI was at an AI meetup, and I talked to someone who was in charge of the emerging technology division at a VC firm. I asked him why focus so hard on AGI, when AI tools are already looking quite impressive and are a much clearer area to focus investment on. His answer was that AGI was "easy" which I laughed at (in a good natured way) and tried to get him to elaborate on, at which point he started to get uncomfortable and made an excuse to go talk to someone else. This same fellow was big on autonomous swarms for problem solving, but when I asked him what the problem autonomous swarms were supposed to solve that you couldn't solve more easily and quickly by a LLM talking to itself, he didn't have an answer.
- dartos 3y agoEveryone is building nonsense betting that openai’s next real innovation will validate their idea and they’ll be ahead of the curve. That’s all it really is
- falcor84 3y agoYou make it sound silly, but it does seem to make good business sense to me. I'd argue that it seems like they learned from The Bitter Lesson[0] and instead of trying to manually solve things with today's technology, are relying on the exponential improvement in general purpose AI and building something that would utilize that. On a somewhat related note, I'm reminded of the game Crysis, which was developed with a custom engine (CryEngine 2) which was famously too demanding to run at high settings on then-existing hardware (in 2007). They bet on the likes of Nvidia to continue rapidly improving the tech, and they were absolutely right, as it was a massive success. [0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
- dartos 3y agoYeah it’s a real valid business strategy. It’s also silly. The crytek case is similar, but there’s a big difference. Crytek was betting that performance of hardware will increase. A lot of AI startups are betting that future LLMs will have an new different, fairly vague, capabilities (AGI, whatever that actually means)
- albertzeyer 3y agoIf you don't know Gary Marcus: He keeps repeating this same argument since many years. He has always downplayed the capabilities of LLMs since the beginning. He never really added much useful (constructive) content to the discussion or to the research, though (except that he is advocating symbolic AI methods). So he is not really taken seriously by the community, but rather annoying on this. Although, of course he is not alone with his argument. E.g. even Yann LeCun is repeating a similar argument on LLMs, and many other serious researches as well, that we probably need a bit more than just LLMs + our current training method + scaling up. E.g. some model extension which handles long-term memory better (lots of work on this already), or other model architectures (e.g. Yann LeCun proposed JEPA), or online learning (unclear how this should work with LLMs), or also different training criteria, etc. In any case, multi-modality (not just text) is important. Maybe embodiment (robots, interaction with the real world) as well. (Edit Wow, the votes on this comment go up and down. I guess it's a controversial topic.)
- nabla9 3y agoI lost all respect to Gary Marcus when he in 2018 claimed that autonomous driving is decades away. Oh wait. I lost all respect to those hype-men who claimed it's just around the corner. Before transformers, people claimed that you just need bigger neural network, not something new. It did not work. Then came transformers, the new thing can again scale up better than previous era. Transformers have obvious limit: the computational complexity of self-attention module scales quadratically with the sequence length. We are trying to get partial fix with sparcity and linear tranformers and other techniques, but it gets harder at every turn. We make progress, but "just scaling" is not working.
- Anotheroneagain 3y agoThere are robotaxis being tested in the west coast US and Wuhan, China.
- prasoonds 3y agoI’m not sure if I’m parsing your sentence right but surely you can’t be implying that autonomous driving isn’t possible or that it is some undermined time away? You can come down to SF and take a waymo today. It works great.
- School-Cotton 3y agoDriverless cars are not just demos, they already work in a few cities (granted, they are still limited). And LLMs are massively useful already. What is this guy talking about?
- deleted 3y ago[deleted]
- jhbadger 3y agoWhile it's true that it is worth being skeptical about the current deep learning/LLM boom, younger people who don't know who Gary Marcus is need to know that he is not really an unbiased observer here -- he has never liked neural net approaches and thinks they were a wrong turn (even though they obviously are far more successful than the older symbolic "GOFAI" methods).
- CuriouslyC 3y agoThe paper that Gary cited is just so stupid. If you're drawing training data from a fixed distribution randomly, of course as the size of your training data set increases, the odds that you will get a random draw that is very similar to an existing piece of data increases, until you have to do a lot of random draws to see a new part of the distribution. That is the cause of the exponential increase in data requirements for linear increase in performance. We just need to methodically sample the under sampled portions of the source distribution. Not more data, better data.
- hartator 3y agoHumans were able to get better data themselves. Which I think it is what we mean by actual AI.
- philipwhiuk 3y agoAh yes, "just sample under sampled portions" The problem is that they don't exist in discrete missed data sets they only exist in larger data sets that also contain large amounts of well sampled data. If you could separate out the existing data you could cherry-pick the under sampled data, but do that requires the very classifier you're trying to build in the first place. This reaction is like saying "if we knew everything we'd be able to answer every question"
- naasking 3y agoHis suggestion sounds more like human curated data sets rather than automated curation like you're assuming, although this is a good distinction to make.
- CuriouslyC 3y agoThat's not really true. You don't need to actually solve the problem, you just need to create a first order solution for optimizing your data set, which can enable better models that can be used to create a solution that is second order accurate, and so on iteratively.
- brookst 3y ago
- lenerdenator 3y agoAnd thank God for that. If there's one thing humanity doesn't need, it's the kind of person who leads a Silicon Valley corporate entity creating a self-aware consciousness.
- mypreciousun 3y ago"Neural networks (at least in the configurations that have been dominant over the last three decades) have trouble generalizing beyond the multidimensional space that surrounds their training examples. That limits their ability to reason and plan reliably." But humans also have this limitation. Every science discovery is just some new data we can train on and create models for, both mentally and mathematically.
- brookst 3y agoAgreed. It sounds like the author’s bar for intelligence is to materialize new knowledge out of nothing. Which sounds more like magic than intelligence.
- anonyfox 3y agoI'd stand by the bitter lesson ( http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html ) here. of course it feels better to do something clever, finetune models, do clever agents or novel algorithms instead, ... only to become beaten by the next larger scaled general model.
- EGreg 3y agoWhy be clever at all anymore?
- CuriouslyC 3y agoYup. With AI being the cleverest, the value comes from having vision and being able to do long term planning.
- EGreg 3y agoBut can't AI do that too? I think AI can easily have more strategy and patience and coordinate 10000 swarm members than you. Why will we need you to do anything at all?
- CuriouslyC 3y agoYou have to keep in mind that humans have a lot of hardwired advantages, and we take in a massive amount of multimodal data. Even when the best high level planning AIs are smarter than humans across the board, they'll still be vastly more expensive to run, so humans will continue to have value barring a complete revolution in computing efficiency.
- iknownthing 3y agoSo models don't do well on data it wasn't trained on? Hasn't that always been true?
- hooby 3y agoI'm skeptical of both the people who claim we'll achieve AGI and self-driving cars in just a few years - as well as of the people who claim it can't be achieved by LLMs at all- ever. I feel that we don't understand well enough (scientifically) what "general intelligence" even is, and how it comes to be in humans - to make claims either way. To me, the only honest answer right now seems to be, that we do not know. We have absolutely no clue how close - or far - we are to AGI.
- thiago_fm 3y agoHe's right, it won't get us to AGI. But scaling is necessary if we ever want to get to AGI. One thing his argument has a point is that we're really close to hitting a ceiling in improvements for LLMs, unless a new groundbreaking research arrives. Just shoving more data, increasing parameters etc won't get us anywhere. Not to mention LLMs behave more like a data compression algorithm than anyform of "artificial" intelligence. What I personally find troubling is what will happen to NVidia and other pick sellers once the LLM hype fades away and people are aware of its limitations, will we have still big investments in scaling? Likely not.
- hiddencost 3y agoStop posting Gary Marcus. Yeesh.
- HarHarVeryFunny 3y agoScaling alone will certainly not get us to AGI, but that's trivially obvious. The "scale is all you need" argument, from anyone intelligent, assumes that other obvious deficiencies such as lack of short-term memory (to enable more capable planning/reasoning) will also be taken care of along the way, and I'd not be surprised to see that particular one addressed in upcoming next-gen models. There's a recent paper here from Google suggesting one way to do it, although may other ways too. https://arxiv.org/pdf/2404.07143.pdf https://arxiv.org/pdf/2404.07143.pdf The more interesting question is what other components/capabilities needs to be added to pre-trained transformers to get to AGI, or are some of the missing pieces so fundamental that they require a new approach, and can't just be retrofitted along the way as we continue to scale up?
- m3kw9 3y agoHe’s somehow a doomer but also at the same time saying it won’t happen
- mediumsmart 3y agoThere *is* intelligent life on Earth, but I leave for Texas on Monday.
- chrsw 3y agoI thought the Numenta approach of trying to emulate a column of neo-cortex was more promising (and ambitious) than just scaling up LLM power. But here we are.