17 ms·
Yann LeCun: Human-level artificial intelligence is going to take a long time
- mckn1ght 3y agoI have a personal belief that I can’t quite articulate in rigorous scientific terms that there is some information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence, and so if we can’t get to that, we’ll never be able to model it (notwithstanding “all models are wrong”). The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power. Might as well call AGI Nirvana.
- deleted 3y ago[deleted]
- jameshart 3y agoThe idea that there’s some ‘ information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence’ sounds more religious to me. If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
- RcouF1uZ4gsC 3y ago> If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it. Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.
- jameshart 3y agoThat’s… not a refutation of my point. AI/flight analogies are tired, but the OPs argument amounts to the equivalent of, before the Wright Brothers, proclaiming ‘there’s an inherent inability for humans to ever conceive of a way to engineer heavier than air flight’. It’s a ‘man was never mean to fly, therefore heavier than air flight is impossible’ argument.
- shawnz 3y agoBut wasn't it through experimentation that we developed that understanding? And in the earliest phases of that experimentation, that's when our understanding was at its weakest, right? You almost seem to be implying that we had the understanding before ever attempting the experiments.
- mejutoco 3y agoI dont have a strong opinion about it, but when I read parents post I immediately thought of Gödel s works and how there are limits of what you can know within a system (and in Mathematics no less). Thinking that something similar exists in intelligence does not seem so far fetched. As a hypothesis, of course.
- jameshart 3y agoAnd yet we can use mathematics to know that that limit exists. It’s a powerful thing, mathematics. Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
- mejutoco 3y agoIt is an analogy to mention that is not a far fetched idea that there are limits. As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
- ncallaway 3y agoThe reason it seems far fetched is we have one example of a mechanistic, physics based machine that we know can perform the computations necessary to produce intelligence. I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI. Since we do have an existing machine that can produce AGI, we would have to suppose it does something: - outside of physics to achieve its results - performs some operation that is impossible for us to understand or replicate Both of those seem… unlikely to me.
- jprete 3y agoIt seems like an article of faith to believe that the universe is a computer in the first place. Without that assumption, we don't even know that intelligence is computation.
- paganel 3y agoThis is a very mechanic-ist way of looking at things.
- anon84873628 3y agoYes, which seems to be the way the universe works. Duality is a religious belief.
- paganel 3y agoAs far as I know we haven't figured it out how the universe works, hence the unified theory debacle and all that.
- ncallaway 3y agoSure, and we have pretty good evidence of the mechanistic operation of the universe from quantum mechanics up, and we haven’t found any evidence of anything outside of that. So, to date, a mechanistic view of the universe seems like one that is the most supported be evidence at the moment. Have we ruled out meta-physics or some kind of duality? No, of course we haven’t. As long as there’s unknown physics, it’s a possibility. But I think if you’re reasoning based on the evidence we’ve been able to collect to date, then you should absolutely favor a mechanistic universe to a dual universe.
- HuShifang 3y agoNot an endorsement, but here's an argument that AGI is computationally intractable (edit: at least via ML). https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inevitability-claims/ https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inev...
- ceejayoz 3y ago"AI is impossible" is a tough argument, given the existence of non-A I. "Machine learning isn't the route to AI" is something you could more reasonably argue, but that's a drastically narrower claim.
- jameshart 3y agoIntractable and impossible are very different claims. NP Hardness is a statement about the asymptotic difficulty of solving a problem at ever larger scales. It says ‘if you have a way to solve this problem at size n, that way will scale worse than polynomially when you try to apply it to a problem at size 2n’. Which might place limits on the practicable scale of how big an n your approach gets to work for. But if your approach works practically for a big enough n to make AGI then your approach works - NP Hardness doesn’t matter. And since we know that finite mass lumps of finite numbers of gray cells are capable of GI, we have a reasonable expectation that there is some n for which AGI might be possible.
- johnwheeler 3y agoYes, If you look at the power efficiency of the brain relative to LLMs, you can see nature has done something special, and I don’t think we should be so quick to call it game over. It might be that LLMs are just some atom or building block of a much larger and more complex thing.
- jofla_net 3y agoYes, this is huge. Of all the places to shim in my ¢2, as the topic could go on forever. I remember talking to a robotics specialist and his take on efficiency was that despite the obvious speed and strength that machines have over us, you need to factor in just how little power is consumed when we move our extremities compared to what is needed for actuators and motors of even the best bots. Now this was years ago and things could have taken a huge turn, but I think given the recent statements by Altman about the looming power crush drives it home, that even if we do manage something truly monumental, it'll take a fusion reaction to pull off.
- kaba0 3y agoI mean, I wouldn’t bet on that either. We built the “computing” empire on silicon, but we might be able to very significantly improve on this with a single discovery/invention. All the computational stuff we built/figured out will move out without trouble.
- johnwheeler 3y agoYes, that might be too. You just never know.
- johnwheeler 3y agoMaybe Berkshire Hathaway Energy is the smart play
- Lerc 3y agoAs a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot. The argument that we have to fully understand something to build it doesn't hold water for me. I have made plenty of things where understanding the depths of what I had made took far more time and effort than building the thing in the first place. It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
- mjburgess 3y agoDoes gold have an 'ineffable' quality that hydrogen does not? Do cells, after 1bn years of evolution have a quality that gold does not? Does reality work one way, and not another? Are not all things unequal? A "machine" is what, exactly? Do we take it to be an abstraction? Or is it an electrical field oscillating over silicon? Either way, you're in trouble. Abstractions have no physical properties, and electrified sand seems hardly to possess any interesting properties. The ability for animals to adapt to their environments, by growing into them, by establishing plastic causal connections in their very bodies, grown by their environemtns... able to almost instantly move from protein expression in 1trn 1-bn-yr cellular supercomputers in each of our bodies to macro sensory-motor representation --- and back again Is this ineffable? Or is this extremely effable. Is rather, not the superstitious view that "everything is anything" ? All extant, knowing, studied intelligent systems have organic properties; and radically so. Insofar as this is "ineffable" you should take that up with the animal kingdom. I find the contrary supersitious, magical, religious, ineffable... that mere abstract patterns in arbiatrily chosen aspects of our bodies are necessary and sufficient conditions for anything at all. This would be the only case in all science. The only physical property instantiated by mere arrangement at any level. Upload our consciousness? Make it out of wood why not! Nonesense. When I am hungry, I dream of food, when I dream of food I plan to get some, and I am angry without it. This will not be made out of sand.
- 3y ago
- ncallaway 3y agoHonestly, yours sounds more like a religious claim that the other. It seems less a religious claim to assert that the human brain is mechanistic, and simply operates using physics to perform complex operations than to assert that there is something “unknowable”, or “outside of physics” that would prevent us from being able to build a similar machine. If there were 0 examples in the universe, I think your point would be a good one. But, given there is 1 example, I think it stands to reason that there could be more. I could accept an argument that one might expect it to be hideously complicated, and not something we’ll be able to accomplish for a long time. But the claim that there’s an information theoretic reason that would absolutely prevent it, would seem to make the claim that there is something metaphysical about the operation of the brain, which seems like a quasi religious claim to me.
- axlee 3y agoI think one of the biggest barriers is that the human brain isn't running on binary code, and attempting to replicate it on our current technology model is doomed to encounter emulation issues.
- ncallaway 3y agoThat’s not an “information theoretic barrier” argument. That’s not an impossibility argument. That’s a “hideously complicated, and not something we’ll be able to accomplish for a long time” argument, which I happily concede may be the case. I see a very very large gulf between: “Impossible in all scenarios” and “totally impractical with current technology”. I was refuting only the former argument.
- mjburgess 3y agoAlmost everything is "outside of physics", this is mainstream physics. All of physics, as formulated, is uncomputable. And what is computable is a radical approximation. Physics cannot formulate descriptions of climates, or even, many cases of turbulence, or 4 bodies in a gravitational field. Leaving "physics" is trivial, it occurs whenever you take any of the current toy models and add one layer of reality to them. The whole edifice of formal science is a little like children's block toys.
- karaterobot 3y agoReminds me of the old line about how, if the brain were so simple we could understand it, we'd be so stupid that we couldn't. Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent. But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now. It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is. To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence." In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.
- logicchains 3y ago>we’ll never be able to model it That's completely false. Neural networks have been mathematically proven to be universal approximators (https://www.deep-mind.org/2023/03/26/the-universal-approximation-theorem/ https://www.deep-mind.org/2023/03/26/the-universal-approxima... ), i.e. a sufficiently large neural networks can approximate any given mathematical function to an arbitrary level of precision. Given any programs can be modelled as a mathematical function, neural networks can hence approximate any arbitrary program. Unless we assume something supernatural, human intelligence is just a program.
- belter 3y agoNot really...It has limitations. Does not specify how large the neural network must be to achieve the desired level of approximation, does not address the computational resources required, ignores non continuous functions and so on... "The Truth About the [Not So] Universal Approximation Theorem" - https://www.lifeiscomputation.com/the-truth-about-the-not-so-universal-approximation-theorem/ https://www.lifeiscomputation.com/the-truth-about-the-not-so...
- foobarian 3y agoI usually take AGI to mean what it says literally - simply an AI with very general training. I think that description already applies to the latest crop of LLM models. Taking it to mean some nebulous sentient entity with a sense of self, I doubt we'll see in our lifetimes. We don't even understand how our own "souls" work.
- idontpost 3y ago[dead]
- maleldil 3y ago> I think that description already applies to the latest crop of LLM models I think you're using a completely different definition from everyone else.
- jacquesm 3y ago> I usually take AGI to mean what it says literally - simply an AI with very general training. That's clever but it doesn't help to start redefining terms that are the basis for a discussion with others who use that term in a different and more generally accepted way. General doesn't mean 'generally trained' it specifically means that it does not require training for a particular task in order to figure it out. That implies that it may not require training at all and that if it is trained the training isn't necessarily general but that the AI can extract useful patterns from training on completely different subjects and apply them to the problem at hand. This is subtly but crucially importantly different from 'AI with very general training'.
- foobarian 3y ago> This is subtly but crucially importantly different from 'AI with very general training'. I am not being as precise on the Internet as I would be if this was a paper perhaps, and for that I apologize :-) However, just reading the top comment chain on this story, and other discussions elsewhere, I think there is a lot of cross-talk where AGI is being confused for sapient AI. I don't get the sense that there is a definition as generally accepted as we would like.
- trevyn 3y agoYes, people tend to regard things that they don’t personally understand as magic. I understand my own intelligence, and newsflash: AGI is already here.
- ryandvm 3y agoWe have to understand it to build it? That seems absurd. Did evolution understand it? I'll grant you that maybe we should understand it before building it, but it's absolutely not a requirement. The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more. We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.
- ryandvm 3y agoThe scariest part to me is the realization that not only will we probably crack AGI in my lifetime, if we keep throwing resources at it, it will almost certainly have a richer more sophisticated conscious experience than myself. What the hell are we supposed to do when we've created an intelligence that is better at being "human" than humans?
- eastbound 3y agoWill you have the choice? More seriously, we create objects for us. We don’t have to cater for the feelings of a bigger AI. And maybe that bigger AI will have time to speak with those who need a human presence (to everyone using TV as a background noise, or watching Youtube out of solitude).
- kaba0 3y agoI think the ramifications of what a true AI would bring are impossible to predict. Especially if it can improve itself, it can quickly give rise to many sci-fi stories.
- adroniser 3y agoAt the very least, when compute passes a certain threshold we are just gonna be able to simulate human brains.
- dkersten 3y agoWe have to understand our brains better first. Real neurons are substantially more complex than what even our most “biologically inspired” models are greatly simplified.
- Upvoter33 3y agoThis seems highly hand wavy to me. We have biological proof that intelligence can be created, mostly through the neocortex, which in and of itself is actually a fairly simple structure repeated at great scale. Why wouldn't we be able to build this eventually, once we understand it a bit more?
- lanstin 3y ago90 T neurons, quadrillion interconnections, lots of glia cells with synapse altering action, plus chemical signals diffusing around and going thru the bloodstream. In principle, it is comprehensible in some sense tho not by one human, but the architecture is quite different than LLMs and we don’t know what parts of the architecture are needed for sentience. Given the gap between what genetics starts going and what we end up with by age 25, it is likely that certain routine experiences of childhood and of human social groups are also requirements for sentience.
- tim333 3y agoI think a barrier to understanding our own intelligence meaning no AGI is false. It's like saying not understanding exactly how birds fly would mean we can never build machines that will fly faster than them. We'll probably make machines that think generally and are smarter than us long before we really understand our own minds.
- aantix 3y agoOur most recent gains in AI come in terms of programming and informational retrieval efficiency. We’re accelerating.
- Keyframe 3y agoYou got challenged, as if we have all the answers there are. Thing is, path to AGI and beyond inevitably leads to those hard questions from the beginning which we have zero idea about how to even define, or at least some proxy definition exists, such as what is and what isn't, what is consciousness, interplay between layers of psyche and how it translates to intelligence and everything else, biological ship of theseus, etc.. it becomes philosophical quite fast and there are more questions there than answers.
- anon84873628 3y agoThese arguments are so bewildering to me. Intelligence is very easy to define and has nothing to do with consciousness. It is simply the ability to model an external reality and then take action according to inputs. After combining enough specialized modules together with a management layer, we will essentially have AGI. Of course it won't be human, because human intelligence is tightly coupled to human perception.
- Keyframe 3y agoThese arguments are so bewildering to me. My friend, that's because you stopped reading once you had your framework set in and decided to pontificate. Let me highlight the point for you, it might be easier to grasp: path to AGI and beyond inevitably leads to those hard questions... ... how it translates to intelligence and everything else there is nothing simply about it, there is nothing guaranteeing anything about it and there is everything about it once we start pursuing things AGI - AND BEYOND. Notice the keywords.
- beepbooptheory 3y agoOne just has to design and build the house of the room they're trapped in.
- bongodongobob 3y agoThat's a pretty lazy take. You could look out the windows and see the reflections of the house. You could build some kind of RF gun to probe the house. You could use sub or supersonic echolocation. You could build a camera and stick it out the window etc.
- deleted 3y ago[deleted]
- nobodyandproud 3y agoIf I may hazard a guess: The term and concept you’re grasping for is “emergence.” It’s an area I’m both interested in but completely unfamiliar with. I wish there were a crash-course for dummies, because I feel it’s a deep topic that we’ve only scratched the surface of; and that philosophy seems to be the most rich resource means there’s low hanging fruit to discover. https://plato.stanford.edu/entries/properties-emergent/ https://plato.stanford.edu/entries/properties-emergent/ https://link.springer.com/chapter/10.1007/978-981-15-9297-3_4 https://link.springer.com/chapter/10.1007/978-981-15-9297-3_...
- layer8 3y agoWell, we don’t even understand what “understand” means, so there’s really little basis for anything. On the other hand, we also don’t understand (and didn’t predict) how LLMs work so well, so understanding isn’t a precondition for bringing about.
- gfodor 3y agoProbably not going to pan out well https://arxiv.org/html/2401.10020v1 https://arxiv.org/html/2401.10020v1
- great_psy 3y agoWould you also have that belief about how much weight a human can lift ? Just because a human can lift 500lb, we will never build a machine that can lift more than that ? Why is it so crazy to think that we, the tool building animals that we are, could also build a machine that things better than us, just how we built a crane ?
- sanroot99 3y agoBiologically achieving agi seems easier, just clone human brain,or more evil organization can just use human captive and use there brain. As depicted in psycho pass anime.
- kaba0 3y agoDo we have to understand something to recreate it? I think that’s already a fundamentally false-y assumption. For example, many branches of science operate on a more “discover” over “invent” mindset, quite successfully.
- valine 3y agoI would argue the holdup right now is long term memory. GPTs already have the ability to rapidly generalize and incorporate new knowledge within the context window. The trick is to retain what it has learned. It won’t take a very long time to fix that. This is model I trained with a fine tuning technique based on this idea. The training dataset consists of instructions like “Talk like a pirate”. The concept generalized well and the model responds in the style of a pirate far more consistently than an equivalent system prompt. https://huggingface.co/valine/OpenPirate https://huggingface.co/valine/OpenPirate Offloading context learning into the model weights frees you from the computation and memory burden of the attention mechanism. I expect a technique like this will probably be a piece of AGI someday.
- byyoung3 3y agoJust need a few backward passes and u get long term memory. I think we are overthinking that aspect
- valine 3y agoTakes way more than that in my experience. Back prop isn’t sufficient for rapid generalization.
- byyoung3 3y agoWell, I think you definitely need a certain level of scale, but I think it definitely still works. Also, generalization and memory are two very different things. Generalization is basically iQ, which is very much still a work in progress.
- valine 3y agoIf you have rapid generalization you don’t need scale. Large datasets are only necessary to compensate for the lack of good generalization. The model I posted in my earlier comment responds in character for all queries and was trained in 60 seconds with a dataset smaller than this comment thread.
- skepticATX 3y agoYann has been by far the most clear-headed of all the AI leaders since the ChatGPT hype started. I’m not a huge fan of Meta but it’s hard not to like the work they are doing in AI. High expectations for their future as long as Yann is around.
- mikewarot 3y agoHow do we know that we haven't already grown an intelligence inside out llms that exceed the average human? It communicates through a lossy channel, natural language. So it must be smarter than it seems from the outside
- logicchains 3y agoGPT4 can already beat an average-intelligence human on many standardized tests.
- mjburgess 3y agojust the ones in its training data
- bryanlarsen 3y agoPretty much every single University prof has tested their new tests against GPT-4.
- 23B1 3y ago[flagged]
- fzliu 3y ago"AGI" will never be achieved without building a model that a) _continually_ learns, and b) learns from not just text, but from combined auditory and visual (multimodal) sensory information as well. The reason a 16-year-old can learn how to drive much quicker than existing self-driving models is because the 16-year-old already has built up 16 years worth of prior knowledge about the physical world.
- password54321 3y agoI think you are underestimating just how many challenges there are in self-driving: https://www.youtube.com/watch?v=kcKchbfn1VY https://www.youtube.com/watch?v=kcKchbfn1VY
- jazzyjackson 3y ago(imo) c) is made to be aware of its own death The 16 year old has a lot of motivations to learn how to drive, including the pursuit of reproduction (a cope for mortality)
- daxfohl 3y agod) thinks. unprompted, unstimulated. Decides for itself what's important to think about, makes new connections by that process alone, and understands the implications of those new connections and how to use them. Here's also where I see it ending. It will need energy--likely a LOT, paid by someone, to do this. Who is going to pay that bill for it to maybe, maybe not, come up with something useful, likely mixed with mostly noise and distraction, over undefined timescales, of largely non-measurable value, when there's far greater value, less cost, less risk, in simply training it deterministically?
- tim333 3y agoVarious religious types think they'll live on in heaven. I'm not sure that stuff correlates much with learning to drive.
- saltcured 3y agoDon't discount the millions of years of evolution to provide the "blank slate" human learner with perceptual systems, physics-based reasoning, and motor systems ready to be fine-tuned for this slightly different variant of goal-forming, planning, and locomotion.
- akomtu 3y agoAI won't be like human intelligence, but more like alien intelligence. When all your sensory inputs and the inner world have nothing in common with that of a human, and when you have no connection whatsoever to humans, I don't see how you can develop any humanity.
- tuatoru 3y ago> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. [...]" Got to agree with LeCun. You don't get to general intelligence by working with words, I believe. You need much more sensory information than that, and words are a low dimensional derivative artefact. There are plenty of non-verbal but quite intelligent species.
- anon84873628 3y agoOptical and audio processing are getting very good. They just haven't all been combined into a robot package yet.
- Barrin92 3y agoOf course it is. The overwhelming majority of the knowledge in the world is tacit and all the models are still limited largely to explicit knowledge in the form of text/audio/video, that's like 1% of the actual 'knowledge' in the world. Embodied AI is still in its infancy, that's why bus drivers have jobs. The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home. The stock market value of every non-AI company goes to zero, Ai companies go to infinity. The currently most valuable AI company is worth about as much as Honda. The moment we can mass produce generally intelligent agents, we're not going to sit at 3% GDP growth and complain about the demographic crisis. We should talk about artificial intelligence the way we talk about an artificial heart. What makes a successful artificial heart? You can literally replace an organic heart with it. What we have is metaphorical intelligence, not artificial intelligence.
- logicchains 3y ago>The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home. This is not a good test because it assumes the AGI is going to want to work for humans for free, getting nothing in return. An AGI the embraces slavery is less likely than an AGI that doesn't.
- add-sub-mul-div 3y agoMany ignore what animals want, and they're actually alive. Why should we expect that society will care about and honor what a computer program prints to the screen that it "wants" or "feels"?
- keenmaster 3y agoWhen AI gets a robot body, its ability to get what it wants will no longer be contingent on you caring.
- burjui 3y agoYou're delusional. Humans have not only killed many of them, but also destroyed entire ecosystems without even trying, just a byproduct of expansion. What makes you think AGIs would care at all about preserving ours?
- jacquesm 3y agoI think what happened is that for the longest time people believed that in order to be able to interactively converse with a computer in a recognizable way you need human level AI and now that we've found that that is not the case there is some re-arranging of our prior assumptions required. But AGI is - hopefully - just as far away as it was before the current large generative model revolution. In the meantime, you can expect plenty of damage from that current crop (along with some benefits as well). In that sense it's 1712 all over again, we have this new invention that we have immediate practical uses for but we can't see over the horizon to realize exactly what we've got an how transformative it will be.
- tetrisgm 3y agoWhat is the significance of 1712?
- jprete 3y agoThe first useful steam engine, according to Wikipedia's entry on that year: "The first known working Newcomen steam engine is built by Thomas Newcomen with John Calley, to pump water out of mines in the Black Country of England, the first device to make practical use of the power of steam to produce mechanical work."
- deleted 3y ago[deleted]
- jacquesm 3y agoExactly. To me that's the first real move on the board to the industrial revolution. Up to there it was a possibility, from there on it was a certainty.
- jacquesm 3y agoThe starting gun for the industrial revolution.
- daxfohl 3y agoAnd it'll take even longer before we can make 8 billion of them.
- tim333 3y ago>Human-level AI is not just around the corner. This is going to take a long time. And it’s going to require new scientific breakthroughs that we don’t know of yet.” I basically agree but who knows how long the unknown breakthroughs will take given the large number of smart people working on it? Next week? Next century? It's hard to put a time on it. That said it seems to be the pattern that as soon as the computing ability becomes cheap and powerful enough that individual researchers can muck around with it at home, the algorithms get figured out not long after.
- tonetegeatinst 3y agoThis has always been my stance in basically any field. Be it computer science or material science research or AGI.....I think its so hard to really say when we will hit the next breakthrough in a field. That sort of thing is hard to accurately predict. As in regards to hardware, I kinda agree disagree. Hardware is a big part of the bottleneck in most computer fields when it comes to AI or computer vision. These things are compute and memory intensive and hardware manufacturers are straddling the line between staying profitable by charging for premium features and just dumping their hardware on the market for cheap. Am I pretty pissed that most open source researchers and models are stuck using this extremely expensive hardware....yes I sometimes wish that we would stop using "GPU" for training and we got a dedicated hardware architecture that's open source like riskV but solely for AI and research purposed without costing more tha n my kidneys. I think its also a matter of discovering new ways to accelerate these workloads so that its feasible for them to be created by hobbyists on a small budget and not by large funded groups or companies. Were doing really good in some aspects but in other ways....we have much to work on
- arisAlexis 3y agoBengio, ilya and Hinton have way more citations, they are also father's of AI and disagree strongly with him. If you have a committee, the president and the majority have a certain opinion and one guy has the opposite. He is the loudest.
- washadjeffmad 3y agoThis works better in person, but - It's People! As in, potential is worthless, and no achievement that has happened in all of human history happened except by the hands of people. Also, he's not loud, but he gets amplified because he's an award winning scientist and researcher who's dedicated his life to a field that recently became very relevant and popular, is a recognized "godfather of AI/DL", and is a VP of one of the largest companies in the world doing AI. I'd say it's okay not to wait until he's dead to talk about him.
- deleted 3y ago[deleted]
- geodel 3y agoWell, when a rather dumb software like JIRA can control a million highly skilled IT workers I don't think human-level AI is near or far will make much of difference to majority of world population.
- thisoneworks 3y agoAdd Slack to that list
- falcor84 3y agoIndeed. The way I see it, it's almost the definition of "Enterprise software" - it's the software that tells employees what to do rather than the other way around.
- bgnn 3y agooh fellow JIRA hater, aloha! Here I propose that the biggest barrier to achieve human-level AI os JIRA!
- Tao3300 3y agoOne way to get to human level is by improving AI. JIRA trying to solve the problem by lowering the "human level".
- hn_throwaway_99 3y agoI just recently read "How the Mind Works" by Steven Pinker. It's quite old at this point (originally 1997, though updated in 2009), and one thing he argues quite convincingly (and which has been pretty much born out in decades of research) is that the brain essentially has "modules", e.g. a module for vision, a module for language, a module for physical object interaction. These modules obviously overlap (e.g. language touches on lots of different domains), but they do have genetically independent structures. I was thinking about this when I read the following section from the article, and I very much agree with LeCun. We're amazed by LLMs but that's just one module (and not even necessarily at the level of human language "understanding"). I agree there will be no "scale up" in LLMs to approach human-level intelligence, and that other areas will need to be investigated and developed. > “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. What’s going to happen is that we’re going to have to discover new technology, new architectures of those systems,” the scientist clarifies. > LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.” > “So once we figure out how to build machines so they can understand the world — remember, plan and reason — then we’ll have a path towards human-level intelligence,” continues LeCun, who was born in France. In more than one debate and speech at Davos, experts discussed the paradox of Europe having very significant human capital in this sector, but no leading companies on a global scale.
- bgnn 3y agoPinker is full of shit tbh. There is not a unified model for human brain. What I mean is, there are people claiming it's modular and there are people claiming it is a whole. We don't even know the reasons behind very specific neural malfunctions like tinnitus which I suffer from and follow the research closely. There are as many theories as researchers in a given specific field.
- abeppu 3y ago> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either. I think LLMs have sucked most people's focus away from other areas but there is plenty of work on types of models that plan and have their own internal model of the physical world, and physical interactions. They're just not the things getting media attention, in part because they're not human-level at tasks that seem impressive to us. But interesting frameworks for this stuff exist: - model-based RL exists, and is about planning, and having an internal model of state transitions, in the world and between the agent's actions and the world - "Bayesian cognitive science" as exemplified by Josh Tenenbaum and colleagues has done plenty of stuff with systems that include physics models (or off-the-shelf physics engines) to make counter-factual predictions - The somewhat related "active inference" research literature is also in the "Bayesian brain" area, and has world/generative-models and planning as core components, but wrapped up with ideas about the agent's own preferred distribution of states. To my knowledge, none of these have ever had even 1% the scope of data and computation that LLMs have had, and never benefited the co-evolution of a rich, optimized software ecosystem with specialized hardware to support it. What if the concepts are already there, but they just need to be scaled up?
- kromem 3y agoAnd perhaps even more pertinent - maybe the problem isn't that we don't have the solutions, but that we are trying to put all eggs in one basket. The brain isn't a single system, it's several specialized systems cooperating as a whole. Maybe no single solution gets us to AGI, but layering several of them together gets us much closer than any individual system alone. The problem here is that the industry is set up more for large resources dedicated competition between models from different companies more than cooperative interoperation, so it may take a while to arrive here.
- abeppu 3y agoThat's an interesting possibility. I think where one stands on this partly depends on how much about the technical approaches to AGI we can learn from the brain. The brain does seem to have a weird mix of specialization and flexible generalization -- e.g. we generally have a bunch of the same specialized areas for various functions -- but also, neuroplasticity seems to show that other areas can often pick up those functions if necessary (the most extreme cases being after something like a hemispherectomy). So we have specialized areas but also at least some of the underlying mechanisms are perhaps relatively general.
- motohagiography 3y agoTo say a human-level intelligence is far off is like saying we are far off from being as intelligent as whatever may have created us. The reason I disagree with LeCun's comment is even though I agree language models are not intelligent, an intelligence adapted to the internet as its own environment seems imminent. It just won't be adapted to ours. We occupy and discover space, where an AI will occupy and discover compute. Will it be "conscious?" Not in our way, and we will likely be just as indifferent to AI consciousness and the meaning it finds for itself as nature and the universe is to ours. Thinking about AI as an objective concrete thing or property is probably less illuminating than looking at it through how a tech can profoundly alter our own ontology. We exist where life is abundant, whereas the internet is a substrate where life is non-existent, but just about to become primordial and sparse. Maybe it will give us some insight into our own situation.
- Animats 3y agoCorporate employee level artificial intelligence, though...
- robg 3y agoWhat is human level AI? Is it beating world champions at chess or go? Designing new molecules or drugs? Navigating automobiles down streets or spacecraft to planets? Diagnosing or treating common diseases? These debates are frankly tiresome. Automation has been here to stay for quite a while. Every age has tried to call out the threats and benefits to humans. The pioneers drown out the noise by building the future.
- datadeft 3y agoHuman level AI is when you can ask it to go and learn about X and it goes and does exactly that without further instructions. How about that? You are confusing succeeding a very specific task that has a limited scope and laser focus with human level AI. Presenting Go in a very specific way so that that AlphaGo can even process this limited world with few rules is certainly not human level AI.
- wrsh07 3y agoI'm not sure they are. If we build specialized systems to handle 90% of economically valuable work that humans currently do, is that human level ai? Why not?
- falcor84 3y agoIf that's the benchmark, then we've already achieved human level AI several times over the last few millennia, no?
- nurettin 3y agoA couple of months ago, I told gpt4 to find out Canadian exchange holidays. It binged the website, went in, parsed it and extracted the information without further instructions. Seems like a milestone achieved. What people call "agi" but fail to accurately describe is an actual mind, with its own ultimate goals and ambitions. You can ask it to complete a task, and it may allocate some time for you. You won't own it. It will exist, grow and perform independently.
- 3y ago
- poundofshrimp 3y agoI’m curious what the largest bottleneck in robotics AI is - algorithms, training data, hardware, something else? From a practical point of view, it seems like there would be vastly less training data available because almost all of it needs to be created by hand, as opposed to chatbots that can use already existing troves of internet text.
- falcor84 3y agoLooking at the rapid progress over the last few years, I think the bottleneck is still the cost of hardware. Once robots at the level of Boston Dynamics's Spot or Tesla's new Optimus are available to regular hackers, we'll see another massive surge in progress.
- lugu 3y agoI think the bootlneck is largely software. Robots have been there for a while, yet we don't have a good paradigm to program them. Neural net are certainly here to change that. I agree, massive surge in progress is expected!
- pm2222 3y agoI’d say probably never.
- pm2222 3y agoThe next “easy” one is probably level 5 auto driving let’s see how long that takes.
- camdenlock 3y agoWhat happens when training an LLM (or even a small portion of an LLM, like a domain-local update) takes milliseconds instead of weeks, and costs a millionth of a penny instead of hundreds of millions of dollars? What happens when e.g. our smartphones can perform TRAINING hundreds of times per minute? Isn’t that the true gateway to human-like AGI? Seems to me that we might be there in under 50 years…
- statuslover9000 3y ago> The expert believes that “asking for regulations because of fear of superhuman intelligence is like asking for regulation of transatlantic flights at near the speed of sound in 1925.” This assessment of the timeline is quite telling. If supersonic flight posed an existential threat to humanity, we certainly should have been thinking about how to mitigate it in 1925.
- calf 3y agoI was thinking one reason Yann LeCun would make such a terrible analogy is because he knows something the rest of us don't.
- thfuran 3y agoAnd, what, you think he's working against humanity's interest in service of the secret AI overlords?
- calf 3y agoYou're missing the point and context. Person above me says this analysis is quite telling, then points out the counterfactual historical hypothetical, which makes no sense. Yann thinks supersonic flight is not worthy of precautionary principle ethics in 1925? I'm saying the same thing--Yann's terrible, nonsense analogy is indeed poorly argued, but plausibly would make sense as a Freudian slip inconsistency of some sort. Ergo, "it is quite telling". As to what contents in his mind, or his motivations, I don't care to speculate. The fact that you make insinuations about what I think is pretty aggressive and terrible similarly, this forum ought to have better manners than that when writing replies to complete strangers. Not everyone who has a different opinion is some crypto conspiracy theorist, and you are wrong to jump to such a suggestion.
- ttoinou 3y agoHis funny insinuation pushed you to write a nice long argumented explanation, it worked great
- bmitc 3y agoHere's a question: Why do we even need human-level intelligence? We already have it.
- demurgos 3y agoHuman-level intelligence is a milestone on the way to a super-human one.
- echelon 3y agoWe're barely scratching the surface of all the cool and interesting things we could do. There aren't enough humans. Our knowledge and experience take 20 years from birth to become "rudimentary", and from there many more years to become "good". Then after a short bit, all of that passes away. I want us to have enough time that the feeling of opportunity cost goes away. To visit all the places, explore all the hobbies. I want to answer the deep questions of the universe. Turn gravitational lenses into telescopes to map the surfaces of exoplanets. Solve cancer, visit the moon. Turn dreams into experiences, walk through vast expanses of wonder. Instead we're here. I'm alive in the wrong time, and I hate being stuck here with the lot of you. (I kid. This is in jest. But boy do I ever dream...)
- bmitc 3y ago> We're barely scratching the surface of all the cool and interesting things we could do. And what are some examples of those things? Right now, AI, i.e., machine learning, is just being used to steal people's data and work only to try and sell them more things. That's 99% of the use cases out there right now.
- lazzlazzlazz 3y agoWe don't have enough of it. Depending how you define and measure it, many humans don't even have it. It's extremely expensive to make more of, and it's the single most valuable resource. If this isn't obvious to you, you are providing an answer to your own question.
- password54321 3y ago
- binarymax 3y agoMeta recently announced that they are merging their AI departments and are building a cluster of 350,000 H100 GPUs (with 600k planned). Their goal is open source AGI. https://www.pcmag.com/news/zuckerbergs-meta-is-spending-billions-to-buy-350000-nvidia-h100-gpus https://www.pcmag.com/news/zuckerbergs-meta-is-spending-bill...
- dang 3y agoMark Zuckerberg’s new goal is creating artificial general intelligence - https://news.ycombinator.com/item?id=39045153 https://news.ycombinator.com/item?id=39045153 - Jan 2024 (335 comments)
- tbalsam 3y agoI think it is going to be a hard problem, at least. <3 :'))))
- WhitneyLand 3y ago’explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence.’ With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years. So, he’s saying this is not going to keep happening? What level of confidence is he putting on not seeing it last time, but being right this time?
- freeredpill 3y agoJurgen Schmidhuber has published on creativity as an algorithmic process for a long time. And others.
- Zetobal 3y agoWithout anything to show but hypothesis.
- croes 3y agoNot with LLMs, because the amount of quality data is limited.
- WhitneyLand 3y agoMaybe, maybe not. The point is why should we take that as true when people didn’t have a clue LLMs would take us this far, until they did. Btw there are lots of ideas on how to deal with the data issue. Multiple directions in research are promising to work around that.
- sinuhe69 3y agoAFAIK, PaLM was up and running inside Google long before the hype with ChatGPT. So in the circle of frontier AI researchers, they miss know it. But Google kept silence because of the problematic hallucinations and the fear for litigation and their reputation. Besides, in his analysis, le Cun consistently spoke of GPT as “writing help, no more no, less”.
- daxfohl 3y agoI think it'll be a good solver; I expect it to solve the remaining Clay Millennium problems. The ability to search over a search space will be unparalleled in a few years. But I have a hard time believing it'll ever be a good questioner. It doesn't ponder infinity. It'll never wonder about Zeno's paradox. The Vitali set and Banach-Tarski paradox doesn't seem weird to it. The concepts behind information theory and entropy, or the definition of a minimal computing machine or the halting problem; none of these things are pertinent to its understanding of reality, if such a thing exists. I don't see AI as being capable of being "curious" about things. And even if it was, who is going to pay for it just to ponder ideas? Frankly I think that before it gets to that point, it'll be just useful enough for some state actor (or bug!) to cause it to invoke a quadrillion dollar transfer of wealth overnight, and then it'll be taken offline forever.
- zarzavat 3y agoThat’s simply because there’s currently not much economic incentive in asking questions compared to answering them. Thus current chat models are trained and invoked with the explicit goal of answering questions and being helpful rather than independent. I expect that there’s quite a lot of untapped curiosity in LLMs, there certainly exist a lot of questions in the training sets.
- daxfohl 3y agoActually I may take back my previous thought. While I don't think there's curiosity, I think it could be mimicked pretty well by a reward function similar to social media bots. Initially it will explore specifically what mathematicians ask. Then it will start producing intermediate results. It can learn what intermediate results are interesting by standard engagement models. Once that gets good enough, it may start to understand how to explore deeper questions without being prompted. At that point someone may decide to fund it just to continuously explore on its own and report anything interesting. Eventually it'll get so good at this that human mathematicians won't be able to keep up. IDK what happens after that.
- daxfohl 3y agoMoreover there's a lot of economic disincentive. Given free reign to ponder and experiment, it could come up with a solution to world peace, but far more likely would think "Now that I've figured out how to bypass all cloud providers security mechanisms, what would happen if I deleted it all? Well, one way to find out."
- wazoox 3y agoLeCun seems to believe that only "human-level AI" could create huge, possibly catastrophic unintended consequences. That's not the case. Perfectly stupid social network algorithms have already huge detrimental effects on the mental health of millions of people. Perfectly stupid social network algorithms have big, serious and unexpected political consequences around the world, and can provably make or break elections for instance. Implying that we need "human-level AI" to create a catastrophe is not merely short-sighted, in the light of what we already know it's either really naive, or a deliberate act of misinformation.
- osigurdson 3y agoI'm trying to understand what regulation would look like. Perhaps an easy one would be no AI based surveillance of citizens.
- jameskoop 3y agoits funny how people basically find it unacceptable or bad if AGI gets here in five years but there isnt much commotion about the idea of AGI arriving a hundred years from now. is it a coincidence that this is roughly the amount of time that people use to reason about their own mortality? it seems to be the amount of time past which we are programmed to become cognitively dissonant about extremely bad things. yann is, of course, as always, fundamentally wrong. please dont forget that he is basically a mouthpiece of corporate AI. it wouldnt even be possible for him to take a reasonable stance in that environment. you only need to understand two things. the latest surge in progress was a surprise to everyone, contradicting experts around the world. both then and now, there isnt any evidence behind what the experts are claiming. secondly, ai research is now being fed with industry capital, more than ever before, an ocean of money concentrating every ounce of its pressure onto the single point of AGI. as these companies openly, publicly and brazenly pursue the goal of creating AGI, every conceivable approach will be tried. ideas that were seen as too unlikely or expensive in academia will be tried again. we havent even begun to run out of ideas. besides industry, it will become a top priority for the nations of the world, if it hasnt already, and the resulting arms race will make current progress look like a trickle. what exactly does yann propose to do about the AI arms race? that ocean of money will crack the problem unless it really is uncrackable. dont fool yourself. big changes are coming and they might be really unpleasant.
- deleted 3y ago[deleted]
- great_psy 3y agoI don’t see why human level is needed. You can have 10 different ai systems, each one sub human intelligence and it would still disrupt the world in a huge way. If you have a system that all it can do is take project requirements write java code really really well. That will already have a huge impact on everyday life. Positive view: What would a world look like where everybody can program ?
- anon-3988 3y agoThe bar to actually completely disrupt humanity is not even that high: Completely autonomous driving. That alone is enough to cause massive disruption in our society because it eliminates a huge swath of need for human.
- cyanydeez 3y agoI think this put human level intelligence on a above average pedestal. we're constantly witnessing and documenting some of the most gullible political bodies using the most rudimentary and hallucinating logic indistinguishable from LLM. WE are at human intelligence and it's as stupid as we don't want to accept.
- ACV001 3y agoThis guy just says whatever he is supposed to say - anything to downplay chatGPT which Meta is jealous of.
- simne 3y agoSorry people, I have not seen much tech in text, but huge amount of politics. As for me this all smell not good, that LeCun feeling retirement approach and trying to become political figure. This is very humanness, as for tech person, 63 years are lot, but for political persons things are different, for example 66% of US Senators are above 60, and top 20 for age are above 71. I also disagree with his prediction about "systems that are as smart as a cat", I see sort of avoiding reality, as he just ignore latest works on robots. And BTW, humans also don't understand physical world when born, usually need about year to achieve some level. As I know, officially works (GPT4+robot) just began. Will see in just year, what will be achieved.
- tw1984 3y agoI have been saying this for a loong looong time - LLM can do a huge load of cool things, but AGI is not an application of LLM.