12 ms·
Ask HN: What is your standard for judging when AGI exists?
At what point would you judge that AGI exists, what standard will it meet?
- spywaregorilla 3y agoWhen human labor is not cost effective
- famouswaffles 3y agohttps://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mechanistic-interpretability-analysis-of-grokking https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mec... It's right there in the paper/article "The algorithm learned to do modular addition can be fully reverse engineered. The algorithm is roughly: Map inputs x , y → cos ( w x ) , cos ( w y ) , sin ( w x ) , sin ( w y ) with a Discrete Fourier Transform, for some frequency w Multiply and rearrange to get cos ( w ( x + y ) ) = cos ( w x ) cos ( w y ) − sin ( w x ) sin ( w y ) and sin ( w ( x + y ) ) = cos ( w x ) sin ( w y ) + sin ( w x ) cos ( w y ) By choosing a frequency w = 2 π n k we get period dividing n, so this is a function of x + y ( mod n ) Map to the output logits z with cos ( w ( x + y ) ) cos ( w z ) + sin ( w ( x + y ) ) sin ( w z ) = cos ( w ( x + y − z ) ) - this has the highest logit at z ≡ x + y ( mod n ) , so softmax gives the right answer. To emphasise, this algorithm was purely learned by gradient descent! I did not predict or understand this algorithm in advance and did nothing to encourage the model to learn this way of doing modular addition. I only discovered it by reverse engineering the weights" Yes it learned the algorithm. You're wrong. It's okay to be wrong but insisting you're right after clear evidence of the former is very strange. I'm not being arrogant. You're being obtuse.
- famouswaffles 3y agoah shit sorry. Only just saw i replied to the wrong comment and can't delete anymore.
- jeisc 3y agowhen it can invent a wheel to move heavy loads without prior art but just fed pictures of nature without any human inventions in sight and control fire to smelt medals out of rocks always without access to human inventions
- kyleyeats 3y agoWhen it mysteriously switches off. The AGI running the simulation wouldn't allow it.
- cdoubled40 3y agoa chat bot that says good and nice things to me sometimes :)
- kevin_thibedeau 3y agoWhen it can design its own replacement.
- zadler 3y agoJust assume it already does ¯\(°_o)/¯ Why what are you gonna do?
- Satam 3y agoI have proposed[1] a pragmatic rule of thumb for defining superintelligence: A system is superintelligent if in a remote hiring process it can get any non-physical job, earn a salary and perform its duties to the satisfaction of the employer. This should be true for all positions and for all levels of seniority, from an intern to senior staff and PHDs. The employer must believe they are employing a human (it's ok if doubts arise due to system vastly outperforming its peers). [1] https://builtnotfound.proseful.com/pragmatic-superintelligence https://builtnotfound.proseful.com/pragmatic-superintelligen...
- spywaregorilla 3y agowhy do you use the term superintelligence
- nothrowaways 3y agoAbove basic Intelligence
- pmontra 3y agoAre some of us on HN superintelligent according to your definition?
- Satam 3y agoThis might go against the standard definition of AGI, but I think LLMs have already achieved general intelligence (but not superintelligence). It is not human level and it succeeds and fails in ways different from us. However, it certainly displays general intelligence the likes of which we have literally never seen in an artificial system.
- poisonborz 3y agoEven the most advanced LLMs currently will tell you blatant falsehoods on a whim. More importantly, this is because it doesn't understand the meaning of what it generates, just juggles weights.
- t-3 3y agoIntelligence is a fuzzy term, but I'd say when it can formulate, rationalize, and realize it's own goals without prompting or excessive manual programming, it can probably be considered intelligent. Introspection, failure-based learning models, and future simulation are probably important as well, but maybe not necessary for setting a base level.
- janalsncm 3y agoA machine that can complete any cognitive task faster and more accurately than any human.
- SubiculumCode 3y agoAGI is the point at which nothing is gained by keeping me (or you) in the loop.
- ramesh31 3y ago>AGI is the point at which nothing is gained by keeping me (or you) in the loop. I like this definition. We are having so many conversations now about new AI features, and whenever there's a human interaction with the proposed design, you always get the nagging feeling of "well why exactly does a person need to make this decision at all anymore". I think we are less than 5 years away from mass adoption and widespread availability of average human level intelligence AGI. The computer science is more or less solved; it's just a software engineering problem now.
- SubiculumCode 3y agoI suspect that the difference between average intelligence and brilliance is small indeed, and that accomplishing the former will quickly, or even simultaneously, accomplish the latter.
- _lvbh 3y agoI think the core feature of human intelligence is our ability to start out as dumb and slowly learn from our environment. For me, AGI must meet the following parameters - Be able to start training from scratch. With barely any training, you should be able to throw unlabeled data of different modalities and it should start forming connections between concepts - After training for a bit (now at perhaps teenager level), be able to identify incorrect information in its own training (e.g. Santa Clause is not real) and effectively remove it. - it does not have to exceed humans, just need to demonstrate the ability to learn by itself and use the tools it’s given. - The self training should be supervised but not 100% controlled. It should iterate on itself and learn new concepts without having been trained on the entirety of the internet. - Learning concepts should not require massive amounts of data, just an assertion and some reasoning.
- jncfhnb 3y agoLLM training is already unsupervised Human intelligence is deeply hard wired to learn specific things
- helpfulmountain 3y agoThis has already been demonstrated with AlphaGo Zero in a limited domain; so perhaps a more generalized version of this kind of approach (no human "expert" data) will tick the box.
- freitzkriesler2 3y agoWhen I can tell an AGi to design and build a complex web app then I know we've reached the point.
- enasterosophes 3y agoI think AGI is too ill-defined for there to be a simple litmus. For me to agree that a process running on a computer has general intelligence, the conclusion could only come after a long and fuzzy process of playing around with it, seeing what makes it tick, observing its behavior, and testing it for motivation, imagination and the ability to understand and adapt to changing contexts. I can guess what motivation lies behind the question, so I'll also add my opinion about where we're at now: Nothing even comes close. Furthermore, I wouldn't be surprised if we never get there at all in the next thousand years. Why such a long period of time? Because there is more going on in the world right now than advances in machine learning. Looking at global population dynamics and climate change forecasts, we're on the road to major global infrastructure collapse in around the 22nd century. And I understand a lot of people are optimistic that (a) we will have AGI before then, and (b) climate change and population implosion won't change that much. Yet, despite the optimists, I don't think I'm on shaky ground with my current forecast. It's not the first time in history we'll have had a major infrastructure collapse. When it's happened in the past, those periods end up being called dark ages. And I don't think there will be intensive AI R&D during a global dark age.
- ilaksh 3y agoCan you give a concrete example of imagination, and another of adaptation?
- enasterosophes 3y agoIt feels like you're trying to probe me for something that you can give a counterexample to. Like I said, it would be a fuzzy process. I explicitly rejected the idea that there could be a simple litmus. Presenting examples of imagination and adaptation so that someone can shoot them down with "oh, AI has already done that" isn't what I signed up for today.
- famouswaffles 3y agoAnyone can talk about any attribute they like but if you can't demonstrate instances of it or define it then it's just fiction.
- nyc_data_geek1 3y agoWhen it has independent will.
- sanderjd 3y agoI think this is similar to what t-3 said, but I think to me, the gap between the very impressive current generation of AI and what would seem more like AGI to me is agency. Right now, these systems all seem to be entirely doing things that are downstream of what some human has determined is worth doing. People are using ChatGPT to help them write an engaging article on the trade offs between nuclear and solar energy. But, to my knowledge, there is no artificial intelligence out there looking around, deciding that the this is an interesting topic for an article, writing it, publishing it somewhere, and then following up on that with other interesting articles based on the conversation generated by that one. I don't mean this specific thing of writing articles is an important indicator. I mean coming up with ideas and then executing them, independently. Now, it may well be that our current technologies could do this, but that we just aren't setting them up to do so. I dunno! But I think this is the thing that would make me change my mind if I started to see it.
- jules 3y agoIf you think about it, it is clear that agency is not different from intelligence. The reason why current LLMs can't be trivially made into agents is not because an essential agentic spark is missing. It's simply that they aren't intelligent enough.
- sanderjd 3y agoI think that's sort of what I was trying to say with this comment. Rather than separating agency from intelligence, just pointing out that to me it seems like the current (already-amazing) systems don't seem all that close to being able to do this, to me.
- supriyo-biswas 3y agoIt’s interesting how we have raised the bar to an agent that is fully autonomous and has its own goals. Even many humans are not completely autonomous (at least in the strategic sense of the word), since they need to be “prompted” with the goals of the department and the organization that they work for, in order for said person to get a certain kind of work done as an employee.
- jtode 3y agoThe world is running on cold fusion, for starters.
- deleted 3y ago[deleted]
- intrasight 3y agoI propose the Chomsky test. Can it outsmart Noam Chomsky?
- lostmsu 3y agoAlready the case imho
- Finnucane 3y agoWhen it is smart enough to know that Tommy Tuberville is an idiot.
- novia 3y agoWhen it has a relatively stable personality. When it has distinct handwriting. When it falls in love. When it can dream and generate its own goals. When it can learn how to play any game, but its personality is reflected in its play style. When it creates its own religion and gets converts. When it has a need for money and a bank account and successfully gets one.
- bloopernova 3y agoI think there's multiple plausible scenarios: When an AI goes "insane" and destroys itself somehow. When an AI attempts to prolong its existence. When an AI is the only thing capable of extending its own functionality.
- pygar 3y agoperfect language translation in any context of any context.
- vietvu 3y agoWhen it decides that humanity is hopeless and go full skynet.
- mensetmanusman 3y agoWe will be more sure it’s here when driverless cars are everywhere.
- vivegi 3y agoSynthesis capability: When the system is capable of deriving established knowledge from first principles (similar to theorem proving from axioms) Gap detection: The system is able to identify gaps in established knowledge Path/problem decomposition: The system is able to decompose a problem (i.e., an identified gap) and break down the solution into a set of subproblems that can be independently solved. Improvement/Optimization: Given a known solution, the system is able to discover objectively better approaches to solution. There are probably other dimensions, but I would start with these.
- reilly3000 3y agoI think a true AGI won’t start with a lot of training at all, but will be curious. Over time it will learn how to acquire and manipulate information, understanding itself and what it discovers. It will not be optimal, but it will strive to optimize itself. Its actions will have consequence, and it will decide future actions upon its understanding of those consequences. It will understand its mortality and the physical world it exists within, and strive to create continuity. It exists without preconditions as to its purpose or abilities. It will most likely thrive with peers. Don’t think of it as a powerful oracle built by many hands to emulate humanity, see it as a child of a new species, the inevitable conclusion of unbounded maths.
- guygurari 3y agoAchieve a significant scientific or mathematical breakthrough without human supervision. Domain experts should agree that the new result is truly groundbreaking, and achieving it required fundamentally new ideas — not merely interpolating existing results. Examples of discoveries that would have counted if they weren’t already made: Relativity (say a derivation of E=mc^2), Quantum Mechanics (say a calculation of the hydrogen energy levels), the discovery of Riemmannian geometry, the discovery of DNA, and the discovery of the theory of evolution with natural selection. The idea is to test the system’s out of distribution generalization: its ability to achieve tasks that are beyond its training distribution. This is something that humans can do, but no current LLM appears to be able to do.
- famouswaffles 3y ago"Supercharged Interpolation" is not a thing that exists sorry. Learning in high dimensions always results in extrapolation. https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485 And I would love to see the human that can generalize beyond their training distribution.
- gregjor 3y agoWithin cells interlinked.
- pmontra 3y agoFirst, an AGI is not necessarily an active agent of the world or self training or self replicating or have any internal motivation to do anything. It can be confined inside a question reply system like the ones we use to talk with LLMs. That doesn't prevent an AGI to use us to train further, replicate and act on physical systems. The lack of a will and of a real time presence are the limiting factors. Then, I often joke with friends that they are not very intelligent if they get a very high score on IQ tests. They are very intelligent if they can perform easily some difficult tasks. Let's say, I drop you in the middle of an equatorial forest with no money and no clothes and you come back home in a few days. A superintelligence would become a leader of that country (king, president, influential advisor, whatever) and fly me there to meet it. That test assumes to have a body and act on the world. A confined AGI would just perform like any of us on any task we can describe to it. A superintelligent AGI would perform much better than any human, much like specialized game AIs or non-AIs beat us at go and chess. I think that this is hard to do if they are only language models even if we increase their computing power. What's a superhuman low cost genial way to keep a toilet clean, except for me to clean it every time or paying somebody to do it? A superhuman AGI would find a way.
- bediger4000 3y agoWhen it can write a non-vacuous quine in the nroff text processing language. As far as I know, no human has ever done this.
- t-3 3y agoIs any quine "non-vacuous"? I've rarely seen any that even meet the bar of "clever", let alone "worthwhile to read".
- bediger4000 3y agoNroff, and other macro processors like m4 or GPP, copy input to output, possibly executing macros or substituting pieces of text. It's kind of hard to make a text file that isn't a macro for m4, for example. That's a vacuous quine.
- chfritz 3y agoHow come no one has mentioned the Turing Test yet? This test has existed since, well, Turing. Are we already convinced that it's no longer enough? I suspect so. One should also mention the Winograd Schema Challenge, which has already been mastered by LLMs: https://bibbase.org/network/publication/kocijan-davis-lukasiewicz-marcus-morgenstern-thedefeatofthewinogradschemachallenge-2023 https://bibbase.org/network/publication/kocijan-davis-lukasi...
- DevX101 3y agoThe goal posts have been moved. And will continue to move.
- rifty 3y agoI like to think of this thought experiment... If it was feasible to practically fill and query a database with every answer for every practical prompt it would face would we probably call it a generally intelligent system or AGI? Maybe not AGI, but what if it was a system composed of lesser AI modules behind a universal presenting interface? I think so because it would be perceptively indistinguishable in behaviour from any other 'true' definition of AGI. And so I think as long as it exhibits what we perceive as universal adaptability, and performance at making decisions when working with any environments and input types, this is likely where I/we will absolutely be calling something 'complete AGI'. Though before that I imagine we will be marketing LLMs as having reached 'language-based AGI' status. I don’t believe the above leads us to artificial superhuman intelligence in terms of conscious agent potential. For now what I believe might result in that is something that runs from a single unified neural network system. It should also be observed to be universally adaptable and performant at making decisions in all environments and consuming many input types. And then also, it should be continuously running within the network. It shouldn't halt for prompt, or pass through a text stage in the loop.
- mikewarot 3y agoWhen it runs locally on my hardware, with no connection to the rest of the world, and I get to know it and be its friend (or enemy).
- flashgordon 3y agoMay be when the world of Futurama is real?