9 ms·

The capabilities of AI post gpt3 have become extraordinary and clearly in many cases superhuman. However (as the article admits) there is still no general agre

The capabilities of AI post gpt3 have become extraordinary and clearly in many cases superhuman. However (as the article admits) there is still no general agreement of what AGI is, or how we (or even if we can) get there from here. What there is is a growing and often naïve excitement that anticipates it as coming into view, and unfortunately that will be accompanied by the hype-merchants desperate to be first to "call it". This article seems reasonable in some ways but unfortunately falls into the latter category with its title and sloganeering. "AGI" in the title of any article should be seen as a cautionary flag. On HN - if anywhere - we need to be on the alert for this.

ashoeafoot 2y ago
AGI is a annonymous good model coming around the corner with no company and no LLM researchers attached. AGI is when the LLM hype train threads are replaced with CEOs and let go researchers demanding UBI.
- MichaelZuo 2y ago
  Yeah formal agreement seems exceedingly unlikely. Since there isn’t even agreement on the defintion of “Artifical Intelligence”.
  - 9dev 2y ago
    Even worse, there isn’t even a working definition of "Intelligence"—neither in computer science nor biology.
    - dcow 2y ago
      This is why I suspect that slapping “Artificial” in the acronym is rather forward of us as a species. If we do end up eventually with something we consider intelligent, there won’t be anything artificial about it.
      - Jensson 2y ago
        Artificial means human made, if we made an intelligent being then it is artificial. What do you think artificial meant here?
- ben_w 2y ago
  It's easy to treat AGI as one thing — I did so myself before everyone's differing reaction to LLMs made me realise we all mean different things by each of the three letters of the initialism, and that none of those initials are really boolean valued. Given how Dutch disease[0] is described, I suspect that if the "G" (general) increases with fixed "I" (intelligence), as the proportion of economic activity for which the Pareto frontier is AI rather than human expands, I think humans will get pay rises for the remaining work right up until they get unemployable. On the other hand, if "G" is fully general and it's "I" which rises for a suitable cost[1], it goes through IQ 55 (displacing no workers) to IQ 100 (probably close to half of workers redundant, but mean of population doesn't have to equal mean of workforce), to IQ 145 (almost everyone redundant), to IQ 200 (definitionally renders everyone redundant). [0] https://en.wikipedia.org/wiki/Dutch_disease https://en.wikipedia.org/wiki/Dutch_disease [1] A fully-general AGI with the equivalent of IQ 200 on any possible test, still can't replace a single human if it costs 200 trillion USD per year to run.
jjeaff 2y ago
I suspect AGI will be one of those things that you can't describe it exactly, but you'll know it when you see it.
- ninetyninenine 2y ago
  I suspect everyone will call it a stochastic parrot because it did this one thing not right. And this will continue into the far far future even when it becomes sentient we will completely miss it.
  - Jensson 2y ago
    Once it pushed out most humans from white collar labor so the remaining humans work in blue collar jobs people wont say its just a stochastic parrot.
    - myk9001 2y ago
      Maybe, maybe not. Power loom pushed a lot of humans out of the textile factory jobs, yet noone claims power loom is the AGI.
      - Jensson 2y ago
        Not a lot, I mean basically everyone, to the point where most companies doesn't need to pay humans to think anymore.
        myk9001 2y ago
        Well, I'm too lazy to look up how many weavers were displaced back then and that's why I said a lot. Maybe all of them, since they weren't trained to operate the new machines. Anyway, sorry for a digression, my point is LLM replacing white collar workers doesn't necessarily imply it's generally intelligent -- it may but doesn't have to be. Although if it gets to a point where companies are running dark office buildings (by analogy with dark factories) -- yes, it's AGI by then.
deleted 2y ago
[deleted]
Zambyte 2y ago
I think a reasonable definition of intelligence is the application of reason on knowledge. An example of a system that is highly knowledgeable but has little to no reason would be an encyclopedia. An example of a system that is highly reasonable, but has little knowledge would be a calculator. Intelligent systems demonstrate both. Systems that have general intelligence are ones that are capable of applying reason to an unbounded domain of knowledge. Examples of such systems include: libraries, wikis, and forums like HN. These systems are not AGI, because the reasoning agents in each of these systems are organic (humans); they are more like a cyborg general intelligence. Artificial general intelligence are just systems that are fully artificial (ie: computer programs) that can apply reason to an unbounded domain of knowledge. We're here, and we have been for years. AGI sets no minimum as to how great the reasoning must be, but it's obvious to anyone who has used modern generative intelligence systems like LLMs that the technology can be used to reason about an unbounded domain of knowledge. If you don't want to take my word for it, maybe Peter Norvig can be more convincing: https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-...
- jimbokun 2y ago
  Excellent article and analysis. Surprised I missed it. It is very hard to argue with Norvig’s arguments that AGI has been around since at least 2023.
  - littlestymaar 2y ago
    It's not: whatever the way you define AGI, you cannot just ignore the key letter of the three letters acronym: G stands for “General”. You can argue that for the first time in the history we have an AI that deserves its name (unlike Deep blue or AlphaGo which aren't really about intelligence at all) but you cannot call that Artificial GENERAL Intelligence before it overcomes the “jagged intelligence” syndrome.
    - Zambyte 2y ago
      It sounds like you have a different definition of "general" in the context of intelligence from the one I shared. What is it?
yeahwhatever10 2y ago
This is the forum that fell the hardest for the superconductor hoax a few years ago. HN has no superiority leg to stand on.
- sandspar 2y ago
  Forums shed and gain members over time. Much of that cohort is gone.
  - Hasu 2y ago
    If anything it's been replaced with a far more naive and gullible cohort, not a more skeptical one.
- Seanambers 2y ago
  LK-99? The room temperature superconductor made by some guys in a small lab in Korea after 20 years of attempts - who doesn't want that to be real. Won´t say people fell for it though it was just the current happening at the time.
  - dcow 2y ago
    Uh… what do you call mass downvoting anything reasonably skeptical? I saw comments saying “I’ll wait until the data can be replicated to believe it” turn grey almost immediately on most threads. And after we knew it didn’t replicate it took the community the better part of a week to grieve and admit that the “it’s not a superconductor” results were real and not operator error.
  - Gerardo1 2y ago
    > who doesn't want that to be real. I think that's exactly the point the person you're responding to is calling out. That's a massive bias.
nightmunnas 2y ago
Low agreeableness will actually be extremely useful in many use cases, such as scientific discovery and of course programming assistance. It's amazing that this venue hasn't been explored more deeply.
- Jensson 2y ago
  Its much easier to sell an agreeable assistant than a disagreeable one, so it isn't that strange the alternative isn't explored.
- alwa 2y ago
  Why would a bad attitude be helpful in those domains? Are the human partners wont to deliver more effort when you’re mean to them? Are we talking about something other than Agreeableness in the personality research sense [0]? The strongest form of your argument I can think of is “willing to contradict you when it thinks you’re wrong”—but you can disagree agreeably, right? The current-gen LLMs certainly have with me, perhaps because my custom prompt encourages them to skepticism—but they do it so nicely! [0] https://en.m.wikipedia.org/wiki/Agreeableness https://en.m.wikipedia.org/wiki/Agreeableness
  - Jensson 2y ago
    > but you can disagree agreeably, right? No, the concepts are linked, agreeable people don't want to be rude and most people see disagreements as being rude no matter how you frame it. You can't call a woman overweight without being rude for example no matter how you frame it, but maybe you want an AI that tells you that you weigh too much.
    - smallwire 2y ago
      Good point, but calling a woman overweight isn't necessarily a disagreement.
j_timberlake 2y ago
The exact definition of AGI is pretty much the least interesting thing about AGI. It's basically bike-shedding at this point: arguing about something easy to understand instead of tackling the really hard questions like "how competent can AI get before it's too dangerous to be in the hands of flakey tech companies?"
- 9dev 2y ago
  > how competent can AI get before it's too dangerous to be in the hands of flakey tech companies? Ever heard of Pandora's Box? Yeah. That ship has sailed. No moratorium you could enact would, at this point, stop the innovation from happening, possibly even independently by multiple teams globally. Economic incentives are stacked in such a way that flakey tech companies will prioritise shareholder value over anything else. Whatever comes next will come, and all we can do is lean back and enjoy the show.
  - tsimionescu 2y ago
    Given the gigantic amount of compute power and storage needed to train and run LLMs, this is certainly not true. It is absolutely feasible for government to check every data center capable of advancing the state of the art in AI to ensure that no such research is taking place. Of course, the political will to do so doesn't exist to even a tiny extent. But if such a will existed, it would be far easier to enforce than the prevention of human cloning, and that one has been successfully implemented for decades now.
    - magic_hamster 2y ago
      If you just bow out of the AI race you are handing it to other countries where practices might not be subjected to the same standards. It's suicide to do this.
      - tsimionescu 2y ago
        That's only if countries can't agree on this. I just gave the example of human cloning, which has been banned globally. You can also look at nuclear non-proliferation, which has been largely successful (though not completely) despite huge incentives for any country to defy it.
mrshadowgoose 2y ago
I've always felt that trying to pin down the precise definition of AGI is as useless as trying to pin down "what it means to truly understand". It's a mental trap for smart people, that distracts them from focusing on the impacts of hard-to-define concepts like AGI. AGI doesn't need to be "called", and there is no need for anyone to come to an agreement as to what its precise definition is. But at some point, we will cross that hard-to-define threshold, and the economic effects will be felt almost immediately. We should probably be focusing on how to prepare society for those changes, and not on academic bullshit.
- throwup238 2y ago
  It's definitely a trap for those who aren't familiar with the existing academic work in philosophy, cognition, and neuroscience. There are no definitive answers but there are lots of relatively well developed ideas and concepts that everyone here on HN seems completely ignorant of, even though some of the ideas were developed by industry giants like Marvin Minsky. Stuff like society of minds (Minksy), embodied cognition (Varela, Rosch, and Thompson), connectionist or subsymbolic views (Rumelhart), multiple intelligences (Gardner), psychometric and factor-analytic theories (Carroll), and all the other work like E. Hutchins. They're far from just academic wankery, there's a lot of useful stuff in there, it's just completely ignored by the AI crowd.
dheera 2y ago
I spent some amount of time trying to create a stock/option trading bot to exploit various market inefficiencies that persist, and did a bunch of code and idea bouncing off these LLMs. What I fund is that even all the various incarnations of GPT 4+ and GPT o+ routinely kept falling for the "get rich quick" option strategies all over the internet that don't work. In cases where 95%+ of the information on the internet is misinformation, the current incarnations of LLMs have a really hard time sorting out and filtering out the 5% of information that's actually valid and useful. In that sense, current LLMs are not yet superhuman at all, though I do think we can eventually get there.
- jimbokun 2y ago
  So they are only as smart as most humans.
daxfohl 2y ago
Until you can boot one up, give it access to a VM video and audio feeds and keyboard and mouse interfaces, give it an email and chat account, tell it where the company onboarding docs are and expect them to be a productive team member, they're not AGI. So long as we need special protocols like MCP and A2A, rather than expecting them to figure out how to collaborate like a human, they're not AGI. The first step, my guess, is going to be the ability to work through github issues like a human, identifying which issues have high value, asking clarifying questions, proposing reasonable alternatives, knowing when to open a PR, responding to code review, merging or abandoning when appropriate. But we're not even very close to that yet. There's some of it, but from what I've seen most instances where this has been successful are low level things like removing old feature flags.
- rafaelmn 2y ago
  Just because we rely on vision to interface with computer software doesn't mean it's optimal for AI models. Having a specialized interface protocol is orthogonal to capability. Just like you could theoretically write code in a proportional font with notepad and run your tools through windows CMD - having an editor with syntax highlighting and monospaced font helps you read/navigate/edit, having tools/navigation/autocomplete etc. optimized for your flow makes you more productive and expands your capability, etc. If I forced you to use unnatural interfaces it would severely limit your capabilities as well because you'd have to dedicate more effort towards handling basic editing tasks. As someone who recently swapped to a split 36key keyboard with a new layout I can say this becomes immediately obvious when you try something like this. You take your typing/editing skills for granted - try switching your setup and see how your productivity/problem solving ability tanks in practice.
  - daxfohl 2y ago
    Agreed, but I also think to be called AGI, they should be capable of working through human interfaces rather than needing to have special interfaces created for them to get around their lack of AGI. The catch in this though isn't the ability to use these interfaces. I expect that will be easy. The hard part will be, once these interfaces are learned, the scope and search space of what they will be able to do is infinitely larger. And moreover our expectations will change in how we expect an AGI to handle itself when our way of working with it becomes more human. Right now we're claiming nascent AGI, but really much of what we're asking these systems to do have been laid out for them. A limited set of protocols and interfaces, and a targeted set of tasks to which we normally apply these things. And moreover our expectations are as such. We don't converse with them as with a human. Their search space is much smaller. So while they appear AGI in specific tasks, I think it's because we're subconsciously grading them on a curve. The only way we have to interact with them prejudices us to have a very low bar. That said, I agree that video feed and mouse is a terrible protocol for AI. But that said, I wouldn't be surprised if that's what we end up settling on. Long term, it's just going to be easier for these bots to learn and adapt to use human interfaces than for us to maintain two sets of interfaces for things, except for specific bot-to-bot cases. It's horribly inefficient, but in my experience efficiency never comes out ahead with each new generation of UIs.
Rebuff5007 2y ago
> clearly in many cases superhuman In what cases is it superhuman exactly? And what humans are you comparing against? I'd bet that for any discipline you chose, one could find an expert in that field that can trick any of today's post-gpt3 ais.
- firecall 2y ago
  I'd bet that for any discipline you chose, one could find an AI that can trick any of today's post-enlightenment humans.
xmprt 2y ago
Superhuman is an interesting term because it's very vague. In some sense, computers have always been superhuman. I don't know anyone who could do a 10 by 10 digit multiplication in a fraction of a second yet even the earliest computers were capable of that. Does that make them superhuman? Today's LLMs are definitely more impressive than a basic calculator but it's still hard to tell if there's anything human about what they're doing or if they're just really powerful calculators with amazing understanding and recall. Does that distinction even matter?
Mithriil 2y ago
Unless it's from LessWrong, I usually read these with a grain of salt.

ashoeafoot 2y ago

AGI is a annonymous good model coming around the corner with no company and no LLM researchers attached. AGI is when the LLM hype train threads are replaced with CEOs and let go researchers demanding UBI.

MichaelZuo 2y ago
Yeah formal agreement seems exceedingly unlikely. Since there isn’t even agreement on the defintion of “Artifical Intelligence”.
- 9dev 2y ago
  Even worse, there isn’t even a working definition of "Intelligence"—neither in computer science nor biology.
  - dcow 2y ago
    This is why I suspect that slapping “Artificial” in the acronym is rather forward of us as a species. If we do end up eventually with something we consider intelligent, there won’t be anything artificial about it.
    - Jensson 2y ago
      Artificial means human made, if we made an intelligent being then it is artificial. What do you think artificial meant here?
ben_w 2y ago
It's easy to treat AGI as one thing — I did so myself before everyone's differing reaction to LLMs made me realise we all mean different things by each of the three letters of the initialism, and that none of those initials are really boolean valued. Given how Dutch disease[0] is described, I suspect that if the "G" (general) increases with fixed "I" (intelligence), as the proportion of economic activity for which the Pareto frontier is AI rather than human expands, I think humans will get pay rises for the remaining work right up until they get unemployable. On the other hand, if "G" is fully general and it's "I" which rises for a suitable cost[1], it goes through IQ 55 (displacing no workers) to IQ 100 (probably close to half of workers redundant, but mean of population doesn't have to equal mean of workforce), to IQ 145 (almost everyone redundant), to IQ 200 (definitionally renders everyone redundant). [0] https://en.wikipedia.org/wiki/Dutch_disease https://en.wikipedia.org/wiki/Dutch_disease [1] A fully-general AGI with the equivalent of IQ 200 on any possible test, still can't replace a single human if it costs 200 trillion USD per year to run.

jjeaff 2y ago

I suspect AGI will be one of those things that you can't describe it exactly, but you'll know it when you see it.

ninetyninenine 2y ago
I suspect everyone will call it a stochastic parrot because it did this one thing not right. And this will continue into the far far future even when it becomes sentient we will completely miss it.
- Jensson 2y ago
  Once it pushed out most humans from white collar labor so the remaining humans work in blue collar jobs people wont say its just a stochastic parrot.
  - myk9001 2y ago
    Maybe, maybe not. Power loom pushed a lot of humans out of the textile factory jobs, yet noone claims power loom is the AGI.
    - Jensson 2y ago
      Not a lot, I mean basically everyone, to the point where most companies doesn't need to pay humans to think anymore.
      - myk9001 2y ago
        Well, I'm too lazy to look up how many weavers were displaced back then and that's why I said a lot. Maybe all of them, since they weren't trained to operate the new machines. Anyway, sorry for a digression, my point is LLM replacing white collar workers doesn't necessarily imply it's generally intelligent -- it may but doesn't have to be. Although if it gets to a point where companies are running dark office buildings (by analogy with dark factories) -- yes, it's AGI by then.

deleted 2y ago

[deleted]

Zambyte 2y ago

I think a reasonable definition of intelligence is the application of reason on knowledge. An example of a system that is highly knowledgeable but has little to no reason would be an encyclopedia. An example of a system that is highly reasonable, but has little knowledge would be a calculator. Intelligent systems demonstrate both. Systems that have general intelligence are ones that are capable of applying reason to an unbounded domain of knowledge. Examples of such systems include: libraries, wikis, and forums like HN. These systems are not AGI, because the reasoning agents in each of these systems are organic (humans); they are more like a cyborg general intelligence. Artificial general intelligence are just systems that are fully artificial (ie: computer programs) that can apply reason to an unbounded domain of knowledge. We're here, and we have been for years. AGI sets no minimum as to how great the reasoning must be, but it's obvious to anyone who has used modern generative intelligence systems like LLMs that the technology can be used to reason about an unbounded domain of knowledge. If you don't want to take my word for it, maybe Peter Norvig can be more convincing: https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-...

jimbokun 2y ago
Excellent article and analysis. Surprised I missed it. It is very hard to argue with Norvig’s arguments that AGI has been around since at least 2023.
- littlestymaar 2y ago
  It's not: whatever the way you define AGI, you cannot just ignore the key letter of the three letters acronym: G stands for “General”. You can argue that for the first time in the history we have an AI that deserves its name (unlike Deep blue or AlphaGo which aren't really about intelligence at all) but you cannot call that Artificial GENERAL Intelligence before it overcomes the “jagged intelligence” syndrome.
  - Zambyte 2y ago
    It sounds like you have a different definition of "general" in the context of intelligence from the one I shared. What is it?

yeahwhatever10 2y ago

This is the forum that fell the hardest for the superconductor hoax a few years ago. HN has no superiority leg to stand on.

sandspar 2y ago
Forums shed and gain members over time. Much of that cohort is gone.
- Hasu 2y ago
  If anything it's been replaced with a far more naive and gullible cohort, not a more skeptical one.
Seanambers 2y ago
LK-99? The room temperature superconductor made by some guys in a small lab in Korea after 20 years of attempts - who doesn't want that to be real. Won´t say people fell for it though it was just the current happening at the time.
- dcow 2y ago
  Uh… what do you call mass downvoting anything reasonably skeptical? I saw comments saying “I’ll wait until the data can be replicated to believe it” turn grey almost immediately on most threads. And after we knew it didn’t replicate it took the community the better part of a week to grieve and admit that the “it’s not a superconductor” results were real and not operator error.
- Gerardo1 2y ago
  > who doesn't want that to be real. I think that's exactly the point the person you're responding to is calling out. That's a massive bias.

nightmunnas 2y ago

Low agreeableness will actually be extremely useful in many use cases, such as scientific discovery and of course programming assistance. It's amazing that this venue hasn't been explored more deeply.

Jensson 2y ago
Its much easier to sell an agreeable assistant than a disagreeable one, so it isn't that strange the alternative isn't explored.
alwa 2y ago
Why would a bad attitude be helpful in those domains? Are the human partners wont to deliver more effort when you’re mean to them? Are we talking about something other than Agreeableness in the personality research sense [0]? The strongest form of your argument I can think of is “willing to contradict you when it thinks you’re wrong”—but you can disagree agreeably, right? The current-gen LLMs certainly have with me, perhaps because my custom prompt encourages them to skepticism—but they do it so nicely! [0] https://en.m.wikipedia.org/wiki/Agreeableness https://en.m.wikipedia.org/wiki/Agreeableness
- Jensson 2y ago
  > but you can disagree agreeably, right? No, the concepts are linked, agreeable people don't want to be rude and most people see disagreements as being rude no matter how you frame it. You can't call a woman overweight without being rude for example no matter how you frame it, but maybe you want an AI that tells you that you weigh too much.
  - smallwire 2y ago
    Good point, but calling a woman overweight isn't necessarily a disagreement.

j_timberlake 2y ago

The exact definition of AGI is pretty much the least interesting thing about AGI. It's basically bike-shedding at this point: arguing about something easy to understand instead of tackling the really hard questions like "how competent can AI get before it's too dangerous to be in the hands of flakey tech companies?"

9dev 2y ago
> how competent can AI get before it's too dangerous to be in the hands of flakey tech companies? Ever heard of Pandora's Box? Yeah. That ship has sailed. No moratorium you could enact would, at this point, stop the innovation from happening, possibly even independently by multiple teams globally. Economic incentives are stacked in such a way that flakey tech companies will prioritise shareholder value over anything else. Whatever comes next will come, and all we can do is lean back and enjoy the show.
- tsimionescu 2y ago
  Given the gigantic amount of compute power and storage needed to train and run LLMs, this is certainly not true. It is absolutely feasible for government to check every data center capable of advancing the state of the art in AI to ensure that no such research is taking place. Of course, the political will to do so doesn't exist to even a tiny extent. But if such a will existed, it would be far easier to enforce than the prevention of human cloning, and that one has been successfully implemented for decades now.
  - magic_hamster 2y ago
    If you just bow out of the AI race you are handing it to other countries where practices might not be subjected to the same standards. It's suicide to do this.
    - tsimionescu 2y ago
      That's only if countries can't agree on this. I just gave the example of human cloning, which has been banned globally. You can also look at nuclear non-proliferation, which has been largely successful (though not completely) despite huge incentives for any country to defy it.

mrshadowgoose 2y ago

I've always felt that trying to pin down the precise definition of AGI is as useless as trying to pin down "what it means to truly understand". It's a mental trap for smart people, that distracts them from focusing on the impacts of hard-to-define concepts like AGI. AGI doesn't need to be "called", and there is no need for anyone to come to an agreement as to what its precise definition is. But at some point, we will cross that hard-to-define threshold, and the economic effects will be felt almost immediately. We should probably be focusing on how to prepare society for those changes, and not on academic bullshit.

throwup238 2y ago
It's definitely a trap for those who aren't familiar with the existing academic work in philosophy, cognition, and neuroscience. There are no definitive answers but there are lots of relatively well developed ideas and concepts that everyone here on HN seems completely ignorant of, even though some of the ideas were developed by industry giants like Marvin Minsky. Stuff like society of minds (Minksy), embodied cognition (Varela, Rosch, and Thompson), connectionist or subsymbolic views (Rumelhart), multiple intelligences (Gardner), psychometric and factor-analytic theories (Carroll), and all the other work like E. Hutchins. They're far from just academic wankery, there's a lot of useful stuff in there, it's just completely ignored by the AI crowd.

dheera 2y ago

I spent some amount of time trying to create a stock/option trading bot to exploit various market inefficiencies that persist, and did a bunch of code and idea bouncing off these LLMs. What I fund is that even all the various incarnations of GPT 4+ and GPT o+ routinely kept falling for the "get rich quick" option strategies all over the internet that don't work. In cases where 95%+ of the information on the internet is misinformation, the current incarnations of LLMs have a really hard time sorting out and filtering out the 5% of information that's actually valid and useful. In that sense, current LLMs are not yet superhuman at all, though I do think we can eventually get there.

jimbokun 2y ago
So they are only as smart as most humans.

daxfohl 2y ago

Until you can boot one up, give it access to a VM video and audio feeds and keyboard and mouse interfaces, give it an email and chat account, tell it where the company onboarding docs are and expect them to be a productive team member, they're not AGI. So long as we need special protocols like MCP and A2A, rather than expecting them to figure out how to collaborate like a human, they're not AGI. The first step, my guess, is going to be the ability to work through github issues like a human, identifying which issues have high value, asking clarifying questions, proposing reasonable alternatives, knowing when to open a PR, responding to code review, merging or abandoning when appropriate. But we're not even very close to that yet. There's some of it, but from what I've seen most instances where this has been successful are low level things like removing old feature flags.

rafaelmn 2y ago
Just because we rely on vision to interface with computer software doesn't mean it's optimal for AI models. Having a specialized interface protocol is orthogonal to capability. Just like you could theoretically write code in a proportional font with notepad and run your tools through windows CMD - having an editor with syntax highlighting and monospaced font helps you read/navigate/edit, having tools/navigation/autocomplete etc. optimized for your flow makes you more productive and expands your capability, etc. If I forced you to use unnatural interfaces it would severely limit your capabilities as well because you'd have to dedicate more effort towards handling basic editing tasks. As someone who recently swapped to a split 36key keyboard with a new layout I can say this becomes immediately obvious when you try something like this. You take your typing/editing skills for granted - try switching your setup and see how your productivity/problem solving ability tanks in practice.
- daxfohl 2y ago
  Agreed, but I also think to be called AGI, they should be capable of working through human interfaces rather than needing to have special interfaces created for them to get around their lack of AGI. The catch in this though isn't the ability to use these interfaces. I expect that will be easy. The hard part will be, once these interfaces are learned, the scope and search space of what they will be able to do is infinitely larger. And moreover our expectations will change in how we expect an AGI to handle itself when our way of working with it becomes more human. Right now we're claiming nascent AGI, but really much of what we're asking these systems to do have been laid out for them. A limited set of protocols and interfaces, and a targeted set of tasks to which we normally apply these things. And moreover our expectations are as such. We don't converse with them as with a human. Their search space is much smaller. So while they appear AGI in specific tasks, I think it's because we're subconsciously grading them on a curve. The only way we have to interact with them prejudices us to have a very low bar. That said, I agree that video feed and mouse is a terrible protocol for AI. But that said, I wouldn't be surprised if that's what we end up settling on. Long term, it's just going to be easier for these bots to learn and adapt to use human interfaces than for us to maintain two sets of interfaces for things, except for specific bot-to-bot cases. It's horribly inefficient, but in my experience efficiency never comes out ahead with each new generation of UIs.

Rebuff5007 2y ago

> clearly in many cases superhuman In what cases is it superhuman exactly? And what humans are you comparing against? I'd bet that for any discipline you chose, one could find an expert in that field that can trick any of today's post-gpt3 ais.

firecall 2y ago
I'd bet that for any discipline you chose, one could find an AI that can trick any of today's post-enlightenment humans.

xmprt 2y ago

Superhuman is an interesting term because it's very vague. In some sense, computers have always been superhuman. I don't know anyone who could do a 10 by 10 digit multiplication in a fraction of a second yet even the earliest computers were capable of that. Does that make them superhuman? Today's LLMs are definitely more impressive than a basic calculator but it's still hard to tell if there's anything human about what they're doing or if they're just really powerful calculators with amazing understanding and recall. Does that distinction even matter?

Mithriil 2y ago

Unless it's from LessWrong, I usually read these with a grain of salt.