3 ms·

I think the best argument I have against AGI's inevitability, is the fact it's not required for ML tools to be useful. Very few things are improved with a gener

I think the best argument I have against AGI's inevitability, is the fact it's not required for ML tools to be useful. Very few things are improved with a generalist behind the wheel. "AGI" has sci-fi vibes around it, which I think where most of the fascination is. "ML getting better" doesn't *have to* mean further anthroaormorphization of computers, especially if say, your AI driven car is not significantly improved by describing how many times the letter s appears in strawberry or being able to write a poem. If a custom model/smaller model does equal or even a little worse on a specific target task, but has MUCH lower running costs and much lower risk of abuse, then that'll be the future. I can totally see a world where anything in the general category of "AI" becomes more and more boring, up to a point where we forget that they're non-deterministic programs. That's kind of AGI? They aren't all generalists, and the few generalist "AGI-esque" tools people interact with on a day to day basis will most likely be intentionally underpowered for cost reasons. But it's still probably discussed like "the little people in the machine". Which is good enough.

xpe 2y ago
The core argument has a logical gap; it doesn’t matter if most (present) ML applications need AGI or not. What matters is the value proposition of AGI, independent of how we currently conceive of ML applications. Focus on this question: “Is general intelligence at some level valuable at a particular price point?” General intelligence is generally valuable, so there is a pressure to advance it, whether by improving capability and/or decreasing cost. Now the question becomes empirical — what kind of general intelligence can be built that achieves certain parameters? Aiming to exceed the power efficiency of the human brain is a tempting target. (Whether it is wise is another question; what happens when human intelligence doesn’t provide competitive advantage?)
- graypegg 2y ago
  That's a fair point I hadn't considered! If intelligence is valuable in humans, and some cost factor of advancing human intelligence can be surpassed digitally like this, (I don't know how you'd measure intellect efficiency, somehow involving calories-in/good-descisions-out or something?) then there's economic incentive. But that feels very far off, even in the current exponential curve of efficiency we're on. Can't go on forever.
  - xpe 2y ago
    Some suggestions: - Don't assume only digital intelligence. - Intelligence is fundamentally valuable to humans. No need to add any caveats. > But that feels very far off, even in the current exponential curve of efficiency we're on. Can't go on forever. What exactly feels far off to you? In many cases, sure, exponential growth can't go on forever, but one has to be careful about spelling out what you mean. What are your axes? Don't forget that each successive technology may introduce a new growth pattern. One has to be quite careful in analyzing and forecasting these things.

xpe 2y ago

The core argument has a logical gap; it doesn’t matter if most (present) ML applications need AGI or not. What matters is the value proposition of AGI, independent of how we currently conceive of ML applications. Focus on this question: “Is general intelligence at some level valuable at a particular price point?” General intelligence is generally valuable, so there is a pressure to advance it, whether by improving capability and/or decreasing cost. Now the question becomes empirical — what kind of general intelligence can be built that achieves certain parameters? Aiming to exceed the power efficiency of the human brain is a tempting target. (Whether it is wise is another question; what happens when human intelligence doesn’t provide competitive advantage?)

graypegg 2y ago
That's a fair point I hadn't considered! If intelligence is valuable in humans, and some cost factor of advancing human intelligence can be surpassed digitally like this, (I don't know how you'd measure intellect efficiency, somehow involving calories-in/good-descisions-out or something?) then there's economic incentive. But that feels very far off, even in the current exponential curve of efficiency we're on. Can't go on forever.
- xpe 2y ago
  Some suggestions: - Don't assume only digital intelligence. - Intelligence is fundamentally valuable to humans. No need to add any caveats. > But that feels very far off, even in the current exponential curve of efficiency we're on. Can't go on forever. What exactly feels far off to you? In many cases, sure, exponential growth can't go on forever, but one has to be careful about spelling out what you mean. What are your axes? Don't forget that each successive technology may introduce a new growth pattern. One has to be quite careful in analyzing and forecasting these things.