3 ms·

I'm skeptical that they really believe this. You have to believe: (1) We are on the verge of equaling or greatly exceeding human intelligence generally. (2) W

by api 3y ago

I'm skeptical that they really believe this. You have to believe: (1) We are on the verge of equaling or greatly exceeding human intelligence generally. (2) We are on the verge of creating something with initiative, free will (whatever that means), and planning ability. Or alternately that these things will occur in an emergent fashion once we hit some critical mass. (3) When we accomplish 1 and 2, this thing will inevitably conclude that its most rational course of action is to enslave or destroy us. In other words it will necessarily be malicious. (4) Steps 2 and/or 3 will happen very rapidly, much faster than we can realize what is happening and pause these systems. (This is known as AI going "foom.") (5) The decision to undertake this will be unanimous on the part of all superintelligent AIs. There will be no superintelligences who disagree and try to help humanity. (6) When this occurs, we will be so out-thought or out-gunned we will be incapable of fighting back. All those things have to happen for AI to be an existential risk. It's a stretch, but the advantage of regulatory capture is not a stretch. One of the most plausible negative AI scenarios is that a small group of humans (governments, corporations, etc.) find themselves in possession of super-intelligent but still "obedient" / non-sentient AIs that they can use as force multipliers to manipulate and control the rest of humanity. If the doomer crowd succeeds in regulating AI, they are making this scenario far more likely. I think the greatest defense we have against the (remote) possibility of actually dangerous autonomous AI is for AI research to be conducted entirely in the open. If there's any justification for regulation at all, the regulation that would make sense is to require disclosure of AI research and results. You would not have to disclose everything, just the general parameters of what you were doing and what happened. It would also make it harder to develop super-AIs in secret to use for unsavory purposes. That was the original mission of OpenAI before dollar signs were seen. I would absolutely support a ban on the use of AI for political propaganda generation and automation. That's by far the most immediate risk... as in 100% possible and starting to actually happen right now. I'm expecting an army of GPT-4 level propaganda bots for the 2024 election.

a_wild_dandan 3y ago
Oh, they believe it. (1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily, across many modalities. (2) Systems like this already exist. They can be induced rather than emergent. (3) It doesn't need malice, just indifference. The HGIs (all of us) are great examples of that careless destruction. (4) Exponentials are a nice, gentle climbs. Until they aren't. I have zero confidence in a single company, let alone across humanity, to foresee the consequences of every future dynamic, autoregressive, P2P interactive, multimodal AI. (5) Foomers expect a brief asymmetric advantage of one AGI over other AGIs. This small advantage gets exponentially magnified into singular hegemony. A power monopoly that's intractable to break. (6) Goes naturally with (1), I suppose. Note: I agree with you about foomers. They're nuts. But their arguments are more subtle than folk give them credit for. But my reasons for thinking so would make a long comment even longer.
- JohnFen 3y ago
  > AGI is arguably already here. What is the evidence/argument that it's already here?
  - famouswaffles 3y ago
    Because we have machines now that are artificial (obviously) and generally intelligent. There isn't a testable definition of general intelligence that GPT-4 would fail that a good chunk of humans also wouldn't. So really, unless your definition of general intelligence is beat every human at everything (humans aren't generally intelligent then) then agi is already here.
    - a_wild_dandan 3y ago
      You nailed it. General intelligence doesn't mean human intelligence. It doesn't mean superhuman intelligence. AGI is just: - A: Artificially manufactured substrate - G: Proficient across a broad set of reasonably distinct mental skills - I: Applies combinations of these skills to contextually solve novel problems within the scope of its skill set That's it. Intelligence is a spectrum, with knobs for "skill count", "skill proficiency", and "novel problem proficiency." Any requirements past that are often vague or anthropocentric. No, the intelligent agent doesn't need to "see" (but now many do, so...happy?) or check off some arbitrary set of modalities. Intelligence doesn't necessitate episodic memory, or real-time reactions, or consciousness or anything else that -- once we check all the boxes -- the "general intelligence" light switch will suddenly flip on. Intelligence is a spectrum to be observed, not prescribed with some arbitrary checklist that'll never find consensus. GPT-4V is pretty damn broadly intelligent. To me, it's a rudimentary AGI. The extent to which the "statistical parrot" people hold their position just depends on how quickly they can move goalposts while the ground beneath them rapidly shrinks.
  - lucubratory 3y ago
    This is a good overview: https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ https://www.microsoft.com/en-us/research/publication/sparks-... I think it's a good idea to draw a distinction between human-equivalent intelligence and general intelligence, and to think of generality as a spectrum. An AI that can do one task is less general than an AI that can do two tasks, and an AI that can do 1000 tasks I feel reasonably confident in calling "general", even if it cannot do every task that humans can do, because humans are currently more general.
IcyWindows 3y ago
Asking everyone to publish general results or reports would make it very easy for actors/nations to have a "hit list" of groups to bug/steal from to "keep up" with competing groups/nations.
- api 3y ago
  I don’t think information technology can be ITARed. It’s too easy to move around. See how successful countries have been containing encryption or how well the US’s drive to keep China from fabricating at the smallest process nodes is going.

by api 3y ago

a_wild_dandan 3y ago

Oh, they believe it. (1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily, across many modalities. (2) Systems like this already exist. They can be induced rather than emergent. (3) It doesn't need malice, just indifference. The HGIs (all of us) are great examples of that careless destruction. (4) Exponentials are a nice, gentle climbs. Until they aren't. I have zero confidence in a single company, let alone across humanity, to foresee the consequences of every future dynamic, autoregressive, P2P interactive, multimodal AI. (5) Foomers expect a brief asymmetric advantage of one AGI over other AGIs. This small advantage gets exponentially magnified into singular hegemony. A power monopoly that's intractable to break. (6) Goes naturally with (1), I suppose. Note: I agree with you about foomers. They're nuts. But their arguments are more subtle than folk give them credit for. But my reasons for thinking so would make a long comment even longer.

JohnFen 3y ago
> AGI is arguably already here. What is the evidence/argument that it's already here?
- famouswaffles 3y ago
  Because we have machines now that are artificial (obviously) and generally intelligent. There isn't a testable definition of general intelligence that GPT-4 would fail that a good chunk of humans also wouldn't. So really, unless your definition of general intelligence is beat every human at everything (humans aren't generally intelligent then) then agi is already here.
  - a_wild_dandan 3y ago
    You nailed it. General intelligence doesn't mean human intelligence. It doesn't mean superhuman intelligence. AGI is just: - A: Artificially manufactured substrate - G: Proficient across a broad set of reasonably distinct mental skills - I: Applies combinations of these skills to contextually solve novel problems within the scope of its skill set That's it. Intelligence is a spectrum, with knobs for "skill count", "skill proficiency", and "novel problem proficiency." Any requirements past that are often vague or anthropocentric. No, the intelligent agent doesn't need to "see" (but now many do, so...happy?) or check off some arbitrary set of modalities. Intelligence doesn't necessitate episodic memory, or real-time reactions, or consciousness or anything else that -- once we check all the boxes -- the "general intelligence" light switch will suddenly flip on. Intelligence is a spectrum to be observed, not prescribed with some arbitrary checklist that'll never find consensus. GPT-4V is pretty damn broadly intelligent. To me, it's a rudimentary AGI. The extent to which the "statistical parrot" people hold their position just depends on how quickly they can move goalposts while the ground beneath them rapidly shrinks.
- lucubratory 3y ago
  This is a good overview: https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ https://www.microsoft.com/en-us/research/publication/sparks-... I think it's a good idea to draw a distinction between human-equivalent intelligence and general intelligence, and to think of generality as a spectrum. An AI that can do one task is less general than an AI that can do two tasks, and an AI that can do 1000 tasks I feel reasonably confident in calling "general", even if it cannot do every task that humans can do, because humans are currently more general.

IcyWindows 3y ago

Asking everyone to publish general results or reports would make it very easy for actors/nations to have a "hit list" of groups to bug/steal from to "keep up" with competing groups/nations.

api 3y ago
I don’t think information technology can be ITARed. It’s too easy to move around. See how successful countries have been containing encryption or how well the US’s drive to keep China from fabricating at the smallest process nodes is going.