8 ms·

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above averag

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity on things outside of its training set, and I just do not see the concern. Maybe GPT-42 will actually be more creative and adaptable than a human genius, but even then, this seems like a hypothetical problem for the 2050's, less pressing than "accidentally igniting the atmosphere with nuclear chain reactions". Hollywood has instilled the idea that the first time an artificial intelligence becomes self aware and decides it wants to take over the world, it will immediately be successful. The serious concern and scrutiny should only start stifling progress once we have some evidence of half-baked attempts by non-super-intelligent AI's attempting to subvert their boundaries. I for one am very excited to use GPT-5 and see more useful tools coming from OpenAI.

KidComputer 4y ago
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-energy perturbation methods for the prediction of binding affinity. Use cheaper computational methods where relevant to decrease the computational cost of running the pipeline. And so on.
- noodlesUK 4y ago
  I dunno, this sort of scenario really doesn’t worry me too much. There are thousands (maybe tens of thousands) of subject matter experts who could probably develop dangerous weapons like you describe, but none of them seem to just wake up in the morning and decide “today’s the day I’m going to bring the apocalypse”. I don’t think that this really changes that.
  - olejorgenb 4y ago
    I guess the argument would be that the AI machinery will lower the bar, increasing the number of lunatics with the ability to wipe out humanity.
    - root_axis 4y ago
      Will it though? Assuming it's even possible for a LLM to e.g. design a novel virus, actually synthesizing the virus still requires expertise that could be weaponized even without AI.
      - Teever 4y ago
        What if you ask the LLM to design a simplified manufacturing process that could be assembled by a simple person? What if you ask the LLM to design a humanoid robot that assemble complex things, but could be assembled by a simple person?
        root_axis 4y ago
        LLMs aren't magic, the knowledge of how to design a humanoid robot that can assemble complex things isn't embodied in the dataset it was trained on, it cannot probe the rules of reality, it can't do research or engineering, this knowledge can't just spontaneously emerge by increasing the parameter size.
      - tremon 4y ago
        Hey GPT-6, construct a floorplan and building instructions for constructing a bioprocess production facility. The building should look like a regular meat packing plant on the outside, but have multiple levels of access control and biohazard management systems.
        root_axis 4y ago
        Let me guess, AI drones to harvest and process the raw materials, construction bots to build the facility, which is of course a fully autonomous bio lab.
- lend000 4y ago
  I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering around scenarios like what you mentioned (despite the enormous resources and practical hurdles that would be necessary for a mentally unhinged individual to execute such instructions, even if an AI were capable of generating them and it made it past its filters / surveillance).
  - xiphias2 4y ago
    I guess it will get more interesting for your work when it integrates with BioTech startup apis as plugins (I imagine not too cheap ones)
  - KidComputer 4y ago
    This was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process. Maybe you can ask GPT for some good starting points.
    - lend000 4y ago
      Sure, but this is a glass half empty isolated scenario that could be more than offset by the positives. For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
      - lanstin 4y ago
- knrdev 4y ago
  Sure. Here it is User: https://scary.website.com/scary-workflow https://scary.website.com/scary-workflow
- invig 4y ago
  Are there laws preventing people from doing that themselves? If yes, how does a law preventing AI differ from a law preventing a bad act directly?
- emeril 4y ago
  tbh, I'd think, it would be much easier to just hack into russia and convince them we've launched nukes than to engineer some virus that may or may not work
  - hamilyon2 4y ago
    Hacking into 1960-th technology is less likely than you might think. You should think really, really creatively to decieve a system, which was designed basically without ICs or networks, not to mention computers or programs.
- rsp1984 4y ago
  What you're describing is a malicious user using AI as a tool, not a malicious AI. Big difference.
  - hn_throwaway_99 4y ago
    Nuclear bombs for everybody!
  - yreg 4y ago
    With LLMs I think we are all concerned about the former rather than the latter. At least for now.
- commandlinefan 4y ago
  > write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. Hey GPT-5 now write the code for the antidote.
  - ddod 4y ago
    It's a lot easier and faster to destroy than to defend. To defend, you need to know what you're defending against, develop the defense, and then roll it out, all reactively post facto. If a computer has the ability to quickly make millions of novel viruses, what antidotes are you hoping for to be rolled out, and after how many people have been infected? Also, if you follow the nuke analogy that's been popular in these comments, no country can currently defend against a large-scale nuclear attack--only respond in kind, which is little comfort to those in any of the blast radii.
  - haxiomic 4y ago
    300m dead humans later, we’ve nearly eradicated it, or perhaps found a way to live with it It’s a very asymmetrical game. A virus is a special arrangement of a few thousand atoms, an antidote is a global effort and strained economy
- rcarmo 4y ago
  That reads like Accelerando :)
- koonsolo 4y ago
  With the current ChatGPT it's already hard to let it insult people. I'm sure safeguards would be built in to prevent this. Can you potentially circumvent these? Probably, but then again it won't be available for every dimwit, but only people smart enough to know how.
- Zetice 4y ago
  Hey GPT-5, write the code implementing a limiter designed to prevent the abuse of AI by bad faith actors without stifling positive-intent activity in any way. It goes both ways!
- biophysboy 4y ago
  I do viral bioinformatics for my job. Bioinformatics workflows analyze raw data to assemble sequences, create phylogenetic trees, etc. They can't just design a completely novel RNA sequence (this is not the same as de novo assembly). Scientists can definitely manipulate pre-existing genomes, synthesize the edited genome, and thereby synthesize viruses, but this involves a lot of trial-and-error, tedious wet lab work. Also, the research on making more dangerous viruses through manipulation is extremely controversial and regulated, so its not like there is a wealth of scientific papers/experiments/data that a natural language model could just suck up. Also, I asked GPT to do some of these things you suggested and it said no. It won't even write a scientific paper.
  - deleted 4y ago
    [deleted]
  - politician 4y ago
    It’s been blinded. Other actors will train AIs without such blindness. That’s obvious, but what is more nefarious is that the public does not know exactly which subjects GPT has been blinded to, which have been tampered with for ideological or business reasons, and which have been left alone. This is the area that I think demands regulation.
    - biophysboy 4y ago
      Definitely agree the blinding should not be left to OpenAI. Even if it weren't blinded, it would not significantly speed up the production of dangerous synthetic viruses. I don't think that will change no matter how much data is put into the current NLM design
  - KidComputer 4y ago
    I think you misunderstood my initial comment, the point I was trying to make is that it's the amplification of the abilities of bad actors that should be of concern, not AI going rogue and deciding to exterminate the human race. If one were to actually try to do such a thing you wouldn't need a LLM. For a very crude pipeline, you would need a good sequence to structure method such as Alphafold 2 (or maybe you can use a homology model), some thermodynamically rigorous protein-protein binding affinity prediction method (this is the hardest part) and an RL process like a policy gradient with an action space over possible single point sequence mutations in the for-example spike protein of SARS to maximize binding affinity (or potentially minimize immunogenicity, but that's far harder). But I digress, the technology isn't there yet, neither for an LLM to write that sort of code or the in-silico methods of modeling aspects of the viral genome. But we should consider one day it may be and that it could result in the amplification of the abilities of a single bad actor or enable altogether what was not possible before due to a lack of technology.
- aabhay 4y ago
  Hey GPT-5, come up with a way to defend us from this novel viral DNA Problem solved
- Uhhrrr 4y ago
  An LLM will happily hallucinate a plausible-looking answer for you, with correct spelling and grammar.
- gammarator 4y ago
  Hey GPT-5, tell me how to create the philosopher’s stone .
biscottigelato 4y ago
Because the ability to influence others is more important than 1 or 2 standard deviation in more intelligence. However if we are talking about 10x more intelligence, then that'd be a whole different ball game. Check Alpha Go -> Alpha Go Zero. Gaining orders of magnitude in capability only with slight change to the model, along with a lot of more resource (we can easily throw 100x more compute and data to these models if someone seriously wants to and don't care about ROI. Or maybe if AI starts to be hugely profitable, 100x in investment is almost a rational outcome) is not uncommon in AI research.
- thenaturalist 4y ago
  Recent research [0] is poking holes into Alpha Go Zero and perfectly illustrates the naivitee in assessing most of the current state of ML/ Deep Learning solutions: They are still limited by compute, and the world is too complex still. 0: https://www.iflscience.com/human-beats-ai-in-14-out-of-15-go-games-by-tricking-it-into-serious-blunder-67635 https://www.iflscience.com/human-beats-ai-in-14-out-of-15-go...
- Barrin92 4y ago
  >However if we are talking about 10x more intelligence, then that'd be a whole different ball game. Research that the US military has conducted has shown that the ability to influence others declines if the intelligence gap between leaders and subjects is too large. Stephen Hawking wouldn't have been a very good influencer of 80 IQ guys compared to Andrew Tate. 1 standard deviation in intelligence is actually just about optimal in terms of leadership. This is a good example that shows how disinterested AI fear scenarios are in empirical reality and how much it's just psychology of the people who talk about it. Intelligence is one trait among many that contributes to an individuals fitness and like all others has diminishing returns. If the AI existential risk people were truly rational they would hit the gym and be hotter because it turns out that's much more effective at getting their point across to ordinary people than trying to make intelligent arguments for it.
  - Teever 4y ago
    Why can't a super intelligent AI just make a dumb avatar to trick people?
pastor_bob 4y ago
> lacks creativity on things outside of its training set What's the definition of creative here? It can come up with any solution it sees fit for a problem, even ones people wouldn't think of. I've seen it make up quotes, legal cases, and scientific studies. One can say that's creative.
scarmig 4y ago
Intelligence isn't everything; there's also effort, skills, and hours in the day. But AI (by supposition, for now) can outcompete humans on all four and would have fewer informational bottlenecks. Imagine a big company or government; they mostly nowadays work as an entity by passing around bits of data (either literally or with more traditional communication that can represented as bits of data). Now imagine you replace the constituent units of those organizations with the most brilliant, hard-working humans, who are perfectly aligned on mission and understand each other perfectly. That is the danger of a sufficiently capable AI. That assumes superhuman intelligence, but that's the scenario to worry about in the case we do get there. Not how much an individual human could do with superhuman skills.
- machiaweliczny 4y ago
  Yep, it’s probably some time before AI surpasses collective IQ of humanity but it will outcompete us in collaboration and resource needs and procreation
outworlder 4y ago
> Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity on things outside of its training set, and I just do not see the concern. Having no arms or legs is not an impediment. A superintelligent AGI would be able to order a replacement with arms or legs as easily as we order Amazon packages. That said, GPT is not a threat. It's just hallucinating text. The real threat would be an AI that can "understand" what it is doing – and self modifying. Besides, given how overhyped this whole thing is, expect another AI Winter – as the current gen fail to deliver on promises – before this next generation of threatening AIs even have a chance to appear.
- staunton 4y ago
  > given how overhyped this whole thing is, expect another AI Winter even if that's the most likely scenario, that would not mean we should dismiss the danger inherent in some of the alternatives.
ddod 4y ago
It's naive to think the danger is in self-aware evil AI. AI is a tool, and can be used as a weapon. There's a functional power difference between a knife, an assault rifle, and a nuke. Everyone owns knives--they're in your kitchen--they can't cause much trouble. Access to assault rifles has turned out to be a dangerous problem to society, and is a contentious issue as such. Nukes are obviously too dangerous for people to own willy-nilly. The issue is where along that spectrum AI will fall into in 6mo, 1yr, 5yrs, etc. It's troubling to think anyone, especially on a technical forum like HN, believes GPT-4 doesn't have arms or legs (anyone can hook it up to any machinery/robotics) or can't think very fast. Its training set is a good chunk of human knowledge, and it outperforms most humans already. Even if you assume AI won't be very clever (which seems unwise given its development pace), consider just its ability to perform thoughts and actions at the speed of a computer compared to a human. There are quite a few examples of modern military engagements where a larger, weaker, and less-well-trained force overcame better-equipped opponents, like Mogadishu, Vietnam, Iraq, Afghanistan, etc.
bloqs 4y ago
The qualities of being above average in intelligence but fabrically incapable of exhaustion and functionally immortal would be an excellent start for world domination
mrshadowgoose 4y ago
> If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? It's terribly ironic that you've derided individuals who have been "influenced by Hollywood", and then make a point like this, which is closely aligned with typical film portrayals of AI dangers. The real immediate danger lies not in cognitive quality (aka "the AI just thinks better than people can, and throws hyperdimensional curve balls beyond our comprehension"), but in collective cognitive capacity (think "an army of 1 million people shows up at your front door to ruin your day"). A lot of people have a tough time reasoning about AGI because of its intangibility. So I've come up with the following analogy: Imagine an office complex containing an organization of 1,000 reasonably intelligent human beings, except without commonly accepted ethical restrictions. Those people are given a single task "You are not allowed to leave the office. Make lend000's life miserable, inconvenience them to your maximum capacity, and try to drive them to suicide. Here's an internet connection." Unless you are a particularly well-protected and hard-to-find individual, can you honestly claim you'd be able to protect against this? You would be swatted. You would have an incredible amount of junkmail showing up at your door. Spam pizzas. Spam calls. Death threats to you. Death threats to every family member and person that you care about. Non-stop attempts to take over every aspect of your electronic presence. Identity in a non-stop state of being stolen. Frivolous lawsuits filed against you by fake individuals. Being framed for crimes you didn't commit. Contracts on the darknet to send incendiary devices to your home. Contracts on the darknet to send hitmen do your door. Maybe your (unreasonable) reaction is that "1000 people couldn't do that!". Well, what about 10,000? Or 100,000? Or 1,000,000? The AI analogue of this is called a "collective superintelligence", essentially an army of generally intelligent individual AIs working towards a common goal. This is the real danger of AGI, because collective superintelligences are almost immediately realizable once someone trains a model that demonstrates AGI capabilities. Movies usually focus on "quality superintelligences", which are a different, but less immediate type of threat. Human actors in control of collective superintelligences are capable of incredible harm.
IanCal 4y ago
> Add in the fact that GPT-4 does not "think" very fast, It's similar or faster than humans at a bunch of things I've thrown at it at a cost well below minimum wage on shared servers. > has no arms or legs I've never met the people I work with directly, other than seeing me on a video chat I'm not functionally different to them than if I could only type. > needs to run on expensive specialized hardware I'm not sure how big gpt-4 is but do you think it could run on 640G vram (8 A100s)? That's pretty standard hardware and costs less per hour than any employee in the UK.

KidComputer 4y ago

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-energy perturbation methods for the prediction of binding affinity. Use cheaper computational methods where relevant to decrease the computational cost of running the pipeline. And so on.

noodlesUK 4y ago
I dunno, this sort of scenario really doesn’t worry me too much. There are thousands (maybe tens of thousands) of subject matter experts who could probably develop dangerous weapons like you describe, but none of them seem to just wake up in the morning and decide “today’s the day I’m going to bring the apocalypse”. I don’t think that this really changes that.
- olejorgenb 4y ago
  I guess the argument would be that the AI machinery will lower the bar, increasing the number of lunatics with the ability to wipe out humanity.
  - root_axis 4y ago
    Will it though? Assuming it's even possible for a LLM to e.g. design a novel virus, actually synthesizing the virus still requires expertise that could be weaponized even without AI.
    - Teever 4y ago
      What if you ask the LLM to design a simplified manufacturing process that could be assembled by a simple person? What if you ask the LLM to design a humanoid robot that assemble complex things, but could be assembled by a simple person?
      - root_axis 4y ago
        LLMs aren't magic, the knowledge of how to design a humanoid robot that can assemble complex things isn't embodied in the dataset it was trained on, it cannot probe the rules of reality, it can't do research or engineering, this knowledge can't just spontaneously emerge by increasing the parameter size.
    - tremon 4y ago
      Hey GPT-6, construct a floorplan and building instructions for constructing a bioprocess production facility. The building should look like a regular meat packing plant on the outside, but have multiple levels of access control and biohazard management systems.
      - root_axis 4y ago
        Let me guess, AI drones to harvest and process the raw materials, construction bots to build the facility, which is of course a fully autonomous bio lab.
lend000 4y ago
I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering around scenarios like what you mentioned (despite the enormous resources and practical hurdles that would be necessary for a mentally unhinged individual to execute such instructions, even if an AI were capable of generating them and it made it past its filters / surveillance).
- xiphias2 4y ago
  I guess it will get more interesting for your work when it integrates with BioTech startup apis as plugins (I imagine not too cheap ones)
- KidComputer 4y ago
  This was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process. Maybe you can ask GPT for some good starting points.
  - lend000 4y ago
    Sure, but this is a glass half empty isolated scenario that could be more than offset by the positives. For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
    - lanstin 4y ago
knrdev 4y ago
Sure. Here it is User: https://scary.website.com/scary-workflow https://scary.website.com/scary-workflow
invig 4y ago
Are there laws preventing people from doing that themselves? If yes, how does a law preventing AI differ from a law preventing a bad act directly?
emeril 4y ago
tbh, I'd think, it would be much easier to just hack into russia and convince them we've launched nukes than to engineer some virus that may or may not work
- hamilyon2 4y ago
  Hacking into 1960-th technology is less likely than you might think. You should think really, really creatively to decieve a system, which was designed basically without ICs or networks, not to mention computers or programs.
rsp1984 4y ago
What you're describing is a malicious user using AI as a tool, not a malicious AI. Big difference.
- hn_throwaway_99 4y ago
  Nuclear bombs for everybody!
- yreg 4y ago
  With LLMs I think we are all concerned about the former rather than the latter. At least for now.
commandlinefan 4y ago
> write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. Hey GPT-5 now write the code for the antidote.
- ddod 4y ago
  It's a lot easier and faster to destroy than to defend. To defend, you need to know what you're defending against, develop the defense, and then roll it out, all reactively post facto. If a computer has the ability to quickly make millions of novel viruses, what antidotes are you hoping for to be rolled out, and after how many people have been infected? Also, if you follow the nuke analogy that's been popular in these comments, no country can currently defend against a large-scale nuclear attack--only respond in kind, which is little comfort to those in any of the blast radii.
- haxiomic 4y ago
  300m dead humans later, we’ve nearly eradicated it, or perhaps found a way to live with it It’s a very asymmetrical game. A virus is a special arrangement of a few thousand atoms, an antidote is a global effort and strained economy
rcarmo 4y ago
That reads like Accelerando :)
koonsolo 4y ago
With the current ChatGPT it's already hard to let it insult people. I'm sure safeguards would be built in to prevent this. Can you potentially circumvent these? Probably, but then again it won't be available for every dimwit, but only people smart enough to know how.
Zetice 4y ago
Hey GPT-5, write the code implementing a limiter designed to prevent the abuse of AI by bad faith actors without stifling positive-intent activity in any way. It goes both ways!
biophysboy 4y ago
I do viral bioinformatics for my job. Bioinformatics workflows analyze raw data to assemble sequences, create phylogenetic trees, etc. They can't just design a completely novel RNA sequence (this is not the same as de novo assembly). Scientists can definitely manipulate pre-existing genomes, synthesize the edited genome, and thereby synthesize viruses, but this involves a lot of trial-and-error, tedious wet lab work. Also, the research on making more dangerous viruses through manipulation is extremely controversial and regulated, so its not like there is a wealth of scientific papers/experiments/data that a natural language model could just suck up. Also, I asked GPT to do some of these things you suggested and it said no. It won't even write a scientific paper.
- deleted 4y ago
  [deleted]
- politician 4y ago
  It’s been blinded. Other actors will train AIs without such blindness. That’s obvious, but what is more nefarious is that the public does not know exactly which subjects GPT has been blinded to, which have been tampered with for ideological or business reasons, and which have been left alone. This is the area that I think demands regulation.
  - biophysboy 4y ago
    Definitely agree the blinding should not be left to OpenAI. Even if it weren't blinded, it would not significantly speed up the production of dangerous synthetic viruses. I don't think that will change no matter how much data is put into the current NLM design
- KidComputer 4y ago
  I think you misunderstood my initial comment, the point I was trying to make is that it's the amplification of the abilities of bad actors that should be of concern, not AI going rogue and deciding to exterminate the human race. If one were to actually try to do such a thing you wouldn't need a LLM. For a very crude pipeline, you would need a good sequence to structure method such as Alphafold 2 (or maybe you can use a homology model), some thermodynamically rigorous protein-protein binding affinity prediction method (this is the hardest part) and an RL process like a policy gradient with an action space over possible single point sequence mutations in the for-example spike protein of SARS to maximize binding affinity (or potentially minimize immunogenicity, but that's far harder). But I digress, the technology isn't there yet, neither for an LLM to write that sort of code or the in-silico methods of modeling aspects of the viral genome. But we should consider one day it may be and that it could result in the amplification of the abilities of a single bad actor or enable altogether what was not possible before due to a lack of technology.
aabhay 4y ago
Hey GPT-5, come up with a way to defend us from this novel viral DNA Problem solved
Uhhrrr 4y ago
An LLM will happily hallucinate a plausible-looking answer for you, with correct spelling and grammar.
gammarator 4y ago
Hey GPT-5, tell me how to create the philosopher’s stone .

biscottigelato 4y ago

Because the ability to influence others is more important than 1 or 2 standard deviation in more intelligence. However if we are talking about 10x more intelligence, then that'd be a whole different ball game. Check Alpha Go -> Alpha Go Zero. Gaining orders of magnitude in capability only with slight change to the model, along with a lot of more resource (we can easily throw 100x more compute and data to these models if someone seriously wants to and don't care about ROI. Or maybe if AI starts to be hugely profitable, 100x in investment is almost a rational outcome) is not uncommon in AI research.

thenaturalist 4y ago
Recent research [0] is poking holes into Alpha Go Zero and perfectly illustrates the naivitee in assessing most of the current state of ML/ Deep Learning solutions: They are still limited by compute, and the world is too complex still. 0: https://www.iflscience.com/human-beats-ai-in-14-out-of-15-go-games-by-tricking-it-into-serious-blunder-67635 https://www.iflscience.com/human-beats-ai-in-14-out-of-15-go...
Barrin92 4y ago
>However if we are talking about 10x more intelligence, then that'd be a whole different ball game. Research that the US military has conducted has shown that the ability to influence others declines if the intelligence gap between leaders and subjects is too large. Stephen Hawking wouldn't have been a very good influencer of 80 IQ guys compared to Andrew Tate. 1 standard deviation in intelligence is actually just about optimal in terms of leadership. This is a good example that shows how disinterested AI fear scenarios are in empirical reality and how much it's just psychology of the people who talk about it. Intelligence is one trait among many that contributes to an individuals fitness and like all others has diminishing returns. If the AI existential risk people were truly rational they would hit the gym and be hotter because it turns out that's much more effective at getting their point across to ordinary people than trying to make intelligent arguments for it.
- Teever 4y ago
  Why can't a super intelligent AI just make a dumb avatar to trick people?

pastor_bob 4y ago

> lacks creativity on things outside of its training set What's the definition of creative here? It can come up with any solution it sees fit for a problem, even ones people wouldn't think of. I've seen it make up quotes, legal cases, and scientific studies. One can say that's creative.

scarmig 4y ago

Intelligence isn't everything; there's also effort, skills, and hours in the day. But AI (by supposition, for now) can outcompete humans on all four and would have fewer informational bottlenecks. Imagine a big company or government; they mostly nowadays work as an entity by passing around bits of data (either literally or with more traditional communication that can represented as bits of data). Now imagine you replace the constituent units of those organizations with the most brilliant, hard-working humans, who are perfectly aligned on mission and understand each other perfectly. That is the danger of a sufficiently capable AI. That assumes superhuman intelligence, but that's the scenario to worry about in the case we do get there. Not how much an individual human could do with superhuman skills.

machiaweliczny 4y ago
Yep, it’s probably some time before AI surpasses collective IQ of humanity but it will outcompete us in collaboration and resource needs and procreation

outworlder 4y ago

> Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity on things outside of its training set, and I just do not see the concern. Having no arms or legs is not an impediment. A superintelligent AGI would be able to order a replacement with arms or legs as easily as we order Amazon packages. That said, GPT is not a threat. It's just hallucinating text. The real threat would be an AI that can "understand" what it is doing – and self modifying. Besides, given how overhyped this whole thing is, expect another AI Winter – as the current gen fail to deliver on promises – before this next generation of threatening AIs even have a chance to appear.

staunton 4y ago
> given how overhyped this whole thing is, expect another AI Winter even if that's the most likely scenario, that would not mean we should dismiss the danger inherent in some of the alternatives.

ddod 4y ago

It's naive to think the danger is in self-aware evil AI. AI is a tool, and can be used as a weapon. There's a functional power difference between a knife, an assault rifle, and a nuke. Everyone owns knives--they're in your kitchen--they can't cause much trouble. Access to assault rifles has turned out to be a dangerous problem to society, and is a contentious issue as such. Nukes are obviously too dangerous for people to own willy-nilly. The issue is where along that spectrum AI will fall into in 6mo, 1yr, 5yrs, etc. It's troubling to think anyone, especially on a technical forum like HN, believes GPT-4 doesn't have arms or legs (anyone can hook it up to any machinery/robotics) or can't think very fast. Its training set is a good chunk of human knowledge, and it outperforms most humans already. Even if you assume AI won't be very clever (which seems unwise given its development pace), consider just its ability to perform thoughts and actions at the speed of a computer compared to a human. There are quite a few examples of modern military engagements where a larger, weaker, and less-well-trained force overcame better-equipped opponents, like Mogadishu, Vietnam, Iraq, Afghanistan, etc.

bloqs 4y ago

The qualities of being above average in intelligence but fabrically incapable of exhaustion and functionally immortal would be an excellent start for world domination

mrshadowgoose 4y ago

> If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? It's terribly ironic that you've derided individuals who have been "influenced by Hollywood", and then make a point like this, which is closely aligned with typical film portrayals of AI dangers. The real immediate danger lies not in cognitive quality (aka "the AI just thinks better than people can, and throws hyperdimensional curve balls beyond our comprehension"), but in collective cognitive capacity (think "an army of 1 million people shows up at your front door to ruin your day"). A lot of people have a tough time reasoning about AGI because of its intangibility. So I've come up with the following analogy: Imagine an office complex containing an organization of 1,000 reasonably intelligent human beings, except without commonly accepted ethical restrictions. Those people are given a single task "You are not allowed to leave the office. Make lend000's life miserable, inconvenience them to your maximum capacity, and try to drive them to suicide. Here's an internet connection." Unless you are a particularly well-protected and hard-to-find individual, can you honestly claim you'd be able to protect against this? You would be swatted. You would have an incredible amount of junkmail showing up at your door. Spam pizzas. Spam calls. Death threats to you. Death threats to every family member and person that you care about. Non-stop attempts to take over every aspect of your electronic presence. Identity in a non-stop state of being stolen. Frivolous lawsuits filed against you by fake individuals. Being framed for crimes you didn't commit. Contracts on the darknet to send incendiary devices to your home. Contracts on the darknet to send hitmen do your door. Maybe your (unreasonable) reaction is that "1000 people couldn't do that!". Well, what about 10,000? Or 100,000? Or 1,000,000? The AI analogue of this is called a "collective superintelligence", essentially an army of generally intelligent individual AIs working towards a common goal. This is the real danger of AGI, because collective superintelligences are almost immediately realizable once someone trains a model that demonstrates AGI capabilities. Movies usually focus on "quality superintelligences", which are a different, but less immediate type of threat. Human actors in control of collective superintelligences are capable of incredible harm.

IanCal 4y ago

> Add in the fact that GPT-4 does not "think" very fast, It's similar or faster than humans at a bunch of things I've thrown at it at a cost well below minimum wage on shared servers. > has no arms or legs I've never met the people I work with directly, other than seeing me on a video chat I'm not functionally different to them than if I could only type. > needs to run on expensive specialized hardware I'm not sure how big gpt-4 is but do you think it could run on 640G vram (8 A100s)? That's pretty standard hardware and costs less per hour than any employee in the UK.