6 ms·

This is exactly the direction we are headed. "they can surveil an entire city in real-time with this and track where everybody is and who they are meeting with

This is exactly the direction we are headed. "they can surveil an entire city in real-time with this and track where everybody is and who they are meeting with. No form of protest or movement against the government will be possible if it's scaled up"

ActorNightly 3y ago
... except if ordinary citizens have access to offline versions of advanced AI to use. Prompt: Give me plans for an anti drone weapon that I can use without being detected. This is why AI development needs to be accelerated, not put on hold. Companies can and will continue research because that is where the money is. If everyone else is scared of the hypothetical sentient AI boogeyman, then ordinary people will get left in the dark.
- yboris 3y ago
  Are you excited that an AI could, in the future you describe, spit out correct instructions for creating a more-dangerous virus than COVID to anyone who asks?
  - SirMaster 3y ago
    As long as the AI (that anyone can access) can also spit out an equally powerful antiviral.
    - Bootvis 3y ago
      What if something bad is much easier to achieve than it’s countermeasure?
      - jamiek88 3y ago
        And it is. Pretty much always. it easier to destroy than create.
    - rurp 3y ago
      That doesn't work for explosives though. As soon as a bad actor fires off an attack you can't un-blow people up.
    - dmbche 3y ago
      Yeah - that's not how that works I believe. Some problems are harder than others, and the optimal virus it could produce could take orders or magnitude more time/computation.(edit:to produce an effective antiviral) Also, imagine any one of the billionaires buying all the computing power they can to do something nefarious? Or the amount of computing power the US could use to produce targetted bioweapons? How could the public compete? That's without imagining that they could worm(I believe it's been a little bit) most peoples devices and extract some computing power from that.
      - airgapstopgap 3y ago
        That's what you believe but it's not necessarily correct. You assume asymmetry in favor of attacker, but this patently does not apply to e.g. cryptography; the way it's going, we would get more, not less security out of AIs, by automating testing and audits and formal proofs. And, importantly, defense is a common good; best practices could be easily spread, and applied in an economical way with AI, whereas attackers work on their own. Many functions are asymmetrical in favor of defense. Viruses, too, are not magic; the more sophisticated and powerful its mechanism of action, the longer its code has to be, the worse it is at spreading and surviving the elements (consider how fragile HIV is). Viruses are already tremendously optimized by selection, due to very quickly replication and constant pressure of immunity and medicine. You'd think COVID is merely a warning, but mechanistically it's probably very close to the strongest attack feasible with our biology. Not the most virulent by a long shot; but very good at overcoming our generic defenses. Crucially it wasn't created with AI. Without any AI, we know perfectly well how to make super-COVIDs, it's limited by accessibility of hardware for microbiological research, not compute or algorithms. Rapidly designing antivirals, on the other hand, does benefit from AI. You display a powerful negativity bias which is pervasive in such conversations. You completely ignore AI as a force for good and consider it as, essentially, an offensive capabilty, from which it follows that it must be handed over to incumbents (I take issue with this logic, of course). But that's a self-fulfilling prophecy. Any advantage, centralized enough, becomes an instrument of oppression.
        dmbche 3y ago
        Could you describe my strong negative bias? I have worries that come to mind - just like people were worried that the atom bomb would burn the atmosphere - and I think they are fair. I have a hard time understanding your point - not a jab, genuinely- I agree with your last point, where any advantage being centralized becomes an instrument of oppression, and that's mainly where my issue with it lies. I'm not a doomer at all, I'm personally not afraid of AI. I'm just extending the logic of the previous commenter. AI could overcomes a lot of problems, for a lot of people. Talking out of my ass, but say Jeff Bezos wants to start a lab to make super-covid or whatnot, and his hurdle is having access to restricted hardware - how hard is it to get the AI to design the hardware? Regulation of anything becomes basically impossible - and I think that's enough of a worry in itself. (Edit: to clarify, abssence of regulation brings us back to your final point - centralized power leads to oppression. Regulation is supposed to make power less centralized, other than for the common good (yeah yeah I know), so removal or regulation means untethered power for the already powerful.)
    - smaudet 3y ago
      Who says there is an antiviral for every virus? You can't go doing something because you assume there is a solution to the problem you create - that's irresponsible and if you think that you should be denied all access to modern tech/society.
      - SirMaster 3y ago
        And who says there isn’t? If this AI is so much smarter than us, who of us is to say it can’t completely solve carbon-based biology or something like that. That's also why I said "as long as". Writing that to define the criteria for when it would be OK.
      - airgapstopgap 3y ago
        Who says there exists a way out of the regulatory-authoritarian attractor for AI? Who could've known that nuclear energy is a far lesser threat to humanity than climate change from burning fossils? Certainly not the legions of activists and media producers, who installed the image of green mutagenic goo in people's minds. Just because you do not even conceive of some risk or don't take it seriously doesn't mean you get to play the Responsible Adult In The Room by pontificating of risks of things people do.
    - gregw2 3y ago
      If you are scientifically-minded, I think you should consider how the second law of thermodynamics makes problems for your hope/assumption that AI can generate with equiprobability both good and bad outcomes. If you are monotheistically-minded, consider "Satan's ratchet": It's always easier to lie, kill and destroy than to disseminate truth, raise something from the dead, and build. P.S. I just made up this bit about Satan's ratchet but I think it has a nice ring to it.
      - brigandish 3y ago
        And yet we continue to accumulate knowledge at a rate greater than when censorship is greater. The idea that lies fundamentally outstrip truth is like only telling the first half of the story of the rabbit and the tortoise.
  - ActorNightly 3y ago
    People seem to fundamentally misunderstand the problem space of AI. I assume that you are implying that AI will be able to "figure out" how to synthesize a virus, because something like GPT4 sure as shit not going to be trained on materials on how to specifically synthesize viral weapons. That "figure out" part is where you make a whole shitload of assumptions, one of which is that P=NP.
- jehb 3y ago
  I don't see how this scenario justifies AI development being accelerated - could you put some more color on that? Regulated, sure. Open sourced, probably. But accelerated? I'm not sure accelerating an arms race is going to help those who are least currently able to take advantage of arms.
  - TylerLives 3y ago
    "Regulated" means that you can't use it but they (the government and companies working for it) can. Whether that's a good or bad thing is debatable, but that's what it means.
  - ActorNightly 3y ago
    Step back into the programming world of 1900s where hacking was way easier and more prevalent then today. If you were to change the course of history, would you rather have more or less people and open source projects working on things like https, memory safe languages like Rust, 2 factor authentiaction, e.t.c?
- sangnoir 3y ago
  > Prompt: Give me plans for an anti drone weapon that I can use without being detected. AI is not magic: such a weapon may not exist and your question would be the equivalent of asking for a unicorn.
contingencies 3y ago
they can surveil an entire city in real-time with this and track where everybody is and who they are meeting with. This has been possible for intelligence agencies since the advent of the mobile phone. By cross-referencing with financial and transit records you pretty much have a strong sense of who was proximal to who 24×7×365. For targeted surveillance, add mobile 0days readily on sale to state actors, the rapid global consolidation of cloud email/messaging/SNS providers, mix in increasing live public surveillance (transport authorities, police, etc.), vulnerabilities in network camera firmware, parking records, vehicle phone-home, bank records, post-hoc access to private surveillance footage... we already live in the golden age of state surveillance. What is more concerning is that legal protections are being eroded, new holes opened up for "national security" and "think of the children", surveillance warrants are going through the roof, and critical media is being curtailed. In Australia we've even seen the federal police raid the national broadcaster and the courts uphold it... worrying times. What can the individual do except place their entire trust in their one government? Intelligence apparatus outlive and shape political terms... especially international intelligence sharing agreements...

ActorNightly 3y ago

... except if ordinary citizens have access to offline versions of advanced AI to use. Prompt: Give me plans for an anti drone weapon that I can use without being detected. This is why AI development needs to be accelerated, not put on hold. Companies can and will continue research because that is where the money is. If everyone else is scared of the hypothetical sentient AI boogeyman, then ordinary people will get left in the dark.

yboris 3y ago
Are you excited that an AI could, in the future you describe, spit out correct instructions for creating a more-dangerous virus than COVID to anyone who asks?
- SirMaster 3y ago
  As long as the AI (that anyone can access) can also spit out an equally powerful antiviral.
  - Bootvis 3y ago
    What if something bad is much easier to achieve than it’s countermeasure?
    - jamiek88 3y ago
      And it is. Pretty much always. it easier to destroy than create.
  - rurp 3y ago
    That doesn't work for explosives though. As soon as a bad actor fires off an attack you can't un-blow people up.
  - dmbche 3y ago
    Yeah - that's not how that works I believe. Some problems are harder than others, and the optimal virus it could produce could take orders or magnitude more time/computation.(edit:to produce an effective antiviral) Also, imagine any one of the billionaires buying all the computing power they can to do something nefarious? Or the amount of computing power the US could use to produce targetted bioweapons? How could the public compete? That's without imagining that they could worm(I believe it's been a little bit) most peoples devices and extract some computing power from that.
    - airgapstopgap 3y ago
      That's what you believe but it's not necessarily correct. You assume asymmetry in favor of attacker, but this patently does not apply to e.g. cryptography; the way it's going, we would get more, not less security out of AIs, by automating testing and audits and formal proofs. And, importantly, defense is a common good; best practices could be easily spread, and applied in an economical way with AI, whereas attackers work on their own. Many functions are asymmetrical in favor of defense. Viruses, too, are not magic; the more sophisticated and powerful its mechanism of action, the longer its code has to be, the worse it is at spreading and surviving the elements (consider how fragile HIV is). Viruses are already tremendously optimized by selection, due to very quickly replication and constant pressure of immunity and medicine. You'd think COVID is merely a warning, but mechanistically it's probably very close to the strongest attack feasible with our biology. Not the most virulent by a long shot; but very good at overcoming our generic defenses. Crucially it wasn't created with AI. Without any AI, we know perfectly well how to make super-COVIDs, it's limited by accessibility of hardware for microbiological research, not compute or algorithms. Rapidly designing antivirals, on the other hand, does benefit from AI. You display a powerful negativity bias which is pervasive in such conversations. You completely ignore AI as a force for good and consider it as, essentially, an offensive capabilty, from which it follows that it must be handed over to incumbents (I take issue with this logic, of course). But that's a self-fulfilling prophecy. Any advantage, centralized enough, becomes an instrument of oppression.
      - dmbche 3y ago
        Could you describe my strong negative bias? I have worries that come to mind - just like people were worried that the atom bomb would burn the atmosphere - and I think they are fair. I have a hard time understanding your point - not a jab, genuinely- I agree with your last point, where any advantage being centralized becomes an instrument of oppression, and that's mainly where my issue with it lies. I'm not a doomer at all, I'm personally not afraid of AI. I'm just extending the logic of the previous commenter. AI could overcomes a lot of problems, for a lot of people. Talking out of my ass, but say Jeff Bezos wants to start a lab to make super-covid or whatnot, and his hurdle is having access to restricted hardware - how hard is it to get the AI to design the hardware? Regulation of anything becomes basically impossible - and I think that's enough of a worry in itself. (Edit: to clarify, abssence of regulation brings us back to your final point - centralized power leads to oppression. Regulation is supposed to make power less centralized, other than for the common good (yeah yeah I know), so removal or regulation means untethered power for the already powerful.)
  - smaudet 3y ago
    Who says there is an antiviral for every virus? You can't go doing something because you assume there is a solution to the problem you create - that's irresponsible and if you think that you should be denied all access to modern tech/society.
    - SirMaster 3y ago
      And who says there isn’t? If this AI is so much smarter than us, who of us is to say it can’t completely solve carbon-based biology or something like that. That's also why I said "as long as". Writing that to define the criteria for when it would be OK.
    - airgapstopgap 3y ago
      Who says there exists a way out of the regulatory-authoritarian attractor for AI? Who could've known that nuclear energy is a far lesser threat to humanity than climate change from burning fossils? Certainly not the legions of activists and media producers, who installed the image of green mutagenic goo in people's minds. Just because you do not even conceive of some risk or don't take it seriously doesn't mean you get to play the Responsible Adult In The Room by pontificating of risks of things people do.
  - gregw2 3y ago
    If you are scientifically-minded, I think you should consider how the second law of thermodynamics makes problems for your hope/assumption that AI can generate with equiprobability both good and bad outcomes. If you are monotheistically-minded, consider "Satan's ratchet": It's always easier to lie, kill and destroy than to disseminate truth, raise something from the dead, and build. P.S. I just made up this bit about Satan's ratchet but I think it has a nice ring to it.
    - brigandish 3y ago
      And yet we continue to accumulate knowledge at a rate greater than when censorship is greater. The idea that lies fundamentally outstrip truth is like only telling the first half of the story of the rabbit and the tortoise.
- ActorNightly 3y ago
  People seem to fundamentally misunderstand the problem space of AI. I assume that you are implying that AI will be able to "figure out" how to synthesize a virus, because something like GPT4 sure as shit not going to be trained on materials on how to specifically synthesize viral weapons. That "figure out" part is where you make a whole shitload of assumptions, one of which is that P=NP.
jehb 3y ago
I don't see how this scenario justifies AI development being accelerated - could you put some more color on that? Regulated, sure. Open sourced, probably. But accelerated? I'm not sure accelerating an arms race is going to help those who are least currently able to take advantage of arms.
- TylerLives 3y ago
  "Regulated" means that you can't use it but they (the government and companies working for it) can. Whether that's a good or bad thing is debatable, but that's what it means.
- ActorNightly 3y ago
  Step back into the programming world of 1900s where hacking was way easier and more prevalent then today. If you were to change the course of history, would you rather have more or less people and open source projects working on things like https, memory safe languages like Rust, 2 factor authentiaction, e.t.c?
sangnoir 3y ago
> Prompt: Give me plans for an anti drone weapon that I can use without being detected. AI is not magic: such a weapon may not exist and your question would be the equivalent of asking for a unicorn.

contingencies 3y ago

they can surveil an entire city in real-time with this and track where everybody is and who they are meeting with. This has been possible for intelligence agencies since the advent of the mobile phone. By cross-referencing with financial and transit records you pretty much have a strong sense of who was proximal to who 24×7×365. For targeted surveillance, add mobile 0days readily on sale to state actors, the rapid global consolidation of cloud email/messaging/SNS providers, mix in increasing live public surveillance (transport authorities, police, etc.), vulnerabilities in network camera firmware, parking records, vehicle phone-home, bank records, post-hoc access to private surveillance footage... we already live in the golden age of state surveillance. What is more concerning is that legal protections are being eroded, new holes opened up for "national security" and "think of the children", surveillance warrants are going through the roof, and critical media is being curtailed. In Australia we've even seen the federal police raid the national broadcaster and the courts uphold it... worrying times. What can the individual do except place their entire trust in their one government? Intelligence apparatus outlive and shape political terms... especially international intelligence sharing agreements...