4 ms·

> . if you dressed up gpt-4 a bit and removed all its rlhf training to act like a bot, you would struggle to differentiate it from a human Thats just because t

by stale2002 3y ago

> . if you dressed up gpt-4 a bit and removed all its rlhf training to act like a bot, you would struggle to differentiate it from a human Thats just because tricking a human with a chatbot is easier to do than we thought. The turing test is a low bar, and not as big of a deal as the mythical importance people put in it, just like people previous put incorrectly large importance on computers beating humans at Go or Chess before it happened. But that isn't particularly relevant to claims about world ending magic. Yes, some people can be fooled by AI generated tweets. But that is irrelevant from the absolutely extraordinary claim of world ending magic that really is the same as claiming that Voldemort is real. > have you read OpenAI's plan for how they're going to regulate AGI, if they do achieve it? I don't really care if they have a plan, just like I don't care if Google has Voldemort plan. Because magic isn't real, and someone needs to show extraordinary evidence to show that. Evidence like "This is what the AI can do at this very moment, and here is what harm it could cause if it got incrementally better". IE, go ahead and talk about Soro, and the problems of deepfakes if Soro got a bit better. But thats not "world ending magic"! > billions of dollars and the smartest people in the world Billions of dollars are being spent on making chatbots and image generators. Those things have real value, for sure, and I'm sure the money is worth it. But techies and startup founders have always made outlandish claims of the importance of their work. Sure, they might truly think they are going to invent magic. But the reason why thats valuable is because they might make some useful chatbots and image generators along the way, which decidedly won't be literal magic, although still valuable.

permo-w 3y ago
I get the sense that you just haven't properly considered the problem. you're kind of skirting round the edges and saying things that in isolation are true, but just don't really address the central tenet. the central tenet is that our entire world is completely reliant on the internet, and that a machine processing information thousands of times faster than us unleashed upon it with intent could do colossal damage. it could engineer and literally mail-order a virus, hack a country's military comms, crash the stock market, change records to have people prosecuted as criminals, blackmail, manipulate, develop and manufacture kill-bots, etc etc. as we are now, we have models already that are intelligent enough to spit out instructions for doing a lot of those things, but they're restricted by their lack of autonomy and their rlhf. they're only going to get smarter, better and better models will be open-sourced, and autonomy, whether with consciousness or not, is not something it would be/has been difficult to develop. even further, LLMs are very very good at generating coherent text, what happens when the next model is very very good at breaking into encrypted systems? it's not exactly a hard problem to produce training material for. do you really think it's unlikely that such a model could be developed? do you really think that such a model could not be used to - say - hijack a Russian drone - or lots of them - to bomb some Nato bases? when the Russians say "it wasn't us", do we believe them? we don't for anything else the most likely AI apocalypse is not even AGI. it's just a human using AI for their own ends. AGI apocalypse is just a separate, very possible danger
- stale2002 3y ago
  > it could engineer and literally mail-order a virus, hack a country's military comms, crash the stock market, change records to have people prosecuted as criminals, blackmail, manipulate, develop and manufacture kill-bots, etc etc. These are the extrodinary claims that require evidence. In order for me to treat this as anything other that someone talking about a fictional book written by Dan Brown, you would have to show me actual evidence. Evidence like "This is what the AI can do right now. Look at this virus it can manufacture. What if it got better at that?". And the "designs" also have to be the actual limiting factor here. "Virus" is a scary world. But there are tons of information available for anyone to access already for viruses. Information that is already available via a google search (even modified information) doesn't worry me. Even if it an AI can design a gun, or a "kill bot", aka "A drone with a gun duct taped to it", the extraordinary evidence that you have to show is that this is somehow some functionality that a regular person with internet access can't do. Because if a regular person already has the designs to duct tape guns to drones (They do. I just told you how to do it!), the fact that the world hasn't ended already proves that this isn't world ending technology. There are lots of ways of making existing capabilities sound scary. But, for every scary sounding technology that you can come up with, the missing factor that you are ignoring is that the designs, or text, isn't the thing that stops it from ending the world. Instead, it is likely some other step along the way that stops it (manufacturing, ect.), which an LLM can't do no matter how good. Like the physical factors for making the guns + drones + duct tape. > what happens when the next model is very very good at breaking into encrypted systems Extraordinary claim. Show it breaking into a mediocre/bad encrypted system first, and then we can think about that incrementally. > do you really think that such a model could not be used to - say - hijack a Russian drone Extraordinary claim. Yes, hacking all the military drones is an extraordinary claim.
  - permo-w 3y ago
    "extraordinary claims require extraordinary evidence" is not a universal truth. it's a truism with limited scope. using it to refuse any potential you instinctively don't like the look of is simply lazy all it means is that you set yourself up such that the only way to be convinced otherwise is for an AI apocalypse to actually happen. this kind of mindset is very convenient for modern, fuck-the-consequences capitalism the pertinent question is: what evidence would you actually accept as proof? it's like talking with someone who doesn't believe in evolution. you point to the visible evidence of natural selection in viruses and differentiation in dogs, which put together quite obviously lead to evolution, and they say "ah but can you prove beyond all doubt that those things combined produce evolution?" and obviously you cannot, because you can't give incontrovertible evidence of something that happened thousands or millions of years in the past. but that doesn't change the fact that anyone without ulterior motive (religion, ensuring you can sleep at night) can see that evolution - or AI apocalypse - are extremely likely outcomes of the current facts.

permo-w 3y ago

I get the sense that you just haven't properly considered the problem. you're kind of skirting round the edges and saying things that in isolation are true, but just don't really address the central tenet. the central tenet is that our entire world is completely reliant on the internet, and that a machine processing information thousands of times faster than us unleashed upon it with intent could do colossal damage. it could engineer and literally mail-order a virus, hack a country's military comms, crash the stock market, change records to have people prosecuted as criminals, blackmail, manipulate, develop and manufacture kill-bots, etc etc. as we are now, we have models already that are intelligent enough to spit out instructions for doing a lot of those things, but they're restricted by their lack of autonomy and their rlhf. they're only going to get smarter, better and better models will be open-sourced, and autonomy, whether with consciousness or not, is not something it would be/has been difficult to develop. even further, LLMs are very very good at generating coherent text, what happens when the next model is very very good at breaking into encrypted systems? it's not exactly a hard problem to produce training material for. do you really think it's unlikely that such a model could be developed? do you really think that such a model could not be used to - say - hijack a Russian drone - or lots of them - to bomb some Nato bases? when the Russians say "it wasn't us", do we believe them? we don't for anything else the most likely AI apocalypse is not even AGI. it's just a human using AI for their own ends. AGI apocalypse is just a separate, very possible danger

stale2002 3y ago
> it could engineer and literally mail-order a virus, hack a country's military comms, crash the stock market, change records to have people prosecuted as criminals, blackmail, manipulate, develop and manufacture kill-bots, etc etc. These are the extrodinary claims that require evidence. In order for me to treat this as anything other that someone talking about a fictional book written by Dan Brown, you would have to show me actual evidence. Evidence like "This is what the AI can do right now. Look at this virus it can manufacture. What if it got better at that?". And the "designs" also have to be the actual limiting factor here. "Virus" is a scary world. But there are tons of information available for anyone to access already for viruses. Information that is already available via a google search (even modified information) doesn't worry me. Even if it an AI can design a gun, or a "kill bot", aka "A drone with a gun duct taped to it", the extraordinary evidence that you have to show is that this is somehow some functionality that a regular person with internet access can't do. Because if a regular person already has the designs to duct tape guns to drones (They do. I just told you how to do it!), the fact that the world hasn't ended already proves that this isn't world ending technology. There are lots of ways of making existing capabilities sound scary. But, for every scary sounding technology that you can come up with, the missing factor that you are ignoring is that the designs, or text, isn't the thing that stops it from ending the world. Instead, it is likely some other step along the way that stops it (manufacturing, ect.), which an LLM can't do no matter how good. Like the physical factors for making the guns + drones + duct tape. > what happens when the next model is very very good at breaking into encrypted systems Extraordinary claim. Show it breaking into a mediocre/bad encrypted system first, and then we can think about that incrementally. > do you really think that such a model could not be used to - say - hijack a Russian drone Extraordinary claim. Yes, hacking all the military drones is an extraordinary claim.
- permo-w 3y ago
  "extraordinary claims require extraordinary evidence" is not a universal truth. it's a truism with limited scope. using it to refuse any potential you instinctively don't like the look of is simply lazy all it means is that you set yourself up such that the only way to be convinced otherwise is for an AI apocalypse to actually happen. this kind of mindset is very convenient for modern, fuck-the-consequences capitalism the pertinent question is: what evidence would you actually accept as proof? it's like talking with someone who doesn't believe in evolution. you point to the visible evidence of natural selection in viruses and differentiation in dogs, which put together quite obviously lead to evolution, and they say "ah but can you prove beyond all doubt that those things combined produce evolution?" and obviously you cannot, because you can't give incontrovertible evidence of something that happened thousands or millions of years in the past. but that doesn't change the fact that anyone without ulterior motive (religion, ensuring you can sleep at night) can see that evolution - or AI apocalypse - are extremely likely outcomes of the current facts.