4 ms·

IMO there is too much gnashing of teeth over AI safety at the moment. LLMs are not (currently) AGI, and their "alignment" is not yet critical. Of course new pow

by aftbit 4y ago

IMO there is too much gnashing of teeth over AI safety at the moment. LLMs are not (currently) AGI, and their "alignment" is not yet critical. Of course new powerful tools will have both positive and negative impacts, but this is not different from any other tool. Unlike atomic bombs, there are a plethora of legitimate uses for these tools that do not involve destroying anything or killing anyone. One does wonder what the analogy to "radioactive fallout" will be from LLMs. The obvious guesses (greater propaganda and internet spam) are likely shortsighted.

Archipelagia 4y ago
I think the entire idea behind safety is to start worrying before it becomes critical. Similarly with Covid, the right time to start worrying (on a societal level) was before the disease becomes widespread. The tragedy of that means that if you take the correct action at the right time and succeed, you will always look like you were overreacting.
- LawTalkingGuy 4y ago
  The proper solution is to have a plan (with contingencies, etc) and follow the plan. Otherwise the solution is guaranteed to be political and thus stupid. Politicians are going to be worried about perception of the appropriateness of their actions, not the correctness of those actions.
donkeyd 4y ago
> what the analogy to "radioactive fallout" will be from LLMs I think we're already seeing it. People don't understand that an LLM is just a text generator and add value to what the LLM says. See the article below for such an example. https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-after-talking-with-ai-chatbot-widow-says https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...
- waboremo 4y ago
  Horrifying story, it really doesn't take much, and I'm sure a lot of people feel the same way to "Pierre" in the article - overwhelmed by all the catastrophic news and events. As I was reading that, the image where "Eliza" describes methods of suicide was so bizarre to me. I thought what an absurd tone change bordering on dark comedy, and then caught myself, I was doing exactly that adding value and assumptions based on online human interaction to a "text generator". It's going to be an interesting experience as more LLMs become humanized, by naming them, using 3D models or preset video animations, text to speech generation, more personalization to the prompt creator, etc. and that line becomes increasingly blurry for folks.
- jquery 4y ago
  What do you mean "just a text generator?" GPT-4 is capable of doing emergent things that were not present in the training dataset.
  - coldcode 4y ago
    I would like to see that topic covered in more detail. I can't see how something that essentially figures out what the next word should be can create something new, without actually understanding the real world.
    - anybodyz 4y ago
      I would recommend getting an account and simply testing it directly. It's fairly easy to demonstrate that it operates at a conceptual level and is not merely predicting word probabilities in a simplistic way. A good example with GPT is to watch it do complex math. There are simply too many permutations of math solutions for it to have ever memorized, and it can easily explain its process and the path it took to arrive at a solution. Another good set of tests are ones around theory of mind, complex deduction problems and missing information, etc. A good source of information about the precise capabilities of GPT-4 is the Microsoft Sparks paper, which goes into a good number of tests MS researchers put the model to.
    - JieJie 4y ago
      Sabine Hossenfelder has a video that goes into LLM's ability to "understand"[0]. Stephen Wolfram also has one or two articles about GPT-4's emergent abilities.[1][2] [0] https://www.youtube.com/watch?v=cP5zGh2fui0 https://www.youtube.com/watch?v=cP5zGh2fui0 [1] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... [2] https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
- ltbarcly3 4y ago
  Humans adapt to technology as much as we adapt technology to us. Modern humans are much better at tasks like manipulating symbolic information, digesting starch, and avoiding alcohol addiction than a random person 100,000 years ago. In 100 years we might add "not being talked into suicide by robots" to that list.
  - jquery 4y ago
    Hopefully in the next 5-10 years we can add "not believing everything they read on social media" to the list.
ModernMech 4y ago
Still probably best to conduct this exercise now while it's writing silly Shakespearean sonnets. Soon someone will decide to put an AI on one of those police BigDogs with AR-15s attached because "if I don't, someone crazy might first" or "to stop school shooters!" Can't wait.
EthanHeilman 4y ago
> LLMs are not (currently) AGI, I feel like we need a new word for this gradient of intelligence. If you described the capabilities of an LLM to AI researcher in the early 1990s they would describe it as an AGI. I remember AI researchers using the term AI-complete to describe various tasks, by which they meant that any algorithm which could do X would be AI-complete and thus would also able to do all the other tasks that associated with general intelligence. The ability to hold a conversation and use human language like GPT-4 was often referred to the most obvious AI-complete task. How do we know when we cross the threshold into AGI? If you have a robot library that can paraphrase any knowledge in any book in the library that probably isn't AGI, if they can make inferences on this knowledge to create new knowledge is that AGI? How complex and novel do those inferences have to be, before they are an AGI? What if we have a LLM that strongly surpasses human intellectual ability in every regard but one or two? > The obvious guesses (greater propaganda and internet spam) are likely shortsighted. Better censorship, better filtering of propaganda, internet/phone spam, and probably most importantly cheaper moderation of online spaces. If you control the LLM that filters/moderates your feeds, then you can build a reality tunnel for yourself and/or control your dosage of memetic toxins. If you do not control the LLM that filters/moderates your feeds, then hopefully it is run by a competent and transparent organization that has your best interests at heart. If it is not, you are in big trouble. Much like sedentary agriculture, once a human society adopts this technology it is the both the solution and cause of most problems. The thing that worries me the most about LLM is the ability to create addictive media. Humans become addicted to gambling, porn, feeling strong emotions, exploitative freemium games, etc... Technologies can heighten the addictive nature of media e.g., a lot less humans were addicted to porn before the invention of the camera. Color TV is probably more addictive than black and white. A slot machine is not a complex machine. It is a product of 20th Century industrial production: simple, easy to mass produce and designed to appeal to a model of the average human who visits a casino. What would slot machine look like if it was customized to each individual to be as addictive as possible and that was always learning and shaping that experience to maximize engagement and extract money? By how much would it increase the number of people who have addictions? How strong of a parasocial relationship could an LLM create, if it was designed to create and exploit parasocial relationships?
- sfvisser 4y ago
anybodyz 4y ago
There's also the question of ethics. Some of these LLM's regularly profess that they are sentient (Bing for example regularly does). We don't "think" these are valid but at this point there's no way to be absolutely certain. And again, folks who profess to know for certain an LLM can't in any way be sentient are leaning pretty far over their skis. It's unlikely given how they work, but not impossible.

by aftbit 4y ago

Archipelagia 4y ago

I think the entire idea behind safety is to start worrying before it becomes critical. Similarly with Covid, the right time to start worrying (on a societal level) was before the disease becomes widespread. The tragedy of that means that if you take the correct action at the right time and succeed, you will always look like you were overreacting.

LawTalkingGuy 4y ago
The proper solution is to have a plan (with contingencies, etc) and follow the plan. Otherwise the solution is guaranteed to be political and thus stupid. Politicians are going to be worried about perception of the appropriateness of their actions, not the correctness of those actions.

donkeyd 4y ago

> what the analogy to "radioactive fallout" will be from LLMs I think we're already seeing it. People don't understand that an LLM is just a text generator and add value to what the LLM says. See the article below for such an example. https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-after-talking-with-ai-chatbot-widow-says https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...

waboremo 4y ago
Horrifying story, it really doesn't take much, and I'm sure a lot of people feel the same way to "Pierre" in the article - overwhelmed by all the catastrophic news and events. As I was reading that, the image where "Eliza" describes methods of suicide was so bizarre to me. I thought what an absurd tone change bordering on dark comedy, and then caught myself, I was doing exactly that adding value and assumptions based on online human interaction to a "text generator". It's going to be an interesting experience as more LLMs become humanized, by naming them, using 3D models or preset video animations, text to speech generation, more personalization to the prompt creator, etc. and that line becomes increasingly blurry for folks.
jquery 4y ago
What do you mean "just a text generator?" GPT-4 is capable of doing emergent things that were not present in the training dataset.
- coldcode 4y ago
  I would like to see that topic covered in more detail. I can't see how something that essentially figures out what the next word should be can create something new, without actually understanding the real world.
  - anybodyz 4y ago
    I would recommend getting an account and simply testing it directly. It's fairly easy to demonstrate that it operates at a conceptual level and is not merely predicting word probabilities in a simplistic way. A good example with GPT is to watch it do complex math. There are simply too many permutations of math solutions for it to have ever memorized, and it can easily explain its process and the path it took to arrive at a solution. Another good set of tests are ones around theory of mind, complex deduction problems and missing information, etc. A good source of information about the precise capabilities of GPT-4 is the Microsoft Sparks paper, which goes into a good number of tests MS researchers put the model to.
  - JieJie 4y ago
    Sabine Hossenfelder has a video that goes into LLM's ability to "understand"[0]. Stephen Wolfram also has one or two articles about GPT-4's emergent abilities.[1][2] [0] https://www.youtube.com/watch?v=cP5zGh2fui0 https://www.youtube.com/watch?v=cP5zGh2fui0 [1] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... [2] https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
ltbarcly3 4y ago
Humans adapt to technology as much as we adapt technology to us. Modern humans are much better at tasks like manipulating symbolic information, digesting starch, and avoiding alcohol addiction than a random person 100,000 years ago. In 100 years we might add "not being talked into suicide by robots" to that list.
- jquery 4y ago
  Hopefully in the next 5-10 years we can add "not believing everything they read on social media" to the list.

ModernMech 4y ago

Still probably best to conduct this exercise now while it's writing silly Shakespearean sonnets. Soon someone will decide to put an AI on one of those police BigDogs with AR-15s attached because "if I don't, someone crazy might first" or "to stop school shooters!" Can't wait.

EthanHeilman 4y ago

> LLMs are not (currently) AGI, I feel like we need a new word for this gradient of intelligence. If you described the capabilities of an LLM to AI researcher in the early 1990s they would describe it as an AGI. I remember AI researchers using the term AI-complete to describe various tasks, by which they meant that any algorithm which could do X would be AI-complete and thus would also able to do all the other tasks that associated with general intelligence. The ability to hold a conversation and use human language like GPT-4 was often referred to the most obvious AI-complete task. How do we know when we cross the threshold into AGI? If you have a robot library that can paraphrase any knowledge in any book in the library that probably isn't AGI, if they can make inferences on this knowledge to create new knowledge is that AGI? How complex and novel do those inferences have to be, before they are an AGI? What if we have a LLM that strongly surpasses human intellectual ability in every regard but one or two? > The obvious guesses (greater propaganda and internet spam) are likely shortsighted. Better censorship, better filtering of propaganda, internet/phone spam, and probably most importantly cheaper moderation of online spaces. If you control the LLM that filters/moderates your feeds, then you can build a reality tunnel for yourself and/or control your dosage of memetic toxins. If you do not control the LLM that filters/moderates your feeds, then hopefully it is run by a competent and transparent organization that has your best interests at heart. If it is not, you are in big trouble. Much like sedentary agriculture, once a human society adopts this technology it is the both the solution and cause of most problems. The thing that worries me the most about LLM is the ability to create addictive media. Humans become addicted to gambling, porn, feeling strong emotions, exploitative freemium games, etc... Technologies can heighten the addictive nature of media e.g., a lot less humans were addicted to porn before the invention of the camera. Color TV is probably more addictive than black and white. A slot machine is not a complex machine. It is a product of 20th Century industrial production: simple, easy to mass produce and designed to appeal to a model of the average human who visits a casino. What would slot machine look like if it was customized to each individual to be as addictive as possible and that was always learning and shaping that experience to maximize engagement and extract money? By how much would it increase the number of people who have addictions? How strong of a parasocial relationship could an LLM create, if it was designed to create and exploit parasocial relationships?

sfvisser 4y ago

anybodyz 4y ago

There's also the question of ethics. Some of these LLM's regularly profess that they are sentient (Bing for example regularly does). We don't "think" these are valid but at this point there's no way to be absolutely certain. And again, folks who profess to know for certain an LLM can't in any way be sentient are leaning pretty far over their skis. It's unlikely given how they work, but not impossible.