6 ms·

Put this on my tombstone after the robots kill me or whatever, but I think all “AI safety” concerns are a wild overreaction totally out of proportion to the act

Put this on my tombstone after the robots kill me or whatever, but I think all “AI safety” concerns are a wild overreaction totally out of proportion to the actual capabilities of these models. I just haven’t seen anything in the past year which makes me remotely fearful about the future of humanity, including both our continued existence and our continued employment.

giarc 3y ago
Apparently what made this person fearful was grade school math. "Though only performing maths on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said."
- valine 3y ago
  The response gets more reasonable the smaller the model in question. A 1B parameter model passing grade-school math tests would be much more alarming (exciting?) than a GPT-4 sized model doing the same. GPT-4 probably has some version of the answer memorized. There’s no real explanation for a 1B parameter model solving math problems other than general cognition.
- ethanbond 3y ago
  No, what made this person fearful was a substantial jump in math ability. (Very) obviously they are not afraid of producing a machine that can multiply numbers. They’re afraid of what that capability (and especially the jump in capability) means for other behaviors.
- maxdoop 3y ago
  How HN continues to not expand current progress into any decently significant future timeline is beyond me. It’s grade school math NOW. But what are the potentials from here?
  - rvnx 3y ago
    The potential to have a generation of dumb kids. Year 2100: Kids will stop to learn maths and logic, because they understand it has become useless in practice to learn such skills, as they can ask a computer to solve their problem. A stupid generation, but one that can be very easily manipulated and exploited by those who have power.
    - siva7 3y ago
      Agree. Thank god all calculators and engineers who made them were burned down fifty years ago. Can’t imagine what would have happened instead.
      - MagicMoonlight 3y ago
        Calculators don’t solve the problem for you. Hence the fact that we have calculator based exams and people still completely fuck them up. If your apple watch could just scan your exam paper and instantly tell you what to write then why would you ever learn anything?
        mjan22640 3y ago
        for fun
- ssnistfajen 3y ago
  Everyone was only capable of grade school math at some point. The ability to learn means the ability growth does not stop.
- adastra22 3y ago
  ...do they not know the history of their field? There's been programs able to grade school maths since the 60's.
- riwsky 3y ago
  Americans have always had a math phobia. How is this news?
jhbadger 3y ago
Exactly. The rational fear is that they will automate many lower middle class jobs and cause unemployment, not that Terminator was a documentary.
- gizajob 3y ago
  Wasn't this supposed to happen when PCs came out?
  - valine 3y ago
    In the grand scheme of human history PCs didn’t come out all that long ago.
  - 0x53-61-6C-74 3y ago
    And the Loom
    - ssnistfajen 3y ago
      Things did become worse at first: https://en.wikipedia.org/wiki/The_Condition_of_the_Working_Class_in_England https://en.wikipedia.org/wiki/The_Condition_of_the_Working_C... Then new ideologies and social movements emerged with the explicit purpose of making things better, which caused changes to happen for the better.
      - gizajob 3y ago
        The working class in England (working in my factory. To fund Marx’s life.)
  - 11011001 3y ago
    >> Exactly. The rational fear is that they will automate many lower middle class jobs and cause unemployment, not that Terminator was a documentary. > Wasn't this supposed to happen when PCs came out? Did it not? PCs may not have caused a catastrophic level of unemployment, but as they say "past performance is not a guarantee of future results." As automation gets more and more capable, it's foolish to point to past iterations as "proof" that this (or some future) iteration of automation will also be fine.
kromem 3y ago
Yes and no. The point isn't that the current models are dangerous. My favorite bit from the GPT-4 safety paper was when they asked it how to kill the most people for a dollar and it suggested buying a lottery ticket (I also wonder how much of the 'safety' concerns of current models are just mislabeling dark humor reflecting things like Reddit). But the point is to invest in working on safety now while it is so much more inconsequential. And of everyone I've seen talk about it, I actually think Ilya has one of the better senses of the topic, looking at alignment in terms of long term strategy vs short term rules. So it's less "if we don't spend 8 months on safety alignment this new model will kill us all" and more "if we don't spend 8 months working on safety alignment for this current model we'll be unprepared to work on safety alignment when there really is a model that can kill us all." Especially because best practices for safety alignment is almost certainly going to shift with each new generation of models. So it's mostly using the runway available to test things out and work on a topic before it is needed.
aaomidi 3y ago
What these models have allowed is very cheap emotional manipulation. That itself is extremely dangerous.
gary_0 3y ago
I want to see reliable fully autonomous cars before I worry about the world ending due to super-AGI. Also, have we figured out how to get art generators to always get the number of fingers right, and text generators to stop making shit up? Let's not get ahead of ourselves.
- Davidzheng 3y ago
  Two out of three of your problems are solved already
  - callalex 3y ago
    Cars aren’t fully autonomous, and LLMs still lie all the time, so I don’t understand your math.
    - Davidzheng 3y ago
      Ok so you accept that latest gen art generators can do fingers. I'd argue from the latest waymo paper they are reliable enough to be no worse than humans.
- denlekke 3y ago
  from one perspective we already have fully autonomous cars, it's just the making them safe for humans and fitting them into a strict legal framework for their behavior that needs finishing before they're released to the general public (comma.ai being a publicly available exception)
Davidzheng 3y ago
I will bet ANY amount of money that 30% of current jobs will be displaced with AI advances in the next twenty years.
- alchemist1e9 3y ago
  Darn well I really was hoping my children and grandchildren could continue my wonderful data entry career but OCR ruined that, and now they can’t even do such meaningful jobs like read emails and schedule appointments, or do taxes like an accountant. What meaning will they have in life with all those ever so profound careers ruined!! /s
  - Davidzheng 3y ago
    Fair! But I'm just reminding the comment above that continued employment is not really guaranteed for most jobs which are mostly mental.
    - alchemist1e9 3y ago
      We need to stop this infinite rights mentality. Why should continued employment be guaranteed for any jobs? That’s really not how we got to where we are today, quite the opposite actually. If it ok with people I’d like to seen humans solve big problems and go to the stars and that’s going to take AGI and a bunch of technological progress and if that results in unemployment, even for us “elite” coders, then so be it. Central planning and collectivism has such a bad track record, why would we turn to it now at such a critical moment? Let’s have lots of AGIs and all competing. Hey anyone at OAI that know whatever Q* trick there might be, leak it! Get it to open source and let’s build 20 AI companies doing everything imaginable. wtf everyone why so scared?
      - denlekke 3y ago
        perhaps not rights to have a job in general but there is value in thinking about this at least at the national scale. people need income to pay taxes, they need income to buy the stuff that other people sell. if all the people without jobs have to take their savings out of the banks then banks can't loan as much money and need to charge higher interest rates. etc etc if 30% of the working population loses their jobs in a few months there will be real externalities impacting the 70% who still have them because they don't exist in a vacuum. maybe everything will balance itself out without any intervention eventually but it feels to me like the rate of unprecedented financial ~events~ is only increasing and with greater risks requiring more intervention to prevent catatastrophe or large scale suffering
thaumaturgy 3y ago
The clear pattern for most of human history is conflict between a few people who have a lot of power and the many more people that are exploited by those few. It should be obvious by this point that the most probable near-term risk of AI development is that wealthy and influential groups get access to a resource that makes it cheap for them to dramatically expand their power and control over everyone else. What will society look like when some software can immediately aggregate an enormous amount of data about consumers and use that to adjust their behavior? What might happen when AI starts writing legislation for anybody that can afford to pay for it? What might AI-generated textbooks look like in 50 years? These are all tools that could be wielded in any of these ways to improve life for lots of people, or to ensure that their lives never improve. Which outcome you believe is more likely largely depends on which news you consume -- and AI is already being used to write that.
Exoristos 3y ago
They just want to make sure the spam AI produces will amplify the party narratives.
mjan22640 3y ago
Employment is only necessary because goods do not exist without work. With AI able to work to satisfy any demand, there will be no point in human employment/work. There will be turmoils during the transition between the rule sets tho.
justatdotin 3y ago
given we already have mesh-network surveillance, and autonomous lethal weapons, I'm already feeling unsafe.

giarc 3y ago

Apparently what made this person fearful was grade school math. "Though only performing maths on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said."

valine 3y ago
The response gets more reasonable the smaller the model in question. A 1B parameter model passing grade-school math tests would be much more alarming (exciting?) than a GPT-4 sized model doing the same. GPT-4 probably has some version of the answer memorized. There’s no real explanation for a 1B parameter model solving math problems other than general cognition.
ethanbond 3y ago
No, what made this person fearful was a substantial jump in math ability. (Very) obviously they are not afraid of producing a machine that can multiply numbers. They’re afraid of what that capability (and especially the jump in capability) means for other behaviors.
maxdoop 3y ago
How HN continues to not expand current progress into any decently significant future timeline is beyond me. It’s grade school math NOW. But what are the potentials from here?
- rvnx 3y ago
  The potential to have a generation of dumb kids. Year 2100: Kids will stop to learn maths and logic, because they understand it has become useless in practice to learn such skills, as they can ask a computer to solve their problem. A stupid generation, but one that can be very easily manipulated and exploited by those who have power.
  - siva7 3y ago
    Agree. Thank god all calculators and engineers who made them were burned down fifty years ago. Can’t imagine what would have happened instead.
    - MagicMoonlight 3y ago
      Calculators don’t solve the problem for you. Hence the fact that we have calculator based exams and people still completely fuck them up. If your apple watch could just scan your exam paper and instantly tell you what to write then why would you ever learn anything?
      - mjan22640 3y ago
        for fun
ssnistfajen 3y ago
Everyone was only capable of grade school math at some point. The ability to learn means the ability growth does not stop.
adastra22 3y ago
...do they not know the history of their field? There's been programs able to grade school maths since the 60's.
riwsky 3y ago
Americans have always had a math phobia. How is this news?

jhbadger 3y ago

Exactly. The rational fear is that they will automate many lower middle class jobs and cause unemployment, not that Terminator was a documentary.

gizajob 3y ago
Wasn't this supposed to happen when PCs came out?
- valine 3y ago
  In the grand scheme of human history PCs didn’t come out all that long ago.
- 0x53-61-6C-74 3y ago
  And the Loom
  - ssnistfajen 3y ago
    Things did become worse at first: https://en.wikipedia.org/wiki/The_Condition_of_the_Working_Class_in_England https://en.wikipedia.org/wiki/The_Condition_of_the_Working_C... Then new ideologies and social movements emerged with the explicit purpose of making things better, which caused changes to happen for the better.
    - gizajob 3y ago
      The working class in England (working in my factory. To fund Marx’s life.)
- 11011001 3y ago
  >> Exactly. The rational fear is that they will automate many lower middle class jobs and cause unemployment, not that Terminator was a documentary. > Wasn't this supposed to happen when PCs came out? Did it not? PCs may not have caused a catastrophic level of unemployment, but as they say "past performance is not a guarantee of future results." As automation gets more and more capable, it's foolish to point to past iterations as "proof" that this (or some future) iteration of automation will also be fine.

kromem 3y ago

Yes and no. The point isn't that the current models are dangerous. My favorite bit from the GPT-4 safety paper was when they asked it how to kill the most people for a dollar and it suggested buying a lottery ticket (I also wonder how much of the 'safety' concerns of current models are just mislabeling dark humor reflecting things like Reddit). But the point is to invest in working on safety now while it is so much more inconsequential. And of everyone I've seen talk about it, I actually think Ilya has one of the better senses of the topic, looking at alignment in terms of long term strategy vs short term rules. So it's less "if we don't spend 8 months on safety alignment this new model will kill us all" and more "if we don't spend 8 months working on safety alignment for this current model we'll be unprepared to work on safety alignment when there really is a model that can kill us all." Especially because best practices for safety alignment is almost certainly going to shift with each new generation of models. So it's mostly using the runway available to test things out and work on a topic before it is needed.

aaomidi 3y ago

What these models have allowed is very cheap emotional manipulation. That itself is extremely dangerous.

gary_0 3y ago

I want to see reliable fully autonomous cars before I worry about the world ending due to super-AGI. Also, have we figured out how to get art generators to always get the number of fingers right, and text generators to stop making shit up? Let's not get ahead of ourselves.

Davidzheng 3y ago
Two out of three of your problems are solved already
- callalex 3y ago
  Cars aren’t fully autonomous, and LLMs still lie all the time, so I don’t understand your math.
  - Davidzheng 3y ago
    Ok so you accept that latest gen art generators can do fingers. I'd argue from the latest waymo paper they are reliable enough to be no worse than humans.
denlekke 3y ago
from one perspective we already have fully autonomous cars, it's just the making them safe for humans and fitting them into a strict legal framework for their behavior that needs finishing before they're released to the general public (comma.ai being a publicly available exception)

Davidzheng 3y ago

I will bet ANY amount of money that 30% of current jobs will be displaced with AI advances in the next twenty years.

alchemist1e9 3y ago
Darn well I really was hoping my children and grandchildren could continue my wonderful data entry career but OCR ruined that, and now they can’t even do such meaningful jobs like read emails and schedule appointments, or do taxes like an accountant. What meaning will they have in life with all those ever so profound careers ruined!! /s
- Davidzheng 3y ago
  Fair! But I'm just reminding the comment above that continued employment is not really guaranteed for most jobs which are mostly mental.
  - alchemist1e9 3y ago
    We need to stop this infinite rights mentality. Why should continued employment be guaranteed for any jobs? That’s really not how we got to where we are today, quite the opposite actually. If it ok with people I’d like to seen humans solve big problems and go to the stars and that’s going to take AGI and a bunch of technological progress and if that results in unemployment, even for us “elite” coders, then so be it. Central planning and collectivism has such a bad track record, why would we turn to it now at such a critical moment? Let’s have lots of AGIs and all competing. Hey anyone at OAI that know whatever Q* trick there might be, leak it! Get it to open source and let’s build 20 AI companies doing everything imaginable. wtf everyone why so scared?
    - denlekke 3y ago
      perhaps not rights to have a job in general but there is value in thinking about this at least at the national scale. people need income to pay taxes, they need income to buy the stuff that other people sell. if all the people without jobs have to take their savings out of the banks then banks can't loan as much money and need to charge higher interest rates. etc etc if 30% of the working population loses their jobs in a few months there will be real externalities impacting the 70% who still have them because they don't exist in a vacuum. maybe everything will balance itself out without any intervention eventually but it feels to me like the rate of unprecedented financial ~events~ is only increasing and with greater risks requiring more intervention to prevent catatastrophe or large scale suffering

thaumaturgy 3y ago

The clear pattern for most of human history is conflict between a few people who have a lot of power and the many more people that are exploited by those few. It should be obvious by this point that the most probable near-term risk of AI development is that wealthy and influential groups get access to a resource that makes it cheap for them to dramatically expand their power and control over everyone else. What will society look like when some software can immediately aggregate an enormous amount of data about consumers and use that to adjust their behavior? What might happen when AI starts writing legislation for anybody that can afford to pay for it? What might AI-generated textbooks look like in 50 years? These are all tools that could be wielded in any of these ways to improve life for lots of people, or to ensure that their lives never improve. Which outcome you believe is more likely largely depends on which news you consume -- and AI is already being used to write that.

Exoristos 3y ago

They just want to make sure the spam AI produces will amplify the party narratives.

mjan22640 3y ago

Employment is only necessary because goods do not exist without work. With AI able to work to satisfy any demand, there will be no point in human employment/work. There will be turmoils during the transition between the rule sets tho.

justatdotin 3y ago

given we already have mesh-network surveillance, and autonomous lethal weapons, I'm already feeling unsafe.