6 ms·

How do we prevent humans from wholly relying on/trusting AI in systems that the world's stability relies on. That to me is the biggest 'unacceptable risk'. A h

How do we prevent humans from wholly relying on/trusting AI in systems that the world's stability relies on. That to me is the biggest 'unacceptable risk'. A human putting a system we don't fully understand in charge of a critical process. Article says all high risk systems will be 'assessed' before being put to market. There's a limit to how well you can understand the side effects before deployment.

RandomLensman 3y ago
What critical processes are you thinking about here?
Jensson 3y ago
> How do we prevent humans from wholly relying on/trusting AI in systems that the world's stability relies on. By being transparent and disclosing how the AI was trained and what content was generated by AI. That removes the magic, once people understand how simple these things are and how easy it is to tune the results to make the AI say whatever you want people will stop trusting them quickly. And that is part of what these regulations are about. Transparency instead of marketing buzzwords.
- nathanaldensr 3y ago
  I like to be very blunt with people who don't know anything about this stuff by just saying "AI doesn't exist." There is nothing intelligent about "AI," and people need to know that.
  - littlestymaar 3y ago
    Ever since ChatGPT has become mainstream, I've talked about it to my non-tech friends as “simulated intelligence” instead of “artificial ingelligence” to emphasize that it is not intelligence, it is just simulating intelligence, and sometimes the simulation is realistic and sometimes it's far off.
    - ImHereToVote 3y ago
      This seems like a very slippery concept. You could say the same thing about many human beings. That they simply regurgitate opinions to fit into a certain role and aren't actually utilizing intelligence when making decisions. It's almost as if there is some sort of Platonic ideal of problem solving that we call intelligence, something you can strive for but also someone can poke holes in. Why not simply use an IQ test? The various multimodal systems coming down the pipe will either improve on taking IQ tests, or stall. If they improve, then they gain more intelligence. If they stall, then they aren't truly intelligent. Just glorified goal to action mappers.
      - allen_berg 3y ago
        What about unintelligent humans is not an argument against stating that there is not intelligence in "artificial intelligence".
      - littlestymaar 3y ago
        > You could say the same thing about many human beings. Humans are, by the mere definition of the word, “intelligent”. AI are doing a simulation of that intelligence, the same way a weather model is a simulation of the weather whereas the actual weather isn't a simulation of anything. That doesn't make weather models useless, far from that, but if you trust them blindly when hiking in the mountains, you may end up in a bad place. > Why not simply use an IQ test? IQ tests are very limited measurements of what intelligence actually is. Using a thermometer isn't a good way to access the trustfulness of a weather model.
        logicchains 3y ago
        >Humans are, by the mere definition of the word, “intelligent”. Humans are absolutely not intelligent by definition, that's why words like "unintelligent", "dumb" and "stupid" exist.
        littlestymaar 3y ago
  - ben_w 3y ago
    I on the other hand like to retort with two quotes: "The question of whether a computer can think is no more interesting than whether a submarine can swim." - Edsger Dijkstra "Deep Blue was intelligent the way your programmable alarm clock is intelligent. Not that losing to a $10 million alarm clock made me feel any better." ― Garry Kasparov I don't care if a statistical model feels anything when analysing pictures of weird lumps to diagnose cancer, I care if it's accurate. I don't care if a statistical model has hopes and dreams when generating the text of a computer program one textual token at a time, I care if this is risking my future employment. I don't care if a statistical model has created the same abstraction of the economy that I have when giving me financial advice, I care that it's got a high rate of return at low risk. I don't care if a statistical model wants to win at chess, or at Go, or at Diplomacy, or at poker, or at paperclip maximisation… I will care if someone figures out how to do brain uploads, but that's a question for the future.
- __loam 3y ago
  They're not going to disclose the data because it would be self incriminating.
tensor 3y ago
As long as it's properly tested in terms of out-of-training testing data with sufficient statistics, I'd welcome AI decision making over human any day. When you have humans in charge you know for sure that you have a substantially biased decision maker with completely unknown error levels.
- verisimi 3y ago
  You prefer your decision making authority to be corporate one?
  - tensor 3y ago
    Critical decision making systems are present across industry and also government today. I'm not interested in discussing the balance of industry vs government control of our lives. Suffice to say, if a decision is being made that affects me I'd like it to be as accurate and bias free as possible, regardless of the organization making it.
    - verisimi 3y ago
      Think about what you are saying here: > I'm not interested in discussing the balance of industry vs government control of our lives. With you, some level of corporate or government control is inevitable - it is assumed in your answer. This control is (immorally) inflicted on everyone. I want no extra governmental or corporate strictures. Where's my option? There is none. That's democracy! With laws served up to you by unrelated bureaucrats.
      - verisimi 3y ago
        * Unelected, not unrelated!
  - CamperBob2 3y ago
    What choice is there?
andrewmutz 3y ago
Today we trust humans to manage critical processes. In the future we will need to trust humans to not run those processes with unreliable tools. This isn't just about AI. How do we trust the humans who manage critical processes to follow good security practices (keeping systems patched, etc)?
grogenaut 3y ago
Do you trust Boeing management or safety right now? Memes about MBAs aside, they're not using AI yet still to most people making bad decisions. Eg this problem already exists. It could be exacerbated by people willing to let ai take the wheel

RandomLensman 3y ago

What critical processes are you thinking about here?

Jensson 3y ago

> How do we prevent humans from wholly relying on/trusting AI in systems that the world's stability relies on. By being transparent and disclosing how the AI was trained and what content was generated by AI. That removes the magic, once people understand how simple these things are and how easy it is to tune the results to make the AI say whatever you want people will stop trusting them quickly. And that is part of what these regulations are about. Transparency instead of marketing buzzwords.

nathanaldensr 3y ago
I like to be very blunt with people who don't know anything about this stuff by just saying "AI doesn't exist." There is nothing intelligent about "AI," and people need to know that.
- littlestymaar 3y ago
  Ever since ChatGPT has become mainstream, I've talked about it to my non-tech friends as “simulated intelligence” instead of “artificial ingelligence” to emphasize that it is not intelligence, it is just simulating intelligence, and sometimes the simulation is realistic and sometimes it's far off.
  - ImHereToVote 3y ago
    This seems like a very slippery concept. You could say the same thing about many human beings. That they simply regurgitate opinions to fit into a certain role and aren't actually utilizing intelligence when making decisions. It's almost as if there is some sort of Platonic ideal of problem solving that we call intelligence, something you can strive for but also someone can poke holes in. Why not simply use an IQ test? The various multimodal systems coming down the pipe will either improve on taking IQ tests, or stall. If they improve, then they gain more intelligence. If they stall, then they aren't truly intelligent. Just glorified goal to action mappers.
    - allen_berg 3y ago
      What about unintelligent humans is not an argument against stating that there is not intelligence in "artificial intelligence".
    - littlestymaar 3y ago
      > You could say the same thing about many human beings. Humans are, by the mere definition of the word, “intelligent”. AI are doing a simulation of that intelligence, the same way a weather model is a simulation of the weather whereas the actual weather isn't a simulation of anything. That doesn't make weather models useless, far from that, but if you trust them blindly when hiking in the mountains, you may end up in a bad place. > Why not simply use an IQ test? IQ tests are very limited measurements of what intelligence actually is. Using a thermometer isn't a good way to access the trustfulness of a weather model.
      - logicchains 3y ago
        >Humans are, by the mere definition of the word, “intelligent”. Humans are absolutely not intelligent by definition, that's why words like "unintelligent", "dumb" and "stupid" exist.
        littlestymaar 3y ago
- ben_w 3y ago
  I on the other hand like to retort with two quotes: "The question of whether a computer can think is no more interesting than whether a submarine can swim." - Edsger Dijkstra "Deep Blue was intelligent the way your programmable alarm clock is intelligent. Not that losing to a $10 million alarm clock made me feel any better." ― Garry Kasparov I don't care if a statistical model feels anything when analysing pictures of weird lumps to diagnose cancer, I care if it's accurate. I don't care if a statistical model has hopes and dreams when generating the text of a computer program one textual token at a time, I care if this is risking my future employment. I don't care if a statistical model has created the same abstraction of the economy that I have when giving me financial advice, I care that it's got a high rate of return at low risk. I don't care if a statistical model wants to win at chess, or at Go, or at Diplomacy, or at poker, or at paperclip maximisation… I will care if someone figures out how to do brain uploads, but that's a question for the future.
__loam 3y ago
They're not going to disclose the data because it would be self incriminating.

tensor 3y ago

As long as it's properly tested in terms of out-of-training testing data with sufficient statistics, I'd welcome AI decision making over human any day. When you have humans in charge you know for sure that you have a substantially biased decision maker with completely unknown error levels.

verisimi 3y ago
You prefer your decision making authority to be corporate one?
- tensor 3y ago
  Critical decision making systems are present across industry and also government today. I'm not interested in discussing the balance of industry vs government control of our lives. Suffice to say, if a decision is being made that affects me I'd like it to be as accurate and bias free as possible, regardless of the organization making it.
  - verisimi 3y ago
    Think about what you are saying here: > I'm not interested in discussing the balance of industry vs government control of our lives. With you, some level of corporate or government control is inevitable - it is assumed in your answer. This control is (immorally) inflicted on everyone. I want no extra governmental or corporate strictures. Where's my option? There is none. That's democracy! With laws served up to you by unrelated bureaucrats.
    - verisimi 3y ago
      * Unelected, not unrelated!
- CamperBob2 3y ago
  What choice is there?

andrewmutz 3y ago

Today we trust humans to manage critical processes. In the future we will need to trust humans to not run those processes with unreliable tools. This isn't just about AI. How do we trust the humans who manage critical processes to follow good security practices (keeping systems patched, etc)?

grogenaut 3y ago

Do you trust Boeing management or safety right now? Memes about MBAs aside, they're not using AI yet still to most people making bad decisions. Eg this problem already exists. It could be exacerbated by people willing to let ai take the wheel