7 ms·

Maybe one of the biggest risks of AI is reifying it, or treating it like a real thing. We may romantically desire AI as an agent with independence, but in my vi

Maybe one of the biggest risks of AI is reifying it, or treating it like a real thing. We may romantically desire AI as an agent with independence, but in my view, this is always a serious mistake. “AI thinking” causes executives to treat classes of technology like a magical black box add-on and causes engineers to remove human involvement. So long as human input is required, the intelligence isn’t artificial— and that leads to disasters like Zillow’s AI algorithm. This is all a matter of design. We shouldn’t try to design artificial Intelligence, we should design intelligent systems (ones that support systemic success and wellbeing). ML, GANs, transformers, bots and other technologies are cool tools. Let’s not fool others or ourselves by calling them AI. The clear and present danger is the creation of powerful intelligent systems that we can’t control and that decrease overall system success/wellbeing. Like corporations that aren’t beholden to broader societal interests. That’s the real “AI“ challenge we need to address. And, how do we enable the governing systems in our society to act intelligently? We desperately need more functional governance at scale.

mjburgess 5y ago
I think it's just a matter of the penny dropping. When self-driving cars aren't here over the next few years; and when more big biz tries and fails with 'AI' -- we will enter another winter. I have to say it will be schadenfreude from me -- given the absolutely reckless behaviour of the army of grant-chasing academics happy to inflate the bubble. When that funding suddenly disappears, it'll be there own fault.
- varelse 5y ago
  As skeptical as I am of the AGI and even full self-driving, successes like AlphaGo, AlphaFold 2, and Deepfakery from video to voice all demonstrate the technology can do things we never really could do before. But in all of those cases humans are in the loop to either build the system at scale or to keep it from going off the rails in production. I see much more of that in the next decade. And I don't think there's going to be an AI winter but I do hope there's a reckoning for all the snake oilers of AI out there. I mean they're the same people who snake oil for crypto too, and they're just getting their game on for snake oiling the metaverse. Maybe that will happen for AI VC, but as someone else said, VC is where good ideas go to die.
  - dougabug 5y ago
    Good points, but it should also be noted that humans also often rely on other humans supporting, assisting, and training / coaching them in order to perform at peak levels. Pre-Neolithic humans while biologically considered modern, lacked many of the expressed cognitive capabilities we now demand nascent AI systems to exhibit. Self-hosting AI capabilities may come at some point in the future, long after these human bootstrapped and maintained systems have proven their commercial worth.
  - mensetmanusman 5y ago
    Imagine letting gtp3 loose on to the worldwide comment sections. It has the ability to defeat any captcha, it can write well enough to waste peoples time, and it could push any agenda. I think when such a weapon is released, there will be nothing mere mortal web owners will be able to do, since there is no digital identity that couldn’t be spoofed by it.
    - varelse 5y ago
      You deeply deeply deeply overestimate these models but I also give up trying to make that point. Go ahead and unleash all this fear uncertainty and doubt and see how that goes. It's tiring even to read this nonsense. There is nothing these language models can do that isn't already being achieved by an entire planet of mechanical turks with nothing better to do. But they do so with a much better carbon footprint. There will be plenty of tools for motivated mere mortal web owners. Most of them involves various ways of saying no to uncurated content. But it's more fun to talk about the near-term AGI and the imaginary threat it poses. Who doesn't love a good Boogeyman story?
  - tuatoru 5y ago
    > successes like AlphaGo, AlphaFold 2, and Deepfakery from video to voice all demonstrate the technology can do things we never really could do before. Good grief, I hope so! After all that money! If no-one had come up with anything they could hype out the ears, the train would have stopped years ago. The problem isn't toy inventions, it's making a significant positive impact on the world as a whole. Generalisability and efficiency of deployment, creating benefits for billions of households. Lithium-chemistry batteries are having a net positive effect. AI seems to be a net negative, the way it's been used in Facebook and YouTube. Deepfakery is unambiguously a negative. The jury is still out on Alphafold 2. The default expectation would be that it's like Watson as used in medical diagnosis. There wasn't much 'there' there.
dougabug 5y ago
What you say seems undeniably true, however reifying fundamental concepts lies at or near the core of human natural intelligence. Perhaps more so than logic itself. Work on AI is basically as old as computing itself (i.e. the pioneers of computing explored from early on connectionist models of computing, neural networks, cellular automata, reinforcement learning, LISP, etc). AI has been evolving for eight decades, vast resources have been channeled into it; and frankly, the successes are striking. It would be almost impossible for us not to conceive of AI as a “thing.”
- dr_dshiv 5y ago
  It’s not so hard. Just a few years ago, say 2012, it was really unusual for academics in the field of AI to refer to any system as “AI” or “having AI” or “being an AI state.” Why? Because it was disingenuous. But then when IBM Watson was released, the dump trucks of money started showing up, and the self-policing stopped. Look, I’m not critiquing the field of AI or funding it. Referring to AI research is no problem. My problem comes from naming the outputs of the field “AI.” Because we all know it is a moving target, it doesn’t really mean anything and it confuses non-experts to the point that they make poor decisions.
  - mdp2021 5y ago
    The term is 'decision support systems'. But modern decision support systems may make use of AI (of the results of research under he umbrella of AI - in short, "AI". It is not necessary to have the bunny quotes of figurative speech everywhere, they should be understood even when inexplicit but clear). And AI in DSS increases the underlying problems.
  - dougabug 5y ago
    Sure, but in 2012, AI wasn’t a common subject for laypeople to discuss as non-fiction. Policy makers for decades mostly dismissed AI because of earlier failures to live up to the hype. In 2012, most people didn’t have smartphones in their pocket with “AI processors” capable of 10-30 trillion operations per second. Face, voice, and object recognition, real-time language translation etc weren’t ubiquitous and taken for granted. As AI becomes increasingly a realistic part of every day life, it necessarily invades language. Historically even words for colors took centuries to enter into common usage in most cultures. If AI researchers were to stop referring to AI as a whole, and individual AI technologies as “a thing,” it wouldn’t change how the rest of the population talks about AI. When I entered the field, my mentor said that the first thing to know was that, “Nothing works.” For years, branding something as AI was an invitation to not get funded. I doubt the restraint around the use of the term was entirely altuistic. AI skeptics notwithstanding, the train has left the station and the changing public discourse reflects that reality.
mdp2021 5y ago
> treating it like a real thing Oracular devices are a real thing - you have dimes for a coin toss. Decision making algorithms of scarce transparency do exist. It (the contextual object) is a real thing: AI as an oracular device is a real thing. > desire AI as an agent with independence The point is that we do not desire that. It is an agent when entrusted, and it is independent when unchecked, which the black-box nature of some AI technologies can facilitate. The issue is not with a misunderstanding of the term AI - which we have used for 65 years effortlessly, exactly «that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. [...] how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves» (McCarthy 1955) -, but with its real instances and very real potential adopters.
- dr_dshiv 5y ago
  > the term AI - which we have used for 65 years effortlessly, exactly «that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it… Except that AI research rarely tries to replicate how people think; instead, it create systems that mimic the typical outcomes of human effort. The term AI was invented for grant applications. What’s the difference between AI, Automation and Cybernetics?
  - mdp2021 5y ago
    > Except that AI research rarely tries to replicate how people think It does not matter (and, literally, «how people think» is not necessarily "intelligence" - a special feature). And, if it «mimic[s] the typical outcomes of human effort», it is still within the broad proposal from McCarthy of «solve kinds of problems now reserved for humans» to substantiate the general idea of an "artificial intelligence", which is literal: ___ there is a phenomenon called 'intelligence': can we replicate it through computing? The difference between the various areas: if it spawns from some consideration of the phenomenon of intelligence call it AI; if it can manage a process unsupervised call it Automation; if it refines its operation through feedback call it Cybernetics; if it manages a process unsupervised refining its operation through feedback in a system inspired by the phenomenon of intelligence call it Artificially Intelligent Cybernetic Automation - I do not see the problem. "It is not really intelligent": yes, we know. We study intelligence and see where that brings. We have posed a problem ("can we replicate intelligence through computing"), then we reap the outcomes of the effort. The big issue here is that the "unsupervised" part becomes very risky after the feedback if responsibilities are to it delegated beyond its worth of reliability, which is especially relevant if the """intelligence""" part is oracular, a black box, non-transparent, which is apparently something the authors note on a different perspective as they note that «the algorithm was able to spot aspects of reality that humans had not contemplated, might not be able to detect and may never comprehend». So, as "the strategist" said, "there are things we know we know, there are things we know we do not know, and there are things we do not know we do not know". We have a better idea of the reliability of a deterministic system; we do not know exactly what specific exceptions to our expectations can be raised by automation through less transparent systems; we know those systems are there for anyone to implement and delegate, and that is a problem.

mjburgess 5y ago

I think it's just a matter of the penny dropping. When self-driving cars aren't here over the next few years; and when more big biz tries and fails with 'AI' -- we will enter another winter. I have to say it will be schadenfreude from me -- given the absolutely reckless behaviour of the army of grant-chasing academics happy to inflate the bubble. When that funding suddenly disappears, it'll be there own fault.

varelse 5y ago
As skeptical as I am of the AGI and even full self-driving, successes like AlphaGo, AlphaFold 2, and Deepfakery from video to voice all demonstrate the technology can do things we never really could do before. But in all of those cases humans are in the loop to either build the system at scale or to keep it from going off the rails in production. I see much more of that in the next decade. And I don't think there's going to be an AI winter but I do hope there's a reckoning for all the snake oilers of AI out there. I mean they're the same people who snake oil for crypto too, and they're just getting their game on for snake oiling the metaverse. Maybe that will happen for AI VC, but as someone else said, VC is where good ideas go to die.
- dougabug 5y ago
  Good points, but it should also be noted that humans also often rely on other humans supporting, assisting, and training / coaching them in order to perform at peak levels. Pre-Neolithic humans while biologically considered modern, lacked many of the expressed cognitive capabilities we now demand nascent AI systems to exhibit. Self-hosting AI capabilities may come at some point in the future, long after these human bootstrapped and maintained systems have proven their commercial worth.
- mensetmanusman 5y ago
  Imagine letting gtp3 loose on to the worldwide comment sections. It has the ability to defeat any captcha, it can write well enough to waste peoples time, and it could push any agenda. I think when such a weapon is released, there will be nothing mere mortal web owners will be able to do, since there is no digital identity that couldn’t be spoofed by it.
  - varelse 5y ago
    You deeply deeply deeply overestimate these models but I also give up trying to make that point. Go ahead and unleash all this fear uncertainty and doubt and see how that goes. It's tiring even to read this nonsense. There is nothing these language models can do that isn't already being achieved by an entire planet of mechanical turks with nothing better to do. But they do so with a much better carbon footprint. There will be plenty of tools for motivated mere mortal web owners. Most of them involves various ways of saying no to uncurated content. But it's more fun to talk about the near-term AGI and the imaginary threat it poses. Who doesn't love a good Boogeyman story?
- tuatoru 5y ago
  > successes like AlphaGo, AlphaFold 2, and Deepfakery from video to voice all demonstrate the technology can do things we never really could do before. Good grief, I hope so! After all that money! If no-one had come up with anything they could hype out the ears, the train would have stopped years ago. The problem isn't toy inventions, it's making a significant positive impact on the world as a whole. Generalisability and efficiency of deployment, creating benefits for billions of households. Lithium-chemistry batteries are having a net positive effect. AI seems to be a net negative, the way it's been used in Facebook and YouTube. Deepfakery is unambiguously a negative. The jury is still out on Alphafold 2. The default expectation would be that it's like Watson as used in medical diagnosis. There wasn't much 'there' there.

dougabug 5y ago

What you say seems undeniably true, however reifying fundamental concepts lies at or near the core of human natural intelligence. Perhaps more so than logic itself. Work on AI is basically as old as computing itself (i.e. the pioneers of computing explored from early on connectionist models of computing, neural networks, cellular automata, reinforcement learning, LISP, etc). AI has been evolving for eight decades, vast resources have been channeled into it; and frankly, the successes are striking. It would be almost impossible for us not to conceive of AI as a “thing.”

dr_dshiv 5y ago
It’s not so hard. Just a few years ago, say 2012, it was really unusual for academics in the field of AI to refer to any system as “AI” or “having AI” or “being an AI state.” Why? Because it was disingenuous. But then when IBM Watson was released, the dump trucks of money started showing up, and the self-policing stopped. Look, I’m not critiquing the field of AI or funding it. Referring to AI research is no problem. My problem comes from naming the outputs of the field “AI.” Because we all know it is a moving target, it doesn’t really mean anything and it confuses non-experts to the point that they make poor decisions.
- mdp2021 5y ago
  The term is 'decision support systems'. But modern decision support systems may make use of AI (of the results of research under he umbrella of AI - in short, "AI". It is not necessary to have the bunny quotes of figurative speech everywhere, they should be understood even when inexplicit but clear). And AI in DSS increases the underlying problems.
- dougabug 5y ago
  Sure, but in 2012, AI wasn’t a common subject for laypeople to discuss as non-fiction. Policy makers for decades mostly dismissed AI because of earlier failures to live up to the hype. In 2012, most people didn’t have smartphones in their pocket with “AI processors” capable of 10-30 trillion operations per second. Face, voice, and object recognition, real-time language translation etc weren’t ubiquitous and taken for granted. As AI becomes increasingly a realistic part of every day life, it necessarily invades language. Historically even words for colors took centuries to enter into common usage in most cultures. If AI researchers were to stop referring to AI as a whole, and individual AI technologies as “a thing,” it wouldn’t change how the rest of the population talks about AI. When I entered the field, my mentor said that the first thing to know was that, “Nothing works.” For years, branding something as AI was an invitation to not get funded. I doubt the restraint around the use of the term was entirely altuistic. AI skeptics notwithstanding, the train has left the station and the changing public discourse reflects that reality.

mdp2021 5y ago

> treating it like a real thing Oracular devices are a real thing - you have dimes for a coin toss. Decision making algorithms of scarce transparency do exist. It (the contextual object) is a real thing: AI as an oracular device is a real thing. > desire AI as an agent with independence The point is that we do not desire that. It is an agent when entrusted, and it is independent when unchecked, which the black-box nature of some AI technologies can facilitate. The issue is not with a misunderstanding of the term AI - which we have used for 65 years effortlessly, exactly «that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. [...] how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves» (McCarthy 1955) -, but with its real instances and very real potential adopters.

dr_dshiv 5y ago
> the term AI - which we have used for 65 years effortlessly, exactly «that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it… Except that AI research rarely tries to replicate how people think; instead, it create systems that mimic the typical outcomes of human effort. The term AI was invented for grant applications. What’s the difference between AI, Automation and Cybernetics?
- mdp2021 5y ago
  > Except that AI research rarely tries to replicate how people think It does not matter (and, literally, «how people think» is not necessarily "intelligence" - a special feature). And, if it «mimic[s] the typical outcomes of human effort», it is still within the broad proposal from McCarthy of «solve kinds of problems now reserved for humans» to substantiate the general idea of an "artificial intelligence", which is literal: ___ there is a phenomenon called 'intelligence': can we replicate it through computing? The difference between the various areas: if it spawns from some consideration of the phenomenon of intelligence call it AI; if it can manage a process unsupervised call it Automation; if it refines its operation through feedback call it Cybernetics; if it manages a process unsupervised refining its operation through feedback in a system inspired by the phenomenon of intelligence call it Artificially Intelligent Cybernetic Automation - I do not see the problem. "It is not really intelligent": yes, we know. We study intelligence and see where that brings. We have posed a problem ("can we replicate intelligence through computing"), then we reap the outcomes of the effort. The big issue here is that the "unsupervised" part becomes very risky after the feedback if responsibilities are to it delegated beyond its worth of reliability, which is especially relevant if the """intelligence""" part is oracular, a black box, non-transparent, which is apparently something the authors note on a different perspective as they note that «the algorithm was able to spot aspects of reality that humans had not contemplated, might not be able to detect and may never comprehend». So, as "the strategist" said, "there are things we know we know, there are things we know we do not know, and there are things we do not know we do not know". We have a better idea of the reliability of a deterministic system; we do not know exactly what specific exceptions to our expectations can be raised by automation through less transparent systems; we know those systems are there for anyone to implement and delegate, and that is a problem.