8 ms·

A DARPA Perspective on Artificial Intelligence [pdf]

186 points by nulluint 5y ago

abel_ 5y ago
The most pressing dangers of AI most researchers see: - error rate too high - you can trick a classifier with noise - it's racist sometimes Actual dangers of AI: - stop problem - infeasibility of sandboxing - difficulty of aligning black boxes with human values
shpat 5y ago
Here’s the presentation in video form with a bit more context & detail. https://youtu.be/-O01G3tSYpU https://youtu.be/-O01G3tSYpU
gwern 5y ago
BTW, the Tay example in the slides is bogus. Zero AI involved. You should ask what's cut off when you see a tweet reply like that. This is what's cut off: https://twitter.com/daviottenheimer/status/712889915533500416 https://twitter.com/daviottenheimer/status/71288991553350041... Tay had a 'repeat after me' feature (an ancient feature of any IRC chatbot to just echo or print a given string). That's all this is. A troll account issued the command 'Repeat after me' and then tweeted '@TayandYou HITLER DID NOTHING WRONG!', and the Tay daemon dutifully repeated it back to the troll in that thread.
- mach1ne 5y ago
  How much of the controversy surrounding Tay is a result of this specific command?
  - gwern 5y ago
    Possibly all of it. As best as I can tell, there may have been a few milquetoast rudenesses, of the usual sort for language models, but the actual quotes everyone cites with detailed statements about the Holocaust or Hitler seem to have all been repeat-after-mes then ripped out of context. It's hard to say given how most of the relevant material has been deleted, and what survives is the usual endless echo chamber of miscitation and simplification and 'everyone knows' which you rapidly become familiar with if you ever try to factcheck anything down to the original sources. (This is why lots of people still 'know' Cambridge Analytica swung the election, or they 'know' the Twitter facecropping algorithm was hugely biased, or that 'Amazon's HR software would only hire you if you played lacrosse', or 'this guy was falsely arrested because face recognition picked him' etc.) If you look at the very earliest reporting, they mostly say it was repeat-after-me functionality, hedging a bit (because who can prove every inflammatory Tay statement was a repeat-after-me?), and then that rapidly gets dropped in favor of narratives about Tay 'learning'. (Even though if you look at the chatbot code MS released later, it's not obvious at all how exactly Tay would 'learn' in the day or so it had before shutdown.) Anyway, long story short, the Tay incident is either entirely or mostly bogus in the way people want to use it (as an AI safety parable). The real story, of an `echo` gone wrong, is vastly less interesting, and is about as important as typing '8008' into your calculator and showing it to your teacher.
    - operator-name 5y ago
      Searching "tay ai" into Google images leads to a handful images that show the initial comment are clearly not "repeat after me" responses. Of those that exist, most of them have been pretty offensive.
      - gwern 5y ago
        I think you are being insufficiently mediate-literate and skeptical and willing to take screenshots at face-value, given that I just demonstrated that a widely-cited example is in fact maliciously and misleadingly edited to remove the context and lie to the viewer about what happened. Did any of the coverage you read mention that? No, they did not. So why do you trust them on everything else and assume they did a good job researching and factchecking when they were so clearly wrong on that one? Why do you believe the others are not repeat-after-mes? Shouldn't the burden of proof be on anyone who claims a specific tweet can be trusted even if those others are bad? If I go through 'tay ai' (as I have before), I see much the same thing: I see a lot of clearly edited squirrely images, which remove all of the context, and when they appear to include context, the UI looks wrong (some of these are clearly using the 'Replies' tab on the Tay page, instead of being on the actual convo thread; why? to remove the context with the repeat-after-mes or which would show Tay is just spitting out lots of canned generic responses, of course), and like some tweets are being edited out and the remainder spliced together. Do you really think that a 2016-era mass market chatbot (sub-char-RNN in power, often relying on heavily engineered template/script databases) really knows how to do Trump memes complete with clap emoji? What dataset has those? Of course not. It's just a repeat-after-me where the initial tweets were edited out. (What? Someone edit a screenshot, especially where no one can check the original, to score political points? You really think someone would do that - just go on the Internet and tell lies?) If we look at ones which seem semi-legitimate, like the genocide one, then all we are seeing is typical chatbot evasiveness and generic Eliza-level responses, cherrypicked out of the 100k or so tweets Tay made before they were all deleted. "Do you support $X?" "I do indeed". Wow! Stunning evidence that 'Microsoft turned an advanced AI loose on the Internet and trolls taught it to be evil', as the narrative goes. So, like I said. All of the long, coherent, strikingly detailed and most offensive ones appear to be either blatantly or probably repeat-after-mes, while the plausible ones are empty chatbot spacefilling cherrypicked out of countless thousands of tweets and made to look more offensive by implying they are the same as the troll-written repeat-after-mes. Neither one supports the broad narrative Tay is used for by OP. And we can note that, beyond the points I've raised here like "how on earth did it 'learn' so quickly" and "there is no evidence it even had 'learning' turned on", there is generally a striking lack of replications of Tay. Which is exactly what you would expect if all the really offensive ones were due to a single ill-considered repeat-after-me feature left enabled, and the rest are just chatbot phaticisms.
- hackernewds 5y ago
  This is quite a tall claim. Are there any other supporting arguments? Missing an edge case abuse could be a vulnerability for any good AI still
readhn 5y ago
cool stuff. thanks for sharing!
bigcat123 5y ago
itsmefaz 5y ago
Nice
Animats 5y ago
"Contextual adaptation" is what they want, but that doesn't mean it's coming in the near future. However, this does mean that funding for research on it will be available. As I've said for years, the big lack in AI is in the "common sense" and unstructured manipulation area. Nobody can build something with squirrel levels of manipulation and agility, even in simulation. Robot manipulation in unstructured situations is still very poor. The people trying to simulate C. elegans at the neuron level can't get that to work, despite a full wiring diagram and years of effort. Something very low level is not understood. There's a Nobel Prize waiting for whomever figures that out.
- hans1729 5y ago
  > Nobody can build something with squirrel levels of manipulation and agility, even in simulation. Isn’t that what alpha go or the StarCraft league are? Organic strategies in well-defined contexts (action options of the squirrel at Tn)? “Squirrel” is a nice reference frame.
  - TaylorAlexander 5y ago
    Manipulation in the sense of being able to physically pick things up and manipulate them. Which is a difficult problem. StarCraft is moving a mouse pointer and clicking.
    - runnerup 5y ago
      AlphaStar dropped the pretenses of even doing that much because it added too much complexity. For its show matches in Starcraft2, AlphaStar just uses an API to say "Target this specific enemy carrier". It doesn't have to do any vision recognition or differentiate between multiple overlapping units like humans do. That was the original intent, but it appears the dropped it before public demos.
  - davidmanheim 5y ago
    No - those systems simulate "control" at the level of giving commands, not actual motor control. So practically, current AI can win at Go and play at superhuman levels, but still cannot drive a car as well as most adults.
mooreds 5y ago
Nice high level overview of the three waves of AI (two have happened, the "Contextual adaptation" wave is yet to occur). Includes examples and successes and failures. ("Young man holding a baseball bat", indeed :) ). "Systems construct contextual explanatory models for classes of real world phenomena" is the next goal. That is, understanding + being able to describe the reasoning for the understanding. No technical depth, really, but lots of words to google if you want to learn more.
- davidmanheim 5y ago
  People may want to see the appendix to our RAND report on areas related to DARPA's focus, for more depth on the relevant uses of automation, ML, and AI -https://www.rand.org/pubs/research_reports/RR2489.html https://www.rand.org/pubs/research_reports/RR2489.html
Ziggy_Zaggy 5y ago
Very thought-provoking.
jll29 5y ago
It's good to see a main funding agency's perspective. As a researcher, I like their non-hype way of defining AI as "programmed ability", which is accurate and realistic -- also puts AI further apart from real intelligence, which means unanticipated activities. I would like to know more what they see as "abstracting", from their perspective. We haven't got much further in our scientific understanding of intelligence - if you bought a psychology text book today and ten years ago there wouldn't be much of a breakthrough change detectable in terms of modeling cognition. And as impressive as some computer science AI models perform certain tasks, I haven't been taken by surprise by them asking me a question out of the blue, which is one of my personal litmus tests for intelligence.
- dataking 5y ago
  > As a researcher, I like their non-hype way of defining AI as "programmed ability", which is accurate and realistic This is the first point of advice in the Heilmeier Catechism https://www.darpa.mil/work-with-us/heilmeier-catechism https://www.darpa.mil/work-with-us/heilmeier-catechism
hiddencost 5y ago
I wonder who is trying to get what funded? "Second wave" is going gangbusters, despite Gary Marcus' every-six-months rant; the review of statistical learning is reasonable given a barely technical audience, but the summary of "third wave" seems designed to extract large amounts of funding from people who aren't up to date on the state of the field.
- gibsonf1 5y ago
  Theranos got a great deal of funding too, and the truth about how none of the "Second Wave" ML self driving car technology comes even remotely close to safe self driving will be coming out probably later this year. The issue is that ML has no conceptual and causal understanding of anything. For confirmation, I've been carefully watching the countless "self driving" startups in San Francisco driving around, and I have almost never seen a driver in those cars not actively steering it.
  - AlotOfReading 5y ago
    You can't always tell whether the car is operating autonomously even from the back seat, let alone another vehicle. A lot of operators keep their hands very close to the wheel and follow its motions to minimize response time when they need to take over. I've worked at companies whose vehicles you've probably seen and have never heard anything like what you're suggesting is "the truth". Can you provide more details on how current solutions don't remotely approach reasonable safety levels?
  - hiddencost 5y ago
    > The issue is that ML has no conceptual and causal understanding of anything Practitioners work with much more tightly defined objectives and methods for improving their systems than "conceptual and causal understanding of anything". From the perspective of folks in the industry we've been making remarkable progress, well beyond what we could have expected. We're saturating major benchmarks, sometimes in one to two years. Back in the day, during the "AI Winter", it was often 10 years before a new breakthrough happened.
  - redytedy 5y ago
GWBullshit 5y ago
- GWBullshit 5y ago
  - GWBullshit 5y ago
    - GWBullshit 5y ago
      - GWBullshit 5y ago
deleted 5y ago
[deleted]
alins 5y ago
FYI the presentation is from December 2016.
- sigmoid10 5y ago
  I was gonna say this seems really outdated. 2016 was ages ago in the AI world. We've had Openai Five, AlphaStar, MuZero, BERT, GTP and tons of other groundbreaking stuff since then. This presentation tells you nothing about the current state of the field.
MaxMoney 5y ago
deleted 5y ago
[deleted]
gone35 5y ago
(2016?) Also: > First wave: Handcrafted knowledge > Second wave: Statistical learning > Third wave: Contextual adaptation I understood clearly enough the first two, but the slides become increasingly ambiguous and fuzzy towards the end; and it seems to me they are mixing up a bunch of not self-evidently related desiderata. It is not immediate, for instance, that small "generative" models that are easy to interpret necessarily lead to better "abstraction" (whatever that means). And whatever this all has to do with "contextual adaptation" is to me anyone's guess. Highly alarming (but sadly, from experience, unsurprising) to see such fuzzy position document from such an important funding agency for AI.
Khelouiati 5y ago
2016 ??????
ochicial 5y ago
Here is the presentation in form of a blog post: https://machinelearning.technicacuriosa.com/2017/03/19/a-darpa-perspective-on-artificial-intelligence/ https://machinelearning.technicacuriosa.com/2017/03/19/a-dar...
- gwern 5y ago
  This submission should have a (2016)/(2017) on it. I looked at the image caption example and immediately went 'wait, that's wrong, current 2020/2021 captioners would get that right', but of course, for 2017 that's plausible.
Khelouiati 5y ago
2016 ?????

186 points by nulluint 5y ago

The most pressing dangers of AI most researchers see: - error rate too high - you can trick a classifier with noise - it's racist sometimes Actual dangers of AI: - stop problem - infeasibility of sandboxing - difficulty of aligning black boxes with human values

shpat 5y ago

Here’s the presentation in video form with a bit more context & detail. https://youtu.be/-O01G3tSYpU https://youtu.be/-O01G3tSYpU

gwern 5y ago

BTW, the Tay example in the slides is bogus. Zero AI involved. You should ask what's cut off when you see a tweet reply like that. This is what's cut off: https://twitter.com/daviottenheimer/status/712889915533500416 https://twitter.com/daviottenheimer/status/71288991553350041... Tay had a 'repeat after me' feature (an ancient feature of any IRC chatbot to just echo or print a given string). That's all this is. A troll account issued the command 'Repeat after me' and then tweeted '@TayandYou HITLER DID NOTHING WRONG!', and the Tay daemon dutifully repeated it back to the troll in that thread.

mach1ne 5y ago
How much of the controversy surrounding Tay is a result of this specific command?
- gwern 5y ago
  Possibly all of it. As best as I can tell, there may have been a few milquetoast rudenesses, of the usual sort for language models, but the actual quotes everyone cites with detailed statements about the Holocaust or Hitler seem to have all been repeat-after-mes then ripped out of context. It's hard to say given how most of the relevant material has been deleted, and what survives is the usual endless echo chamber of miscitation and simplification and 'everyone knows' which you rapidly become familiar with if you ever try to factcheck anything down to the original sources. (This is why lots of people still 'know' Cambridge Analytica swung the election, or they 'know' the Twitter facecropping algorithm was hugely biased, or that 'Amazon's HR software would only hire you if you played lacrosse', or 'this guy was falsely arrested because face recognition picked him' etc.) If you look at the very earliest reporting, they mostly say it was repeat-after-me functionality, hedging a bit (because who can prove every inflammatory Tay statement was a repeat-after-me?), and then that rapidly gets dropped in favor of narratives about Tay 'learning'. (Even though if you look at the chatbot code MS released later, it's not obvious at all how exactly Tay would 'learn' in the day or so it had before shutdown.) Anyway, long story short, the Tay incident is either entirely or mostly bogus in the way people want to use it (as an AI safety parable). The real story, of an `echo` gone wrong, is vastly less interesting, and is about as important as typing '8008' into your calculator and showing it to your teacher.
  - operator-name 5y ago
    Searching "tay ai" into Google images leads to a handful images that show the initial comment are clearly not "repeat after me" responses. Of those that exist, most of them have been pretty offensive.
    - gwern 5y ago
      I think you are being insufficiently mediate-literate and skeptical and willing to take screenshots at face-value, given that I just demonstrated that a widely-cited example is in fact maliciously and misleadingly edited to remove the context and lie to the viewer about what happened. Did any of the coverage you read mention that? No, they did not. So why do you trust them on everything else and assume they did a good job researching and factchecking when they were so clearly wrong on that one? Why do you believe the others are not repeat-after-mes? Shouldn't the burden of proof be on anyone who claims a specific tweet can be trusted even if those others are bad? If I go through 'tay ai' (as I have before), I see much the same thing: I see a lot of clearly edited squirrely images, which remove all of the context, and when they appear to include context, the UI looks wrong (some of these are clearly using the 'Replies' tab on the Tay page, instead of being on the actual convo thread; why? to remove the context with the repeat-after-mes or which would show Tay is just spitting out lots of canned generic responses, of course), and like some tweets are being edited out and the remainder spliced together. Do you really think that a 2016-era mass market chatbot (sub-char-RNN in power, often relying on heavily engineered template/script databases) really knows how to do Trump memes complete with clap emoji? What dataset has those? Of course not. It's just a repeat-after-me where the initial tweets were edited out. (What? Someone edit a screenshot, especially where no one can check the original, to score political points? You really think someone would do that - just go on the Internet and tell lies?) If we look at ones which seem semi-legitimate, like the genocide one, then all we are seeing is typical chatbot evasiveness and generic Eliza-level responses, cherrypicked out of the 100k or so tweets Tay made before they were all deleted. "Do you support $X?" "I do indeed". Wow! Stunning evidence that 'Microsoft turned an advanced AI loose on the Internet and trolls taught it to be evil', as the narrative goes. So, like I said. All of the long, coherent, strikingly detailed and most offensive ones appear to be either blatantly or probably repeat-after-mes, while the plausible ones are empty chatbot spacefilling cherrypicked out of countless thousands of tweets and made to look more offensive by implying they are the same as the troll-written repeat-after-mes. Neither one supports the broad narrative Tay is used for by OP. And we can note that, beyond the points I've raised here like "how on earth did it 'learn' so quickly" and "there is no evidence it even had 'learning' turned on", there is generally a striking lack of replications of Tay. Which is exactly what you would expect if all the really offensive ones were due to a single ill-considered repeat-after-me feature left enabled, and the rest are just chatbot phaticisms.
hackernewds 5y ago
This is quite a tall claim. Are there any other supporting arguments? Missing an edge case abuse could be a vulnerability for any good AI still

readhn 5y ago

cool stuff. thanks for sharing!

bigcat123 5y ago

itsmefaz 5y ago

Nice

Animats 5y ago

"Contextual adaptation" is what they want, but that doesn't mean it's coming in the near future. However, this does mean that funding for research on it will be available. As I've said for years, the big lack in AI is in the "common sense" and unstructured manipulation area. Nobody can build something with squirrel levels of manipulation and agility, even in simulation. Robot manipulation in unstructured situations is still very poor. The people trying to simulate C. elegans at the neuron level can't get that to work, despite a full wiring diagram and years of effort. Something very low level is not understood. There's a Nobel Prize waiting for whomever figures that out.

hans1729 5y ago
> Nobody can build something with squirrel levels of manipulation and agility, even in simulation. Isn’t that what alpha go or the StarCraft league are? Organic strategies in well-defined contexts (action options of the squirrel at Tn)? “Squirrel” is a nice reference frame.
- TaylorAlexander 5y ago
  Manipulation in the sense of being able to physically pick things up and manipulate them. Which is a difficult problem. StarCraft is moving a mouse pointer and clicking.
  - runnerup 5y ago
    AlphaStar dropped the pretenses of even doing that much because it added too much complexity. For its show matches in Starcraft2, AlphaStar just uses an API to say "Target this specific enemy carrier". It doesn't have to do any vision recognition or differentiate between multiple overlapping units like humans do. That was the original intent, but it appears the dropped it before public demos.
- davidmanheim 5y ago
  No - those systems simulate "control" at the level of giving commands, not actual motor control. So practically, current AI can win at Go and play at superhuman levels, but still cannot drive a car as well as most adults.

mooreds 5y ago

Nice high level overview of the three waves of AI (two have happened, the "Contextual adaptation" wave is yet to occur). Includes examples and successes and failures. ("Young man holding a baseball bat", indeed :) ). "Systems construct contextual explanatory models for classes of real world phenomena" is the next goal. That is, understanding + being able to describe the reasoning for the understanding. No technical depth, really, but lots of words to google if you want to learn more.

davidmanheim 5y ago
People may want to see the appendix to our RAND report on areas related to DARPA's focus, for more depth on the relevant uses of automation, ML, and AI -https://www.rand.org/pubs/research_reports/RR2489.html https://www.rand.org/pubs/research_reports/RR2489.html

Ziggy_Zaggy 5y ago

Very thought-provoking.

jll29 5y ago

It's good to see a main funding agency's perspective. As a researcher, I like their non-hype way of defining AI as "programmed ability", which is accurate and realistic -- also puts AI further apart from real intelligence, which means unanticipated activities. I would like to know more what they see as "abstracting", from their perspective. We haven't got much further in our scientific understanding of intelligence - if you bought a psychology text book today and ten years ago there wouldn't be much of a breakthrough change detectable in terms of modeling cognition. And as impressive as some computer science AI models perform certain tasks, I haven't been taken by surprise by them asking me a question out of the blue, which is one of my personal litmus tests for intelligence.

dataking 5y ago
> As a researcher, I like their non-hype way of defining AI as "programmed ability", which is accurate and realistic This is the first point of advice in the Heilmeier Catechism https://www.darpa.mil/work-with-us/heilmeier-catechism https://www.darpa.mil/work-with-us/heilmeier-catechism

hiddencost 5y ago

I wonder who is trying to get what funded? "Second wave" is going gangbusters, despite Gary Marcus' every-six-months rant; the review of statistical learning is reasonable given a barely technical audience, but the summary of "third wave" seems designed to extract large amounts of funding from people who aren't up to date on the state of the field.

gibsonf1 5y ago
Theranos got a great deal of funding too, and the truth about how none of the "Second Wave" ML self driving car technology comes even remotely close to safe self driving will be coming out probably later this year. The issue is that ML has no conceptual and causal understanding of anything. For confirmation, I've been carefully watching the countless "self driving" startups in San Francisco driving around, and I have almost never seen a driver in those cars not actively steering it.
- AlotOfReading 5y ago
  You can't always tell whether the car is operating autonomously even from the back seat, let alone another vehicle. A lot of operators keep their hands very close to the wheel and follow its motions to minimize response time when they need to take over. I've worked at companies whose vehicles you've probably seen and have never heard anything like what you're suggesting is "the truth". Can you provide more details on how current solutions don't remotely approach reasonable safety levels?
- hiddencost 5y ago
  > The issue is that ML has no conceptual and causal understanding of anything Practitioners work with much more tightly defined objectives and methods for improving their systems than "conceptual and causal understanding of anything". From the perspective of folks in the industry we've been making remarkable progress, well beyond what we could have expected. We're saturating major benchmarks, sometimes in one to two years. Back in the day, during the "AI Winter", it was often 10 years before a new breakthrough happened.
- redytedy 5y ago

GWBullshit 5y ago

GWBullshit 5y ago
- GWBullshit 5y ago
  - GWBullshit 5y ago
    - GWBullshit 5y ago

deleted 5y ago

[deleted]

alins 5y ago

FYI the presentation is from December 2016.

sigmoid10 5y ago
I was gonna say this seems really outdated. 2016 was ages ago in the AI world. We've had Openai Five, AlphaStar, MuZero, BERT, GTP and tons of other groundbreaking stuff since then. This presentation tells you nothing about the current state of the field.

MaxMoney 5y ago

deleted 5y ago

[deleted]

gone35 5y ago

(2016?) Also: > First wave: Handcrafted knowledge > Second wave: Statistical learning > Third wave: Contextual adaptation I understood clearly enough the first two, but the slides become increasingly ambiguous and fuzzy towards the end; and it seems to me they are mixing up a bunch of not self-evidently related desiderata. It is not immediate, for instance, that small "generative" models that are easy to interpret necessarily lead to better "abstraction" (whatever that means). And whatever this all has to do with "contextual adaptation" is to me anyone's guess. Highly alarming (but sadly, from experience, unsurprising) to see such fuzzy position document from such an important funding agency for AI.

Khelouiati 5y ago

2016 ??????

ochicial 5y ago

Here is the presentation in form of a blog post: https://machinelearning.technicacuriosa.com/2017/03/19/a-darpa-perspective-on-artificial-intelligence/ https://machinelearning.technicacuriosa.com/2017/03/19/a-dar...

gwern 5y ago
This submission should have a (2016)/(2017) on it. I looked at the image caption example and immediately went 'wait, that's wrong, current 2020/2021 captioners would get that right', but of course, for 2017 that's plausible.

Khelouiati 5y ago

2016 ?????