4 ms·

This paper is written by a non-technical author, criticising AI based on predictions made by non-technical pundits. By concentrating on binary outcome predicti

by nl 6y ago

This paper is written by a non-technical author, criticising AI based on predictions made by non-technical pundits. By concentrating on binary outcome predictions (Do we have "full self driving cars" available to the general public?) it misses the real progress made in a huge number of areas. For example, one of the claims this paper claims to be false is Zuckerberg's 2015 declaration that: One of [Facebook’s] goals for the next five to 10 years is to basically get better than human level at all of the primary human senses: vision, hearing, language, general cognition There's little doubt that AI systems are already better than humans in vision and hearing and there is very clear progress in language. Cognition is ill-defined, but on most benchmarks attempting to measure this there is steady progress too. I think the real reason AI is harder than we think is because any time progress is made, humans redefine AI as "not that thing we just solved" Take Marcus' claim that charades is too hard for AIs. I think models like Open AI's Dall-E show clear progress towards developing the kinds of techniques needed to solve this, and if there was a benchmark for it I bet computers would out-perform humans in less than 5 years.

usgroup 6y ago
I think computers have been better at stuff then humans the whole time . And when they are better at all the particular things it still doesn’t result in common sense. That’s one of the fallacies.
punkspider 6y ago
Just wanted to point out that I don't think she's a non-technical author https://en.wikipedia.org/wiki/Melanie_Mitchell https://en.wikipedia.org/wiki/Melanie_Mitchell
- YeGoblynQueenne 6y ago
  Also the paper mainly comments on the work and sayings of AI and CS researchers like John McCarthy, Stuart Russel, Andrew Ng, Geoff Hinton, Hans Moravec, Drew McDermot, Claude Shannon, Alan Turing etc. i.e. hardly "non-technical pundits".
YeGoblynQueenne 6y ago
The paper criticises the benchmarks that are typically used to show that "AI systems are already better than humans in vision and [speech processing]" etc. For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems. The paper refers to various pieces of work within natural language processing and other sub-fields of AI that analyse the weaknesses of such benchmarks and investigate the ability of modern systems to beat benchmarks by finding shortcuts or exploiting surface statistical regularities etc. I recommend, for example: T. McCoy, E. Pavlick, and T. Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics innatural language inference. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3428–3448, 2019. https://www.aclweb.org/anthology/P19-1334.pdf https://www.aclweb.org/anthology/P19-1334.pdf
- nl 6y ago
  > For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems. I'd note that I said "clear progress on language understanding". > we could all be AIs debating the paper if those benchmarks really did measure language understanding in terms of the level of proficiency between the researchers, then it's no surprise that the results are very different. The next chart shows the results of the most common tests for language comprehension. If we look at the results of the different tests, then we see that the average proficiency is very high (12.9 out of 20). If we look at the results of the tests for other things, then the average proficiency is very low. Again, the charts are very similar to the results of the two separate results of the two separate tests for language comprehension. So, the question is, what does this tell us about the impact of language learning on literacy? I'll give you what I think is the relevant point. I think the key point here is that language learning is the process of learning new things. So, there are some things that are learning in the language. The text here was generated by a GPT-2 based model (aitextgen). I think there is a fair argument that it isn't far off the average level of discourse on HN.
Borrible 6y ago
So, am I right, that 'non-technical author' part means, she doesn't sell AI?

by nl 6y ago

usgroup 6y ago

I think computers have been better at stuff then humans the whole time . And when they are better at all the particular things it still doesn’t result in common sense. That’s one of the fallacies.

punkspider 6y ago

Just wanted to point out that I don't think she's a non-technical author https://en.wikipedia.org/wiki/Melanie_Mitchell https://en.wikipedia.org/wiki/Melanie_Mitchell

YeGoblynQueenne 6y ago
Also the paper mainly comments on the work and sayings of AI and CS researchers like John McCarthy, Stuart Russel, Andrew Ng, Geoff Hinton, Hans Moravec, Drew McDermot, Claude Shannon, Alan Turing etc. i.e. hardly "non-technical pundits".

YeGoblynQueenne 6y ago

The paper criticises the benchmarks that are typically used to show that "AI systems are already better than humans in vision and [speech processing]" etc. For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems. The paper refers to various pieces of work within natural language processing and other sub-fields of AI that analyse the weaknesses of such benchmarks and investigate the ability of modern systems to beat benchmarks by finding shortcuts or exploiting surface statistical regularities etc. I recommend, for example: T. McCoy, E. Pavlick, and T. Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics innatural language inference. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3428–3448, 2019. https://www.aclweb.org/anthology/P19-1334.pdf https://www.aclweb.org/anthology/P19-1334.pdf

nl 6y ago
> For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems. I'd note that I said "clear progress on language understanding". > we could all be AIs debating the paper if those benchmarks really did measure language understanding in terms of the level of proficiency between the researchers, then it's no surprise that the results are very different. The next chart shows the results of the most common tests for language comprehension. If we look at the results of the different tests, then we see that the average proficiency is very high (12.9 out of 20). If we look at the results of the tests for other things, then the average proficiency is very low. Again, the charts are very similar to the results of the two separate results of the two separate tests for language comprehension. So, the question is, what does this tell us about the impact of language learning on literacy? I'll give you what I think is the relevant point. I think the key point here is that language learning is the process of learning new things. So, there are some things that are learning in the language. The text here was generated by a GPT-2 based model (aitextgen). I think there is a fair argument that it isn't far off the average level of discourse on HN.

Borrible 6y ago

So, am I right, that 'non-technical author' part means, she doesn't sell AI?