4 ms·

Ugh. I dislike the author s permanent negativity, but he s right about a lot. I think it’s worth asking why people feel the need to lie about the future of AI?

Ugh. I dislike the author s permanent negativity, but he s right about a lot. I think it’s worth asking why people feel the need to lie about the future of AI? If they are confident about its future (and I don’t know of a fundamental reason why they would not be) then there is no reason to rush half assed results out the door and overcompensate (like gpt2). There is plenty of theoretical questions and answers to debate publicly instead. I don’t know if its the fear of some impending tech recession or fear of their own incompetence

Barrin92 7y ago
>Ugh. I dislike the author s permanent negativity, but he s right about a lot. Yes, in a way it must suck to always act like the grumpy party-crasher but Gary Marcus is absolutely spot on when it comes to the facts. That interview with the economist had me shaking my head, everyone had to know within five seconds that the chance that this is uncurated is zero.
deleted 7y ago
[deleted]
Fishysoup 7y ago
It's the bullshit process by which people get funding. Make something that has some potentially interesting engineering value, hype it up to hell and back, and funding agencies, investors etc. are more likely to back you. Probably this wasn't everyone's first plan all along, but when people who make a marginally better hot-dog / not-hot-dog classifier put so much spin on it, then everyone else has to just to remain visible. Moreover, people have to publish findings before their competition does, meaning sloppier and less interesting work. It's the snake-eating-its-own-tail plague that impacts so much of academia.
- buboard 7y ago
  > impacts so much of academia i think the trend started from the industry , but you re right warrantless self-promotion very pervasive in academia , and it's sad that it works!
- unishark 7y ago
  I'd say its the job of academic researchers to dream up high-risk ideas to work on, and to hype up incremental advances as potentially proof-of-concept for those ideas (when they're decades away still, with many potentially unsolvable hurdles left). Generally they are careful to give all sides though, since their reputation among their peers is the basis for their career. The media is a very different story. They just need to catch the interest of the mostly-ignorant, then move on to something new the next story.
ACow_Adonis 7y ago
I dislike how, when dealing with a general industry of spruikers and boosters, speaking the truth is perceived as permanent negativity :P He's not negative in this article. He's right.
6gvONxR4sf7o 7y ago
> I think it’s worth asking why people feel the need to lie about the future of AI? I wonder if it's because it's so vague and fuzzy, or because the techniques are so general. Like with self driving cars. The will someday probably be safer than people and that's huge. People get excited. We want that! Self driving cars save lives! But in those four sentences we went from "will someday probably" to let's do it now. The class of thing we're talking about now and the class of thing in the future are one and the same, so it's hard to talk about the future versions as distinctly separate from today's. AI will probably be able to talk to people well. We have AI today that talks to people. They're not the same thing, but these two sentences don't make that clear, because it's all AI. In principle, polynomials can learn any function! And we have polynomials today! We can learn anything! Rinse and repeat with fourier series or (as a totally random example) deep learning and it sounds like tomorrow's techniques are the same as today's, so we're done, right? Or maybe it's on lay people's poor math and stats skills and lack of understanding of the simple stuff. If I tell lay people I do stats, they think I'm taking an average with a lot of bureaucracy they don't really get. They won't think I'm using simple logistic regression to do really cool stuff like classify documents. They didn't know "stats" could do that! So they might be even more misled about what I do if I call it "statistics" than if I call it "AI." If they're mislead whatever I call it, we're already screwed.
gwern 7y ago
He's also wrong about a lot. For all his insistence on accuracy, he himself is misleading or ignorant. Like his slam of the Dartmouth project - even if you knew nothing about it, all you have to do is click through to see the claims of 'solving vision' are sheer projection and urban legend. And he's happy to make up claims out of whole cloth: for example, when he says "AlphaGo works fine on a 19x19 board, but would need to be retrained to play on a rectangular board; the lack of transfer is telling.", for which he provides precisely zero evidence, he ignores the fact that AG training works fine on a mixture of board sizes and such progressive growing/curriculum training in fact seems to accelerate training (https://arxiv.org/abs/1902.10565 https://arxiv.org/abs/1902.10565) and that rectangular convolutions are a thing that exist, and there would be plenty of transfer if anyone tried. If no one has tried that exact thing, it's because it would be pointless and it's obvious to everyone not named 'Marcus' that it'd work fine, being rectangular doesn't mystically stop it from working anymore than using a 13x13 rather than 19x19 board makes AG-style training stop working. (This is not the first time Marcus has claimed that something didn't work; I first realized that he doesn't actually keep up with the AI literature when I pointed out to him on Twitter that plenty of knowledge graphs were in use combined with DL, and Google Search was the biggest example of this, and he had no idea what I was talking about.) No matter what DL or DRL does, or how little his own preferred paradigm does, Marcus will never ever admit anything. AlphaGo beats humans? Well, it just copied humans. AlphaZero learns from scratch? Well, he wrote a whole paper explaining how akshully it still copies humans because the tree search encodes the rules. MuZero throws out even the tree search's knowledge of rules? Crickets and essays about 'misinformation'.
- YeGoblynQueenne 7y ago
  Gary Marcus' tone amounts to base trolling and it's a shame that's how he chooses to carry out his criticism. I understand your frustration. Regarding the lack of transfer, yes, AlphaGo, AlphaZero and most of their variants have boards of fixed size and shape hard-coded in their architecture (as they have the types of piece moves-hard coded) and need architectural modifications and re-training before they can play on different boards or with different pieces (e.g. AlphaGo can't play Chess and Shoggi unmodified). The KataGo paper (the paper you linked) is one exception to this. Personally, I don't know others. Anyway general game-playing is a hard task and nobody claims it's solved by AlphaGo. Regarding KataGo its main contribution is a significant reduction to the cost of training an AlpahGo variant while maintaining a competitive performance. This is very promising- after DeepBlue, creating a chess engine became cheaper and cheaper until they could run on a smartphone. We are far from that with Go computer players. However, in the KataGo paper, major gains are claimed to come from a) game-playing specific or MCTS-specific improvements (playout cap randomisation, forced playouts and policy target pruning) or architecture-specific improvements (global pooling) or, b) domain-specific improvements (auxiliary ownership and score targets). Finally, KataGo has a few game-specific features (liberties, pass-alive regions and ladder features). The KataGo paper itself says it very clearly. I quote, snipping for brevity: Second, our work serves as a case study that there is still a significant efficiency gap between AlphaZero's methods and what is possible from self-play. We find nontrivial further gains from some domain-specific methods (...) We also find that a set of standard game-specific input features still significantly accelerates learning, showing that AlphaZero does not yet obsolete even simple additional tuning. Finally, "it would obviously work so nobody tried" would make sense if it wasn't for the extremely competitive nature of machine learning research where every novel result is presented as a big breakthrough. Also, if something is obvious but never seems to make it to publication the chances are someone has tried and it didn't work as expected so they shelved the paper. We all know what happens to negative results in machine learning.

Barrin92 7y ago

>Ugh. I dislike the author s permanent negativity, but he s right about a lot. Yes, in a way it must suck to always act like the grumpy party-crasher but Gary Marcus is absolutely spot on when it comes to the facts. That interview with the economist had me shaking my head, everyone had to know within five seconds that the chance that this is uncurated is zero.

deleted 7y ago

[deleted]

Fishysoup 7y ago

It's the bullshit process by which people get funding. Make something that has some potentially interesting engineering value, hype it up to hell and back, and funding agencies, investors etc. are more likely to back you. Probably this wasn't everyone's first plan all along, but when people who make a marginally better hot-dog / not-hot-dog classifier put so much spin on it, then everyone else has to just to remain visible. Moreover, people have to publish findings before their competition does, meaning sloppier and less interesting work. It's the snake-eating-its-own-tail plague that impacts so much of academia.

buboard 7y ago
> impacts so much of academia i think the trend started from the industry , but you re right warrantless self-promotion very pervasive in academia , and it's sad that it works!
unishark 7y ago
I'd say its the job of academic researchers to dream up high-risk ideas to work on, and to hype up incremental advances as potentially proof-of-concept for those ideas (when they're decades away still, with many potentially unsolvable hurdles left). Generally they are careful to give all sides though, since their reputation among their peers is the basis for their career. The media is a very different story. They just need to catch the interest of the mostly-ignorant, then move on to something new the next story.

ACow_Adonis 7y ago

I dislike how, when dealing with a general industry of spruikers and boosters, speaking the truth is perceived as permanent negativity :P He's not negative in this article. He's right.

6gvONxR4sf7o 7y ago

> I think it’s worth asking why people feel the need to lie about the future of AI? I wonder if it's because it's so vague and fuzzy, or because the techniques are so general. Like with self driving cars. The will someday probably be safer than people and that's huge. People get excited. We want that! Self driving cars save lives! But in those four sentences we went from "will someday probably" to let's do it now. The class of thing we're talking about now and the class of thing in the future are one and the same, so it's hard to talk about the future versions as distinctly separate from today's. AI will probably be able to talk to people well. We have AI today that talks to people. They're not the same thing, but these two sentences don't make that clear, because it's all AI. In principle, polynomials can learn any function! And we have polynomials today! We can learn anything! Rinse and repeat with fourier series or (as a totally random example) deep learning and it sounds like tomorrow's techniques are the same as today's, so we're done, right? Or maybe it's on lay people's poor math and stats skills and lack of understanding of the simple stuff. If I tell lay people I do stats, they think I'm taking an average with a lot of bureaucracy they don't really get. They won't think I'm using simple logistic regression to do really cool stuff like classify documents. They didn't know "stats" could do that! So they might be even more misled about what I do if I call it "statistics" than if I call it "AI." If they're mislead whatever I call it, we're already screwed.

gwern 7y ago

He's also wrong about a lot. For all his insistence on accuracy, he himself is misleading or ignorant. Like his slam of the Dartmouth project - even if you knew nothing about it, all you have to do is click through to see the claims of 'solving vision' are sheer projection and urban legend. And he's happy to make up claims out of whole cloth: for example, when he says "AlphaGo works fine on a 19x19 board, but would need to be retrained to play on a rectangular board; the lack of transfer is telling.", for which he provides precisely zero evidence, he ignores the fact that AG training works fine on a mixture of board sizes and such progressive growing/curriculum training in fact seems to accelerate training (https://arxiv.org/abs/1902.10565 https://arxiv.org/abs/1902.10565) and that rectangular convolutions are a thing that exist, and there would be plenty of transfer if anyone tried. If no one has tried that exact thing, it's because it would be pointless and it's obvious to everyone not named 'Marcus' that it'd work fine, being rectangular doesn't mystically stop it from working anymore than using a 13x13 rather than 19x19 board makes AG-style training stop working. (This is not the first time Marcus has claimed that something didn't work; I first realized that he doesn't actually keep up with the AI literature when I pointed out to him on Twitter that plenty of knowledge graphs were in use combined with DL, and Google Search was the biggest example of this, and he had no idea what I was talking about.) No matter what DL or DRL does, or how little his own preferred paradigm does, Marcus will never ever admit anything. AlphaGo beats humans? Well, it just copied humans. AlphaZero learns from scratch? Well, he wrote a whole paper explaining how akshully it still copies humans because the tree search encodes the rules. MuZero throws out even the tree search's knowledge of rules? Crickets and essays about 'misinformation'.

YeGoblynQueenne 7y ago
Gary Marcus' tone amounts to base trolling and it's a shame that's how he chooses to carry out his criticism. I understand your frustration. Regarding the lack of transfer, yes, AlphaGo, AlphaZero and most of their variants have boards of fixed size and shape hard-coded in their architecture (as they have the types of piece moves-hard coded) and need architectural modifications and re-training before they can play on different boards or with different pieces (e.g. AlphaGo can't play Chess and Shoggi unmodified). The KataGo paper (the paper you linked) is one exception to this. Personally, I don't know others. Anyway general game-playing is a hard task and nobody claims it's solved by AlphaGo. Regarding KataGo its main contribution is a significant reduction to the cost of training an AlpahGo variant while maintaining a competitive performance. This is very promising- after DeepBlue, creating a chess engine became cheaper and cheaper until they could run on a smartphone. We are far from that with Go computer players. However, in the KataGo paper, major gains are claimed to come from a) game-playing specific or MCTS-specific improvements (playout cap randomisation, forced playouts and policy target pruning) or architecture-specific improvements (global pooling) or, b) domain-specific improvements (auxiliary ownership and score targets). Finally, KataGo has a few game-specific features (liberties, pass-alive regions and ladder features). The KataGo paper itself says it very clearly. I quote, snipping for brevity: Second, our work serves as a case study that there is still a significant efficiency gap between AlphaZero's methods and what is possible from self-play. We find nontrivial further gains from some domain-specific methods (...) We also find that a set of standard game-specific input features still significantly accelerates learning, showing that AlphaZero does not yet obsolete even simple additional tuning. Finally, "it would obviously work so nobody tried" would make sense if it wasn't for the extremely competitive nature of machine learning research where every novel result is presented as a big breakthrough. Also, if something is obvious but never seems to make it to publication the chances are someone has tried and it didn't work as expected so they shelved the paper. We all know what happens to negative results in machine learning.