4 ms·

What do you think about this? (from part of my other comment) * 9 out of the top 10 algorithms for the Stanford Question Answering Dataset are by Chinese teams

What do you think about this? (from part of my other comment) * 9 out of the top 10 algorithms for the Stanford Question Answering Dataset are by Chinese teams, backed by HIT & iFlLYTEK, Alibaba, Microsoft Research Asia (in Beijing), NUDT & Fudan University, and others. They beat out teams by Samsung, Salesforce, CMU, and Facebook, for example. https://rajpurkar.github.io/SQuAD-explorer/ https://rajpurkar.github.io/SQuAD-explorer/ It is not like others do not try. This is a highly commercializable area of research and there is no requirement to publish the technical details of the submitted solution. * Based on publications from 2015 onward, Peking and Tsinghua University are ranked in the world's top 5 for published papers in top AI conferences (at #3 and #5). http://csrankings.org/#/fromyear/2015/toyear/2018/index?ai&vision&mlmining&nlp&world http://csrankings.org/#/fromyear/2015/toyear/2018/index?ai&v... See my previous comment for more info & counterpoints about talent and pollution.

Eridrus 9y ago
It's basically the same in computer security where you now see most vulnerabilities reported to vendors are from China. I wouldn't necessarily say China has "beaten" the rest of the world, but rather people in the west have basically decided that form of signaling is too expensive. There's a lot of people publishing ML work on public datasets which barely move the needle. It's usually not too hard to squeeze something extra into a model to do better than the previous work, and if not there are always other datasets, so these raw numbers don't really means lot IMO. Not to say China won't overtake the rest of the world, eg lots of great work coming out of MSRA, but I think that there is a lot of truth to the idea that once you start measuring something, people start gaming it, and I think the political environment in China encourages this gaming, so I would pay less attention to metrics like this that can be gamed. Even paper metrics are gameable. Most reviewers acknowledge that once you remove the obvious accepts and obvious rejects you've got a large pile of passable but forgettable work. So if you can just keep cranking out passable work that fits the zeitgeist, you'll get more papers accepted. I might be wildly off base here, I haven't tried to construct statistics, and academia has a large amount of starfucking going on, so it may be a cultural bias thing, but I don't get the impression that China is anywhere near dominant yet, but yet all these statistics look great!
- nopinsight 9y ago
  Thanks for your thoughtful reply! I think everyone, including the Chinese, agrees that they are not (yet) dominant in AI, just that they are catching up very fast to the best in the world. They might even beat all others in optimizing in existing paradigms. Part of their 2030 plan to become preeminent in AI is to develop indigenous breakthroughs by 2025 and be equal to the leading nation by 2020. They are very much on track to achieve the closest goal given evidence to date.
seanmcdirmid 9y ago
Just to be clear, not all researchers in Microsoft Asia are Chinese (I wasn't, anyways). One of the best ML guys I knew at MSRA was actually European (he started working on GPU-based deep learning way early in 2010 when we were still at Sigma, he works in Redmond now but was a long timer like I was but had to leave because of the pollution, which is why most laowai left). If China could attract more of that world talent, they might be able to do it, but otherwise, its like a talent pool of 1.4 billion competing with a talent pool of 7.4 billion (many of the ML researchers in the states aren't American either). With that out of the way, all of those are not very relevant. Its like judging the quality of programmers by ACM programming contest wins, or by judging hardware advancement on super computing benchmarks that no one really cares about anymore. China has declared that they will be the winner in AI in the future. But they definitely aren't today, and that assumes that everyone else stays absolutely where they are already.
- nopinsight 9y ago
  I agree China does not yet surpass all other nations regarding creating innovations in AI, although they have come close to the top of the league. As shown by PISA and other international tests, the percentage of AI-capable people varies wildly among the populations in that 7.4 billion global pool. China performs very well, esp relative to their development level, and above most OECD countries. http://ncee.org/2016/12/pisa-2015-results/ http://ncee.org/2016/12/pisa-2015-results/ If China can hold onto most of its talent, it would be a relatively large percentage (say 4-5%) of 1.4 billion vs a smaller percentage (2.0-2.5%) of 1.3 billion (OECD country populations) and a very small percentage (say 0.2-0.5%) from 4.9 billion (the rest of the world) who can potentially work productively in AI. This assumes the Gaussian distribution of education quality, skills, and perserverance and a fairly high bar (2+ SD above OECD average) for a lower threshold to work competently in AI. It is obvious that not all talent in the latter two would like to immigrate to the US, so it is unclear if the US will continue to have a quantitative advantage despite a more global talent pool. Chinese government's deliberate policy in nurturing and developing AI talent will also add to the number of local practitioners. There is also clear evidence that more Chinese nationals are moving back home after graduating from abroad. The rough estimates above are relevant for AI optimizations and applications in many real-world settings since a sufficient quantity of R&D practitioners is necessary for that. For breakthroughs, other factors might come into play and it is not clear if cultural factors and upbringing would affect the potentials of people to create paradigm-shifting ideas, which means quantitative analysis in isolation is not nearly as predictive.

Eridrus 9y ago

It's basically the same in computer security where you now see most vulnerabilities reported to vendors are from China. I wouldn't necessarily say China has "beaten" the rest of the world, but rather people in the west have basically decided that form of signaling is too expensive. There's a lot of people publishing ML work on public datasets which barely move the needle. It's usually not too hard to squeeze something extra into a model to do better than the previous work, and if not there are always other datasets, so these raw numbers don't really means lot IMO. Not to say China won't overtake the rest of the world, eg lots of great work coming out of MSRA, but I think that there is a lot of truth to the idea that once you start measuring something, people start gaming it, and I think the political environment in China encourages this gaming, so I would pay less attention to metrics like this that can be gamed. Even paper metrics are gameable. Most reviewers acknowledge that once you remove the obvious accepts and obvious rejects you've got a large pile of passable but forgettable work. So if you can just keep cranking out passable work that fits the zeitgeist, you'll get more papers accepted. I might be wildly off base here, I haven't tried to construct statistics, and academia has a large amount of starfucking going on, so it may be a cultural bias thing, but I don't get the impression that China is anywhere near dominant yet, but yet all these statistics look great!

nopinsight 9y ago
Thanks for your thoughtful reply! I think everyone, including the Chinese, agrees that they are not (yet) dominant in AI, just that they are catching up very fast to the best in the world. They might even beat all others in optimizing in existing paradigms. Part of their 2030 plan to become preeminent in AI is to develop indigenous breakthroughs by 2025 and be equal to the leading nation by 2020. They are very much on track to achieve the closest goal given evidence to date.

seanmcdirmid 9y ago

Just to be clear, not all researchers in Microsoft Asia are Chinese (I wasn't, anyways). One of the best ML guys I knew at MSRA was actually European (he started working on GPU-based deep learning way early in 2010 when we were still at Sigma, he works in Redmond now but was a long timer like I was but had to leave because of the pollution, which is why most laowai left). If China could attract more of that world talent, they might be able to do it, but otherwise, its like a talent pool of 1.4 billion competing with a talent pool of 7.4 billion (many of the ML researchers in the states aren't American either). With that out of the way, all of those are not very relevant. Its like judging the quality of programmers by ACM programming contest wins, or by judging hardware advancement on super computing benchmarks that no one really cares about anymore. China has declared that they will be the winner in AI in the future. But they definitely aren't today, and that assumes that everyone else stays absolutely where they are already.

nopinsight 9y ago
I agree China does not yet surpass all other nations regarding creating innovations in AI, although they have come close to the top of the league. As shown by PISA and other international tests, the percentage of AI-capable people varies wildly among the populations in that 7.4 billion global pool. China performs very well, esp relative to their development level, and above most OECD countries. http://ncee.org/2016/12/pisa-2015-results/ http://ncee.org/2016/12/pisa-2015-results/ If China can hold onto most of its talent, it would be a relatively large percentage (say 4-5%) of 1.4 billion vs a smaller percentage (2.0-2.5%) of 1.3 billion (OECD country populations) and a very small percentage (say 0.2-0.5%) from 4.9 billion (the rest of the world) who can potentially work productively in AI. This assumes the Gaussian distribution of education quality, skills, and perserverance and a fairly high bar (2+ SD above OECD average) for a lower threshold to work competently in AI. It is obvious that not all talent in the latter two would like to immigrate to the US, so it is unclear if the US will continue to have a quantitative advantage despite a more global talent pool. Chinese government's deliberate policy in nurturing and developing AI talent will also add to the number of local practitioners. There is also clear evidence that more Chinese nationals are moving back home after graduating from abroad. The rough estimates above are relevant for AI optimizations and applications in many real-world settings since a sufficient quantity of R&D practitioners is necessary for that. For breakthroughs, other factors might come into play and it is not clear if cultural factors and upbringing would affect the potentials of people to create paradigm-shifting ideas, which means quantitative analysis in isolation is not nearly as predictive.