5 ms·
I don't think the comparison to 'neo tokyo' fantasies is warranted. When people were scared of Japan taking over the world with robots, those fifth generation p
by Barrin92 9y ago
I don't think the comparison to 'neo tokyo' fantasies is warranted. When people were scared of Japan taking over the world with robots, those fifth generation projects were moonshot technologies without any clear commercial application.
The difference is that ML today (at least the kind of technologies where China has and is gaining an edge) are deeply embedded in the commercial sector and public security.
We're not talking about China building a superintelligence in some kind of basement of a research facility, we're talking about facial recognition in public spaces, consumer products that get data from a billion customers who all pay and live on a centralised platform and so on. That's a reality already and it's not a stretch to say that China's environment is more suited for this than any other place on the planet.
- seanmcdirmid 9y agoChina has no technological edge in ML yet. Even the most ardent supporter talks about how China will have edges in 2020, but not today. It is still very aspirational. I never said anything about neo tokyo, the fifth generation computing project was never vivid in our imaginations, all most of of us remember about it is the government throwing a bunch of money at AI research and hardware, and that it led directly to the 2nd AI winter. It also correlates nicely with other economic excesses (like China's problematic property bubble). China still has a problem with attracting world-class talent, mainly because of their environment (great firewall, still way too wicked pollution). There are more Chinese working on ML in Silicon Valley then probably all foreigners working in tech in China. So if ML is a talent game, then I don't see how China will be able to win that in the near term, even with all their local human resources.
- Barrin92 9y agoI don't believe that ML is a talent game at all. I think ML is a data and policy game. There's a lot of talk about AI braininess but from a theoretical standpoint there isn't much new to the currently commercially used ML products. The only thing that's changed over the last five years is data. Backpropagation is decades old. And on the data and policy side China has hugely favourable conditions. I think we're making a great mistake if we're over-hyping the role of talent.
- candiodari 9y agoI disagree. Time and time again data advantages and computing power advantages are clobbered by algorithm advances. For instance, there is no amount of cluster power that'll let you best resnet with a non-residual network. Well perhaps there is, but even Google's mighty clusters can't make up the difference. In audio, end-to-end models that take 2 weeks to train on a dual GTX 1080 machine, and take > 1 week to train on an entire datacenter beat everything we've ever come up with in linguistics. And it was recently shown that taking the convolutions and redoing in hand-coded assembly gains you more performance than the quite-meager performance a cluster gets you. The same goes for data. We can outperform VGGNet trained on all of imagenet (~ 15 million images) using resnet trained on a 1% random sampling of imagenet. We have papers putting forth statistics on specific model architectures outperforming with < 0.1% of data of benchmarks and just-barely-worse performance using a decent, but fixed and tiny number of data. I understand why Google and Microsoft are pushing "you can't beat clusters", but anyone fiddling with batch-size on actual models will quickly realize that it's ... let's say "riddled with MAJOR caveats" (using clusters implicitly forces large batch sizes, which costs you a LOT in training efficiency, especially once you get past the first few rounds, and the better your model, the more it costs).
- Barrin92 9y ago>The same goes for data. We can outperform VGGNet trained on all of imagenet (~ 15 million images) using resnet trained on a 1% random sampling of imagenet. We have papers putting forth statistics on specific model architectures outperforming with < 0.1% of data of benchmarks and just-barely-worse performance using a decent, but fixed and tiny number of data. okay granted, but isn't this largely part of the academic field, openly accessible and transferable? People at Baidu and Tencent are likely up to date on the state of the art. Contrast this for example with the American military edge or the economics of the German Mittelstand. There's a degree of ingrained knowledge that gives countries decades of advance, but academic research can often be transferred. Understanding in ML to me seems to fall mostly in the second category, and when that category dominates then data and policy really does become relevant. I'm not sure we have seen an ML intellectual arms race yet. For the most part techniques seem to spread fast and as a result data functions like oil of sorts and gives the biggest players the largest advantage.
- nopinsight 9y agoWhat do you think about this? (from part of my other comment) * 9 out of the top 10 algorithms for the Stanford Question Answering Dataset are by Chinese teams, backed by HIT & iFlLYTEK, Alibaba, Microsoft Research Asia (in Beijing), NUDT & Fudan University, and others. They beat out teams by Samsung, Salesforce, CMU, and Facebook, for example. https://rajpurkar.github.io/SQuAD-explorer/ https://rajpurkar.github.io/SQuAD-explorer/ It is not like others do not try. This is a highly commercializable area of research and there is no requirement to publish the technical details of the submitted solution. * Based on publications from 2015 onward, Peking and Tsinghua University are ranked in the world's top 5 for published papers in top AI conferences (at #3 and #5). http://csrankings.org/#/fromyear/2015/toyear/2018/index?ai&vision&mlmining&nlp&world http://csrankings.org/#/fromyear/2015/toyear/2018/index?ai&v... See my previous comment for more info & counterpoints about talent and pollution.
- Eridrus 9y agoIt's basically the same in computer security where you now see most vulnerabilities reported to vendors are from China. I wouldn't necessarily say China has "beaten" the rest of the world, but rather people in the west have basically decided that form of signaling is too expensive. There's a lot of people publishing ML work on public datasets which barely move the needle. It's usually not too hard to squeeze something extra into a model to do better than the previous work, and if not there are always other datasets, so these raw numbers don't really means lot IMO. Not to say China won't overtake the rest of the world, eg lots of great work coming out of MSRA, but I think that there is a lot of truth to the idea that once you start measuring something, people start gaming it, and I think the political environment in China encourages this gaming, so I would pay less attention to metrics like this that can be gamed. Even paper metrics are gameable. Most reviewers acknowledge that once you remove the obvious accepts and obvious rejects you've got a large pile of passable but forgettable work. So if you can just keep cranking out passable work that fits the zeitgeist, you'll get more papers accepted. I might be wildly off base here, I haven't tried to construct statistics, and academia has a large amount of starfucking going on, so it may be a cultural bias thing, but I don't get the impression that China is anywhere near dominant yet, but yet all these statistics look great!
- jcranmer 9y agoThe fifth generation computer was predicated on logic programming being the next big thing. At the time, it wasn't a clearly wrong bet: the AI community was very big on logic programming, and the trend in programming languages was very much from low-level programming to high-level abstractions. The project was announced before the AI winter, after all. What happened was that advances in scalar processors completely blew the specialized processors out of the water, to the point that general-purpose hardware could implement the desired programming features faster than the dedicated hardware. On top of that, the expert system programming model turned out to be a massive bust. In effect, Japan was suddenly faced with expertise building machines that solved problems no one wanted to solve, and were inferior even at that. I think that the same risk is equally valid today: it's possible to so focus on one particular aspect of AI that, if the programming model ends up being revolutionized, you're left with useless hardware.