5 ms·

It's hard to say how much is this about scaring the government into action (AI-McCarthyism), or about being jealous that China is not constrained like Western c

It's hard to say how much is this about scaring the government into action (AI-McCarthyism), or about being jealous that China is not constrained like Western countries are when experimenting with AI because of ingrained in society respect towards privacy, human rights, having to listen to the public opinion (democracy), etc. In another HN thread about AI, big corporations and China, someone mentioned that big American IT corporations are very active in China, offering their tech for pennies just to be able to test it on a wider scale, because in the West they can't do it (regulations, human rights, etc.). There are many problems unsolved, where a lot of grunt work is needed and little progress has been made in the last decades - healthcare is expensive, many diseases are still uncurable, production is still cheaper to outsource to other countries than do it locally because automation (not just AI, also sensors, computing power, materials, etc.) are not there yet. I think the general trend to see current AI iteration as the magical solutions to all problems is dangerous, as usual the reality is much more complex and should be tackled in a more thorough manner. Why not pay AI-specialist pay to get the best people into healthcare, new material research, etc?

seanmcdirmid 9y ago
Ya, it sounds like a scare tactic. China does have advantages, mainly, they have lots of data with less concern for privacy. The Chinese are making huge bets on AI, but they are beginning to sound like Japan in the 1980s with their Fifth generation computing project. Actually, there are so many analogues of China today to Japan in the 1980s (huge housing bubble, booming economy that doesn't seem to stop, a huge bet on AI....) that it doesn't really seem like a coincidence.
- Barrin92 9y ago
  I don't think the comparison to 'neo tokyo' fantasies is warranted. When people were scared of Japan taking over the world with robots, those fifth generation projects were moonshot technologies without any clear commercial application. The difference is that ML today (at least the kind of technologies where China has and is gaining an edge) are deeply embedded in the commercial sector and public security. We're not talking about China building a superintelligence in some kind of basement of a research facility, we're talking about facial recognition in public spaces, consumer products that get data from a billion customers who all pay and live on a centralised platform and so on. That's a reality already and it's not a stretch to say that China's environment is more suited for this than any other place on the planet.
  - seanmcdirmid 9y ago
    China has no technological edge in ML yet. Even the most ardent supporter talks about how China will have edges in 2020, but not today. It is still very aspirational. I never said anything about neo tokyo, the fifth generation computing project was never vivid in our imaginations, all most of of us remember about it is the government throwing a bunch of money at AI research and hardware, and that it led directly to the 2nd AI winter. It also correlates nicely with other economic excesses (like China's problematic property bubble). China still has a problem with attracting world-class talent, mainly because of their environment (great firewall, still way too wicked pollution). There are more Chinese working on ML in Silicon Valley then probably all foreigners working in tech in China. So if ML is a talent game, then I don't see how China will be able to win that in the near term, even with all their local human resources.
    - Barrin92 9y ago
      I don't believe that ML is a talent game at all. I think ML is a data and policy game. There's a lot of talk about AI braininess but from a theoretical standpoint there isn't much new to the currently commercially used ML products. The only thing that's changed over the last five years is data. Backpropagation is decades old. And on the data and policy side China has hugely favourable conditions. I think we're making a great mistake if we're over-hyping the role of talent.
      - candiodari 9y ago
        I disagree. Time and time again data advantages and computing power advantages are clobbered by algorithm advances. For instance, there is no amount of cluster power that'll let you best resnet with a non-residual network. Well perhaps there is, but even Google's mighty clusters can't make up the difference. In audio, end-to-end models that take 2 weeks to train on a dual GTX 1080 machine, and take > 1 week to train on an entire datacenter beat everything we've ever come up with in linguistics. And it was recently shown that taking the convolutions and redoing in hand-coded assembly gains you more performance than the quite-meager performance a cluster gets you. The same goes for data. We can outperform VGGNet trained on all of imagenet (~ 15 million images) using resnet trained on a 1% random sampling of imagenet. We have papers putting forth statistics on specific model architectures outperforming with < 0.1% of data of benchmarks and just-barely-worse performance using a decent, but fixed and tiny number of data. I understand why Google and Microsoft are pushing "you can't beat clusters", but anyone fiddling with batch-size on actual models will quickly realize that it's ... let's say "riddled with MAJOR caveats" (using clusters implicitly forces large batch sizes, which costs you a LOT in training efficiency, especially once you get past the first few rounds, and the better your model, the more it costs).
otakucode 9y ago
>Why not pay AI-specialist pay to get the best people into healthcare, new material research, etc? Because that's expensive, and companies are still structured in a manner built and optimized for repetitive manufacturing. They are not built to, and as a result can not, value things like knowledge and skill. Everything from why they have offices, to how those offices are built, to how the organizations are structured is anathema to having a person or persons with valuable domain knowledge as a key component. Read essentially any book on business management written since 1980 up to today. Every single one of them will advise managers to highly prioritize the dispensability of employees and inherently emphasizes accounting ('business') oriented roles over technical ones. Remember, IT is a cost center. Anything spent on it, or those with skills in it, is pure loss that offers no value. You're a healthcare company, not an IT company, and IT people don't even see patients.

seanmcdirmid 9y ago

Ya, it sounds like a scare tactic. China does have advantages, mainly, they have lots of data with less concern for privacy. The Chinese are making huge bets on AI, but they are beginning to sound like Japan in the 1980s with their Fifth generation computing project. Actually, there are so many analogues of China today to Japan in the 1980s (huge housing bubble, booming economy that doesn't seem to stop, a huge bet on AI....) that it doesn't really seem like a coincidence.

Barrin92 9y ago
I don't think the comparison to 'neo tokyo' fantasies is warranted. When people were scared of Japan taking over the world with robots, those fifth generation projects were moonshot technologies without any clear commercial application. The difference is that ML today (at least the kind of technologies where China has and is gaining an edge) are deeply embedded in the commercial sector and public security. We're not talking about China building a superintelligence in some kind of basement of a research facility, we're talking about facial recognition in public spaces, consumer products that get data from a billion customers who all pay and live on a centralised platform and so on. That's a reality already and it's not a stretch to say that China's environment is more suited for this than any other place on the planet.
- seanmcdirmid 9y ago
  China has no technological edge in ML yet. Even the most ardent supporter talks about how China will have edges in 2020, but not today. It is still very aspirational. I never said anything about neo tokyo, the fifth generation computing project was never vivid in our imaginations, all most of of us remember about it is the government throwing a bunch of money at AI research and hardware, and that it led directly to the 2nd AI winter. It also correlates nicely with other economic excesses (like China's problematic property bubble). China still has a problem with attracting world-class talent, mainly because of their environment (great firewall, still way too wicked pollution). There are more Chinese working on ML in Silicon Valley then probably all foreigners working in tech in China. So if ML is a talent game, then I don't see how China will be able to win that in the near term, even with all their local human resources.
  - Barrin92 9y ago
    I don't believe that ML is a talent game at all. I think ML is a data and policy game. There's a lot of talk about AI braininess but from a theoretical standpoint there isn't much new to the currently commercially used ML products. The only thing that's changed over the last five years is data. Backpropagation is decades old. And on the data and policy side China has hugely favourable conditions. I think we're making a great mistake if we're over-hyping the role of talent.
    - candiodari 9y ago
      I disagree. Time and time again data advantages and computing power advantages are clobbered by algorithm advances. For instance, there is no amount of cluster power that'll let you best resnet with a non-residual network. Well perhaps there is, but even Google's mighty clusters can't make up the difference. In audio, end-to-end models that take 2 weeks to train on a dual GTX 1080 machine, and take > 1 week to train on an entire datacenter beat everything we've ever come up with in linguistics. And it was recently shown that taking the convolutions and redoing in hand-coded assembly gains you more performance than the quite-meager performance a cluster gets you. The same goes for data. We can outperform VGGNet trained on all of imagenet (~ 15 million images) using resnet trained on a 1% random sampling of imagenet. We have papers putting forth statistics on specific model architectures outperforming with < 0.1% of data of benchmarks and just-barely-worse performance using a decent, but fixed and tiny number of data. I understand why Google and Microsoft are pushing "you can't beat clusters", but anyone fiddling with batch-size on actual models will quickly realize that it's ... let's say "riddled with MAJOR caveats" (using clusters implicitly forces large batch sizes, which costs you a LOT in training efficiency, especially once you get past the first few rounds, and the better your model, the more it costs).

otakucode 9y ago

>Why not pay AI-specialist pay to get the best people into healthcare, new material research, etc? Because that's expensive, and companies are still structured in a manner built and optimized for repetitive manufacturing. They are not built to, and as a result can not, value things like knowledge and skill. Everything from why they have offices, to how those offices are built, to how the organizations are structured is anathema to having a person or persons with valuable domain knowledge as a key component. Read essentially any book on business management written since 1980 up to today. Every single one of them will advise managers to highly prioritize the dispensability of employees and inherently emphasizes accounting ('business') oriented roles over technical ones. Remember, IT is a cost center. Anything spent on it, or those with skills in it, is pure loss that offers no value. You're a healthcare company, not an IT company, and IT people don't even see patients.