4 ms·

It is noticeable that all of the top 4 results in the recent Stanford Conversational Question Answering (CoQA) challenge are from groups based in China (the top

It is noticeable that all of the top 4 results in the recent Stanford Conversational Question Answering (CoQA) challenge are from groups based in China (the top 2 are from Microsoft Research Asia; the 3th and 4th, Sogou and Fudan; the 5th, anonymous): https://stanfordnlp.github.io/coqa/ https://stanfordnlp.github.io/coqa/ Chinese universities are ranked at 1st, 3th, and 4th in the world for papers published in top AI conferences in 2018. (Tsinghua at the top; followed by CMU, Beijing, CAS, and Stanford) Although some people might disagree with the methodology (e.g. larger departments have an advantage), it is still a decent indicator for China’s progress in the field. http://csrankings.org/#/fromyear/2018/toyear/2018/index?ai&vision&mlmining&nlp&ir&world http://csrankings.org/#/fromyear/2018/toyear/2018/index?ai&v... Perhaps it is time for the US as a nation to spend more resources to develop and attract more people into AI research. See also an Economist’s review of a recent book: “In the struggle for AI supremacy, China will prevail; Or so reckons Kai-Fu Lee, a tech insider, in “AI Superpowers”” https://www.economist.com/books-and-arts/2018/09/27/in-the-struggle-for-ai-supremacy-china-will-prevail https://www.economist.com/books-and-arts/2018/09/27/in-the-s...

theblackcat1002 8y ago
FYI, the top 6 results are mainly based on BERT which is proposed by Google[1]. [1] https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805
- nopinsight 8y ago
  Yes, BERT is a significant advance. I think most would agree that many key advances still originate from the US. (And the US is probably still ahead overall in research.) China is apparently faster at adapting and deploying these changes in some areas and they are very focused on excelling at the technology, which has many major implications on society.
antpls 8y ago
Those rankings based on countries don't say much IMHO. It depends on : - the score. If the top 5 achieves 85/100 and the the rest of the top 20 achieves 83/100, they are all on par. - the field. If it's academic with open source code and scientific results shared with everyone on Earth, who cares who is in the top 5? It's a global effort
- nopinsight 8y ago
  Actual figures for CoQA from the link above: No.1 is 86.8%; no.5, 81.4%; no.20, 66.5%. No.5, though quite impressive (humans are at 88.8%), still give about two-fifth more errors than no.1. These technologies tend toward natural monopolies as users tend to choose among the top 3-4 providers which allow them to spread costs, lower prices, and acquire yet more data. The winners at the right time tend to hold long-term monopolistic/oligopolistic power, until a major breakthrough happens. Also, all results in the top 20 from the same link are from either the US or China (except for a couple of anonymous results). At least in this subfield, there could be only two national contenders, despite many key papers being open. Exact algorithms, implementation details, and system engineering and integration count.
1_over_n 8y ago
Yesterday i was researching IP strategies for AI companies and came across this paper from a legal firm. https://www.bereskinparr.com/files/file/IAM88_AI-and-IP_Jim%20Hinton.pdf https://www.bereskinparr.com/files/file/IAM88_AI-and-IP_Jim%... It blew my mind how much China is filing patents around AI compared to the rest of the world. It should also be considered that from an IP perspective American companies could be going down the trade secret route and academia is open sourcing. I don't have any data on this. I know in the USA trade secrets do now have some legal protection. https://www.uspto.gov/patents-getting-started/international-protection/trade-secrets-policy https://www.uspto.gov/patents-getting-started/international-...
- speedplane 8y ago
  China has been filing tons of patents because of the national prestige it brings. A large percentage are of dubious quality and they are setting themselves up for a huge patent troll problem 10 years from now, much like the US had as a result of bogus late 1990/early 2000s patents. The US patent system has become much better (although Europe is still even better). They now reject over 90% of business method patents (i.e., doing business with a computer). Many of those would be granted in China. The number of granted patents in any given country is not a good indicator of technical progress.
  - 1_over_n 8y ago
    So the question becomes - will China respect the USA's patents, will the USA respect Chinese patents?
    - speedplane 8y ago
      There are well established treaties for this. No country's patent office is forced to grant a patent filed in another country. For example, if you get a patent granted in the U.S., there's no guarantee it'll be granted in Britain. If you want a patent in both countries, you'll need to file a separate patent application in both countries. The same goes for the U.S. and China. If a U.S. company only has U.S. patents, companies operating in China can make a product that directly infringes it. That said, the second that product is imported into the U.S., the U.S. patent will kick in, and you can get a judge to block it at the border. Going back to the AI space, the fact that China has a ton of AI patents in their country won't stop Google from developing the same in the U.S. However, if Google tried to offer those infringing services to Chinese residents, they could be sued/stopped in China.

theblackcat1002 8y ago

FYI, the top 6 results are mainly based on BERT which is proposed by Google[1]. [1] https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805

nopinsight 8y ago
Yes, BERT is a significant advance. I think most would agree that many key advances still originate from the US. (And the US is probably still ahead overall in research.) China is apparently faster at adapting and deploying these changes in some areas and they are very focused on excelling at the technology, which has many major implications on society.

antpls 8y ago

Those rankings based on countries don't say much IMHO. It depends on : - the score. If the top 5 achieves 85/100 and the the rest of the top 20 achieves 83/100, they are all on par. - the field. If it's academic with open source code and scientific results shared with everyone on Earth, who cares who is in the top 5? It's a global effort

nopinsight 8y ago
Actual figures for CoQA from the link above: No.1 is 86.8%; no.5, 81.4%; no.20, 66.5%. No.5, though quite impressive (humans are at 88.8%), still give about two-fifth more errors than no.1. These technologies tend toward natural monopolies as users tend to choose among the top 3-4 providers which allow them to spread costs, lower prices, and acquire yet more data. The winners at the right time tend to hold long-term monopolistic/oligopolistic power, until a major breakthrough happens. Also, all results in the top 20 from the same link are from either the US or China (except for a couple of anonymous results). At least in this subfield, there could be only two national contenders, despite many key papers being open. Exact algorithms, implementation details, and system engineering and integration count.

1_over_n 8y ago

Yesterday i was researching IP strategies for AI companies and came across this paper from a legal firm. https://www.bereskinparr.com/files/file/IAM88_AI-and-IP_Jim%20Hinton.pdf https://www.bereskinparr.com/files/file/IAM88_AI-and-IP_Jim%... It blew my mind how much China is filing patents around AI compared to the rest of the world. It should also be considered that from an IP perspective American companies could be going down the trade secret route and academia is open sourcing. I don't have any data on this. I know in the USA trade secrets do now have some legal protection. https://www.uspto.gov/patents-getting-started/international-protection/trade-secrets-policy https://www.uspto.gov/patents-getting-started/international-...

speedplane 8y ago
China has been filing tons of patents because of the national prestige it brings. A large percentage are of dubious quality and they are setting themselves up for a huge patent troll problem 10 years from now, much like the US had as a result of bogus late 1990/early 2000s patents. The US patent system has become much better (although Europe is still even better). They now reject over 90% of business method patents (i.e., doing business with a computer). Many of those would be granted in China. The number of granted patents in any given country is not a good indicator of technical progress.
- 1_over_n 8y ago
  So the question becomes - will China respect the USA's patents, will the USA respect Chinese patents?
  - speedplane 8y ago
    There are well established treaties for this. No country's patent office is forced to grant a patent filed in another country. For example, if you get a patent granted in the U.S., there's no guarantee it'll be granted in Britain. If you want a patent in both countries, you'll need to file a separate patent application in both countries. The same goes for the U.S. and China. If a U.S. company only has U.S. patents, companies operating in China can make a product that directly infringes it. That said, the second that product is imported into the U.S., the U.S. patent will kick in, and you can get a judge to block it at the border. Going back to the AI space, the fact that China has a ton of AI patents in their country won't stop Google from developing the same in the U.S. However, if Google tried to offer those infringing services to Chinese residents, they could be sued/stopped in China.