4 ms·
I don't think data is as important as they make it to be. Yes, it was important at some point when they scaled supervised learning from thousands to millions or
by visarga 10y ago
I don't think data is as important as they make it to be. Yes, it was important at some point when they scaled supervised learning from thousands to millions or billions of examples. But after that, did they continue scaling up to trillions of examples? No. Because there are diminishing returns. Maybe the model can't handle that much data, or the accuracy doesn't increase as much, or the data is just too big for the current computing power. The kind of data that is used today even at Google scale is just millions or billions of examples.
On the other hand, what kind of data could be so secret that only the big companies could accumulate? If you have access to a service, you can mine it and construct a dataset. Then you can train your own model to imitate their results. If this process is done right, it becomes easy to do transfer learning from other AI systems, even if they keep their algorithms secret.
For example, Google's latest POS tagger SyntaxNet, which made a splash a few days ago, was trained on the results of the Stanford parser. Interestingly, the student model surpassed the teacher model - probably because it was better at cancelling out errors in the training set and generalizing better from examples.
If data is the end game of these companies, then it will be a target of espionage, leaks and public disclosures. A hard drive containing the prized dataset of some company can be copied in a few minutes and if one copy escapes, then it is circulated and becomes public domain. So it's hard to protect data. Data likes to flow (remember the Ok Cupid dataset - that was crawled without permission from the behind the login wall?). Flowing data can be turned back into machine learning models.
- deleted 10y ago[deleted]
- adrianratnapala 10y agoI assume people claim IP protection of one kind or another on these datasets, and even if their claims were spurious, they would still make trouble. That's not going to stop some hacker could crawl OkCupid and balckmail some people. But if Google or Microsoft suspect FooStartup Inc. is using one of their datasets in its service, then the lawyers will descend.