3 ms·
> More data is necessary with current technoloogy, in the sense that modern statistical machine learning algorithms are very bad at generalising to unseen data,
by darawk 8y ago
> More data is necessary with current technoloogy, in the sense that modern statistical machine learning algorithms are very bad at generalising to unseen data, and the only way to overcome this is to give them more examples.
Precisely.
> There are machine learning techniques that generalise well from few data, but they are not very well known in the industry.
Sure, and we'd all love to be using those. But even if you generalize well from small datasets, you still generalize better from larger ones.
> Also, though more speculatively, I think the idea of "lots of data" is attractive to marketing departments. There's something about algorithms that need huge amounts of data and compute, that only a select few companies can use. I guess it gives bragging rights, of a sort: "we got the biggest data around. Buy our stuff!".
It may be attractive to marketing departments, but it is also essential to data science projects like this.
- YeGoblynQueenne 8y ago>> Sure, and we'd all love to be using those. But even if you generalize well from small datasets, you still generalize better from larger ones. Not to my knowledge. What techniques did you have in mind that work like that?
- darawk 8y agoLiterally all of them? Linear regression, neural networks, KNN, I could just enumerate all ML methods here, but I think the foregoing is sufficient.
- YeGoblynQueenne 8y agoI'm sorry, I don't understand. Which of the above generalises well from small datasets?
- darawk 8y agoWho said they do? I said they generalize better from larger datasets. The entire point of this discussion is that more data is better.
- YeGoblynQueenne 8y agoI was referring to this part of our exchange: ME: There are machine learning techniques that generalise well from few data, but they are not very well known in the industry. YOU: Sure, and we'd all love to be using those. But even if you generalize well from small datasets, you still generalize better from larger ones. That is not how those techniques work to my knowledge, so I was asking which ones you had in mind.
- darawk 8y agoMy point is that they all generalize better from larger datasets. Size is relative and some techniques work better with more or less data. Linear regression, for instance, can work quite well with much less data than a neural net. It just depends on the complexity of the problem.
- YeGoblynQueenne 8y ago>> My point is that they all generalize better from larger datasets. Like I say, this is not the case. There are learning algorithms that generalise so well from few data that their performance can improve only marginally with increasing amounts of data, or not at all. I appreciate that you probably have no idea what I'm talking about. I certainly don't mean linear regression.
- darawk 8y ago> Like I say, this is not the case. There are learning algorithms that generalise so well from few data that their performance can improve only marginally with increasing amounts of data, or not at all. Erm, no. Not unless they are solving the problem perfectly. > I appreciate that you probably have no idea what I'm talking about. I certainly don't mean linear regression. I work in the field. I'm quite certain i'm familiar with whatever it is that you think you're talking about. The category of algorithms that attempt to learn things from few examples is called 'One shot learning'. It's usually in the context of image classification, but it applies equally well elsewhere. These algorithms still learns better from more data. Do feel free to share an example of an algorithm that generalizes better from less data. I'll wait.