3 ms·

Yeah that'd be great. I also want cars that run on salt water. I'm not saying that small data ai is equally impossible, but simply saying "we should make this

by TOMDM 5y ago

Yeah that'd be great. I also want cars that run on salt water. I'm not saying that small data ai is equally impossible, but simply saying "we should make this better thing" isn't enough.

atulsnj 5y ago
Atleast someone's working on it.
Datenstrom 5y ago
> simply saying "we should make this better thing" isn't enough. Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same results. It has also been my primary focus for the past 5 years and the core of the company I founded.
- riku_iki 5y ago
  > it is possible to smartly choose the correct few samples for the network that yield the same results. And then, someone is using pretrained 500B model, and fine-tuning your few examples, and getting new SOTA.
  - deleted 5y ago
    [deleted]
  - YeGoblynQueenne 5y ago
    They might get new SOTA because the metric is accuracy, but if the metric was accuracy weighted by sample efficiency, then SOTA would look a lot less impressive. Simplest way to weigh by sample efficiency: multiply accuracy by ratio of test set to training set sizes. Everyone's training/testing on 80/20 splits, so everybody's SOTA would go down by 3/4s.
sanxiyn 5y ago
It's more of "this direction seems higher ROI than that direction", in particular quality vs quantity of data. Already in 2018 SenseTime reported that for face recognition, clean dataset surpasses accuracy of 4x larger raw dataset. https://arxiv.org/abs/1807.11649 https://arxiv.org/abs/1807.11649
technocratius 5y ago
I would say that Andrew Ng has some credibility in putting practice to his preaching.
mdp2021 5y ago
«Small data /ai/» is not "impossible", it is actually necessary: AI, opposed to this ML, implies perfectioned digestion of the input data. Only, the article seemed to show a very conservative Ng about the algorithms, a focus on data management - so it's still ML.

by TOMDM 5y ago

Yeah that'd be great. I also want cars that run on salt water. I'm not saying that small data ai is equally impossible, but simply saying "we should make this better thing" isn't enough.

atulsnj 5y ago

Atleast someone's working on it.

Datenstrom 5y ago

> simply saying "we should make this better thing" isn't enough. Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same results. It has also been my primary focus for the past 5 years and the core of the company I founded.

riku_iki 5y ago
> it is possible to smartly choose the correct few samples for the network that yield the same results. And then, someone is using pretrained 500B model, and fine-tuning your few examples, and getting new SOTA.
- deleted 5y ago
  [deleted]
- YeGoblynQueenne 5y ago
  They might get new SOTA because the metric is accuracy, but if the metric was accuracy weighted by sample efficiency, then SOTA would look a lot less impressive. Simplest way to weigh by sample efficiency: multiply accuracy by ratio of test set to training set sizes. Everyone's training/testing on 80/20 splits, so everybody's SOTA would go down by 3/4s.

sanxiyn 5y ago

It's more of "this direction seems higher ROI than that direction", in particular quality vs quantity of data. Already in 2018 SenseTime reported that for face recognition, clean dataset surpasses accuracy of 4x larger raw dataset. https://arxiv.org/abs/1807.11649 https://arxiv.org/abs/1807.11649

technocratius 5y ago

I would say that Andrew Ng has some credibility in putting practice to his preaching.

mdp2021 5y ago

«Small data /ai/» is not "impossible", it is actually necessary: AI, opposed to this ML, implies perfectioned digestion of the input data. Only, the article seemed to show a very conservative Ng about the algorithms, a focus on data management - so it's still ML.