2 ms·

I think the ideal of more data == faster advancements may turn out to be a false equivalency. Similar to prior innovations, a healthy headcount, headstart and s

I think the ideal of more data == faster advancements may turn out to be a false equivalency. Similar to prior innovations, a healthy headcount, headstart and steady R&D funding will more likely make a difference. Besides, rather than vacuuming up exisiting data, future resources might be better spent on creating bespoke datasets with user consent and content licensing baked in from the start.

kaibee 3y ago
> Besides, rather than vacuuming up exisiting data future resources might be better spent on creating bespoke datasets with user consent and content licensing baked in from the start. Not mutually exclusive.
- dleeftink 3y ago
  Probably not, but I think it is worthwhile to see if models can achieve sota with less, and if not, have individuals clearly onboard and informed of their data contributions. We have conducted large statistical surveys in the past, so I see little reason why these and other established data collection practices could not be updated and utilised for refining contemporary models.
graphe 3y ago
For DNA and genetics more data is better. Data and it's interpretation is the bottleneck for some subjects but not all. Getting more data from search engines about purchasing habits might not yield more/better results as MoE showed when 2-3 experts was better than 4-6.
- dleeftink 3y ago
  At first, data is needed to overcome scarcity in domains where it has traditionally been difficult to obtain -- but I don't think it's quite the same as the 'data' being (colloquially) referred to here. We may want to discern between user generated content and genetic features that one has little influence in creating, among the many other data types and kinds. It's probably a large reason why this space is so difficult to license/regulate: data to one may simply be 'noise' to another, or not even register as such. Definitions unfortunately, do not always map.

kaibee 3y ago

> Besides, rather than vacuuming up exisiting data future resources might be better spent on creating bespoke datasets with user consent and content licensing baked in from the start. Not mutually exclusive.

dleeftink 3y ago
Probably not, but I think it is worthwhile to see if models can achieve sota with less, and if not, have individuals clearly onboard and informed of their data contributions. We have conducted large statistical surveys in the past, so I see little reason why these and other established data collection practices could not be updated and utilised for refining contemporary models.

graphe 3y ago

For DNA and genetics more data is better. Data and it's interpretation is the bottleneck for some subjects but not all. Getting more data from search engines about purchasing habits might not yield more/better results as MoE showed when 2-3 experts was better than 4-6.

dleeftink 3y ago
At first, data is needed to overcome scarcity in domains where it has traditionally been difficult to obtain -- but I don't think it's quite the same as the 'data' being (colloquially) referred to here. We may want to discern between user generated content and genetic features that one has little influence in creating, among the many other data types and kinds. It's probably a large reason why this space is so difficult to license/regulate: data to one may simply be 'noise' to another, or not even register as such. Definitions unfortunately, do not always map.