2 ms·
It's not just quality but it's also the available features in the data. Quality is just a signal to noise problem. If you have enough data, you generally can re
by dumbfoundded 8y ago
It's not just quality but it's also the available features in the data. Quality is just a signal to noise problem. If you have enough data, you generally can reasonably segment it to get higher quality. Obtaining features not present in other data sets is probably the most significant factor.
For example, let's say you want to build a speech rec engine. You need 15K hours of data to build/validate a model. How would you get that? You could farm it out to some people on mechanical turk and get 15K hours of audio transcribed. With enough money, you could duplicate the transcriptions enough times to actually be pretty sure about the quality of the data set. If you're clever and have a large enough dataset, segmentation generally gets you decent quality. The big gains come in when you have features not present. For example, google realized when you build a speech rec engine, you can include video data and image processing to actually use the way people move their mouths to significantly increase the quality of an automated transcription.
- state_less 8y agoI think creative new ways of harvesting data will continue to be profitable. That's fine with today's mindset, but over time it seems like we'll want to go beyond big data and think about how it is that a child can become quite capable without seeing millions of instances of people crossing a road, etc... We have to make our machines make models and 'want' to gather data to test them. By having the machines do the work of today's data scientist, they may well put themselves out of business, and quickly everyone else.