4 ms·

It is unfortunate and it looks like it was a very interesting project, but I have so many questions. How come they looked at the data just now when they “collec

It is unfortunate and it looks like it was a very interesting project, but I have so many questions. How come they looked at the data just now when they “collected enough” and not along the way. Quality of data and mis-labeling issues is something that would be apparent right away, no? What kind of “data quality” we are talking about? Is it raw waveform data? This has the ceiling of 44 KHz and pretty much everything is very close to it. Quality of existing labeling?

nerdponx 4y ago
> We finally got around to doing this recenty, and realised that the data simply isn’t of high enough quality to be useful for much at all. I suspect that they just didn't have the right people on the team, or those people were busy and haven't gotten around to it. This is sad, but it's an important cautionary tale for businesses that plan to collect data first, and later hire a data person to turn that data into gold. Many, many businesses in the 2010s probably failed because they thought they could do this. The truth is that you need data people involved from the beginning, and continuously throughout the project, in order to monitor and evaluate the data being collected, building proof-of-concept models along the way, and to adjust the data collection process as problems are discovered and new techniques are developed.
- wpietri 4y ago
  Oh god yes. The "collect data" was such an "underpants gnomes" business model thing. Just thinking about it gives me flashbacks to industry events where insufferable people droned at me about this. And yes, another vote here for early proofs of concept and rigorous testing. Not everybody gets what really matters in data quality, but the surest way to find out is to try to build something and see if it really works.
arbitrage 4y ago
> Quality of data and mis-labeling issues is something that would be apparent right away, no? You are correct, those issues were apparent right from the beginning, and they never really got better. The acoustic fingerprinting worked sorta OK for very popular albums -- but even for that, it was never 100% accurate. It never worked well for live performances, imports, classical music, jazz, or jam bands. Using this software always required a lot of manual intervention, which at least for me negates the whole point of using it in the first place.

nerdponx 4y ago

> We finally got around to doing this recenty, and realised that the data simply isn’t of high enough quality to be useful for much at all. I suspect that they just didn't have the right people on the team, or those people were busy and haven't gotten around to it. This is sad, but it's an important cautionary tale for businesses that plan to collect data first, and later hire a data person to turn that data into gold. Many, many businesses in the 2010s probably failed because they thought they could do this. The truth is that you need data people involved from the beginning, and continuously throughout the project, in order to monitor and evaluate the data being collected, building proof-of-concept models along the way, and to adjust the data collection process as problems are discovered and new techniques are developed.

wpietri 4y ago
Oh god yes. The "collect data" was such an "underpants gnomes" business model thing. Just thinking about it gives me flashbacks to industry events where insufferable people droned at me about this. And yes, another vote here for early proofs of concept and rigorous testing. Not everybody gets what really matters in data quality, but the surest way to find out is to try to build something and see if it really works.

arbitrage 4y ago

> Quality of data and mis-labeling issues is something that would be apparent right away, no? You are correct, those issues were apparent right from the beginning, and they never really got better. The acoustic fingerprinting worked sorta OK for very popular albums -- but even for that, it was never 100% accurate. It never worked well for live performances, imports, classical music, jazz, or jam bands. Using this software always required a lot of manual intervention, which at least for me negates the whole point of using it in the first place.