4 ms·
“It has been a common trope that 80% of a data scientist’s valuable time is spent simply finding, cleaning, and organizing data, leaving only 20% to actually pe
by appleiigs 8y ago
“It has been a common trope that 80% of a data scientist’s valuable time is spent simply finding, cleaning, and organizing data, leaving only 20% to actually perform analysis.”
Is it really trope? For my experience I almost think collecting data is >80%.
- visarga 8y agoI get to spend 90% collecting, cleaning and tagging data.
- pbhjpbhj 8y agoSounds like poor role definition? You don't have your laboratory scientist cleaning lab equipment or ordering reactants.
- frozenport 8y agoIf you do image processing: 10% collecting, 90% "analyzing" it to fit our bullshit models.
- narens 8y agohttps://dremio.com https://dremio.com tries to solve this exact problem. It’s open source too.
- FuriouslyAdrift 8y agoso I checked out the website and I still don't get what Dremio actually is... I know what I need which is a wide open easy to use (think CEO's personal assistant who has zero background in anything other than basic business processes easy) visual ETL tool. We have found some decent VERY expensive and VERY user unfriendly tools but nothing cheaper than just hiring a person to custom handle each batch. Basically...we're stuck with Cognos, et al.
- gaius 8y ago80% of time is spent on getting and cleaning the data, 20% of time is spent preparing and delivering reports and presentations. The super-cool ML stuff that attracts people to the field in the first place, accounts for little more than a rounding error in how the time is really spent