3 ms·
They're all data scientists and ML engineers now, which means all the data mining, cleaning/preparation and labeling is going to be beneath them You have absol
by gaius 8y ago
They're all data scientists and ML engineers now, which means all the data mining, cleaning/preparation and labeling is going to be beneath them
You have absolutely hit the nail on the head there and this mirrors my observations of what's happening in the wider industry. Data science and AI are super cool viewed from the outside but the reality of the work day to day is that it is not fun. Getting a result and making a meaningful impact is very satisfying, but getting there requires careful, painstaking, meticulous work, getting the data and getting it into a format you can use is the vast majority of it. It is essential of course, but noone enjoys spending weeks (or months) decoding exactly what the fields are in this big weird CSV file you got from the mainframe and how exactly they marry up with the XML you got from this other system then doing that 100 times to mash up all your data sources, there's no documentation and the people who programmed it originally are long gone and get something that you can finally feed into your ML step. And then you come up with some recommendations which are immediately shot down because they are actually illegal and noone in the business can believe you even suggested it because that gets taught in Compliance 101 (I have really seen this happen).
Someone with the patience and the good attitude to do the data prep, and who has a bit of basic domain knowledge, armed with even the most rudimentary ML techniques will in any practical sense run rings around any rockstar researcher who just jumps in straight away with the AI. You would hope that PhDs who spend literally years doing research before writing up would understand this but it seems to be the first thing they forget!
- Ftuuky 8y ago>Someone with the patience and the good attitude to do the data prep, and who has a bit of basic domain knowledge, armed with even the most rudimentary ML techniques will in any practical sense run rings around any rockstar researcher who just jumps in straight away with the AI. You would hope that PhDs who spend literally years doing research before writing up would understand this but it seems to be the first thing they forget! You articulated so well something that I've been trying to say to my managers. Thank you for your post.