3 ms·
Like, this blows my mind. I can't even imagine a world where one would model without data cleaning. In the olden times, your model would normally error out if
by disgruntledphd2 6y ago
Like, this blows my mind. I can't even imagine a world where one would model without data cleaning.
In the olden times, your model would normally error out if you hadn't done any data cleaning (length 1 factors, constant columns etc) but i suppose with NN's they just ignore stuff like that because of the regularisation, I assume.
Every good DS I know regards the data as much more important than the models, normally from bitter, bitter experience. Models are "sexy" though, so people love them. Additionally you can do the XKCD compiling thing x 10 with models, as they take forever to fit on any non-trivial datasets.
I actually think that the cause of this is poor management. ML/DS//statistics are tools, they do not supply any business value by themselves. If people focus too much on the tools, then their manager should refocus them on the actual problem.
Mind you, if you don't have the background in this area as a manager, you're less likely to be able to spot these pathologies.
Personally, as an experienced DS, I know that my time is best spent understanding the data and the business problem, as that will drive better results. It's sad that people don't realise this (including some of my current management chain, unfortunately).