3 ms·
This is a common problem I've both experienced as a data scientist at a start up, and heard about from friends who work in data science/machine roles at larger
by wespiser_2018 6y ago
This is a common problem I've both experienced as a data scientist at a start up, and heard about from friends who work in data science/machine roles at larger companies. I'm both not surprised, but sorry to hear, this experience will result in you leaving your current position. That stress sucks!
I think there is a multifactor problem with building a work product that other people in the company don't fully appreciate, and without gaining full "buy-in" to take the risks and time investment needed for success.
Part of this problem is communication: you need to be your projects own advocate, and communicate clearly how "model performance is X" and for profitability under assumption Y, we need "performance of at least Z given ...". This could take substantial time, but someone needs to do it, and it's the only way to justify the time investment in your model as a necessary business expense. Model training is just not important, where the model is in terms of business application is the only thing you should be communicating to a small team.
The other problem, I've found, is that some teams are far less willing or able to understand that the technical challenges of machine learning/data science are not quite the same class of problem as software engineering, and the deliver/work cycle can be both slower and less determinant. People read this as you not getting stuff done (your fault). The best remedy I've found is just to deliver early and often, which is sounds like you've done.