3 ms·
It depends a bit on the definition of the roles, however this is basically what I am arguing against. I think you are right that there are organizational trade
by zwaps 5y ago
It depends a bit on the definition of the roles, however this is basically what I am arguing against.
I think you are right that there are organizational trade-offs, especially in day-to-day operations.
However, I would claim that if DE and DS are divorced enough, there is essentially no way that DS work can be done correctly and it would be up to chance whether the DS division in your firm actually does useful work, or does all the things that the article bemoans.
Much frustration with Data Science (also as indicated in the article) arise because DS is seen as an ad-hoc creative enterprise to squeeze "insights" out of an existing data-repository and pipeline.
Instead, I'd argue, that anything that touches data needs to be handled by both a DE and a DS (not necessarily the same person).
There are fundamental reasons for this. If you really want to draw insights from data, then you can not simply take some data set and go at it. From a standpoint of statistical identification, data and model go hand in hand. It's not a model that is identified, it's a model on a specific statistical process (that results in a set of data). And frequently, thinking about how to generate and process the data is much more important than the choice of which model to run.
In other words, DS need to be involved from the moment the data starts to exist as such, in particular in the moment the data is collected or generated.Otherwise, DS are working blindfolded - which is really the norm right now.
Think of it this way: if the DS process in your company starts with the tabular data that goes into the model, then there is essentially no way that the process of creating a statistical model can be right.
If the outcome works, it is as much chance as it is skill.
If the work of DS is defined by starting with some mysterious yet usable set of input data, collected somewhere else "in the machine" and squeezed together for the DS's convenience, then I highly doubt that DS will be a popular job description in the next ten years!
Again, this does not mean DS and DE should be the same person. But since many points about data are as important as they are subtle, an organizational disconnect is at least risky.
Edit:
Currently the endorsed process is to build a model, eventually be surprised by "data drift" and then rebuild everything.
In my view, this is all backwards. Sure, data changes and one needs to iterate. But this is something the DS team should have been aware of beforehand (in most cases). What I mean here is that the surprise factor conclusively shows that no one really thought about crucial properties of the data while using it.
From a statistical standpoint, this casts great doubts on these sort of analyses and models. If you are surprised by data drift, at least to the degree that it seems totally unanticipated, most likely you spent too little time on data and too much time on model.
We all know examples of this. You get "some" data collected on marketing impact on young adults and you find out that doing X does Y. And then later on, the whole thing stops working. NOW you begin an error analysis and find out that data was collected in some semi-random fashion from phone users in the US, whereas now you have new audiences from Twitch (or whatever) in SE Asia. The problem here is not the model. The problem is also not that you need to iterate. The problem is that through the opaque process of data generation no one knew, could know, or cared to know what exactly the properties of the data were.
Right now we are using an ad-hoc approach. Either, because nobody really knows how to characterize the data and what implications for inference this has (for example because statistical issues are dense, poorly communicated and consequently rarely taught in anything ML and DS), or because the organizational disconnect from data collection makes it impossible to reason about these issues.