4 ms·

I suspect this may actually be an issue of school vs real world rather than scientist vs engineer. Data in the classroom setting is pristine and beautiful; dat

I suspect this may actually be an issue of school vs real world rather than scientist vs engineer. Data in the classroom setting is pristine and beautiful; data in the real world is messy and buggy. You have to get burned by buggy data a few times (or maybe a bunch of times) in the real world to learn to look for bad data smells -- I don't think schools effectively teach this kind of intuition, regardless of whether the students are training as data engineers or data scientists. If data scientists are spending more time in school getting advanced degrees, they're not getting as much exposure to buggy data, whereas data engineers with a BS and a few years of industry experience would already have built up this skill.

avs733 6y ago
>Data in the classroom setting is pristine and beautiful; data in the real world is messy and buggy. I got to take over our department's undergraduate statistics course a few years back. The first change I made was all homework, tests, and projects used real data set. I intentionally have them collect bad data (they don't know its bad before hand). First day of class we collect data using the board game operation...I give basic instructions and then halfway through ask everyone to stop and agree on how they are entering data for the variable of 'success or failure' of the surgery. Oops... In my experience teaching the course, the reason the students (engineers) find statistical reasoning hard is: * They have never been given anything 'broken', everything is curated to avoid things not working. The result is they think data has inherent meaning. A right answer. * Their entire learning experience has been stripped of context and the need to make decisions with information. They can give me a p value but are terrified (not unable, just unwilling) to interpret it or give it meaning. * They have never encountered the concept of variability...everything is presented as systems with exact inputs and outputs. When I work with postdocs, I sometimes (less frequently) encounter many of the same challenges. Data is treated as sacred and external and inherent. It's wild to me.

avs733 6y ago

>Data in the classroom setting is pristine and beautiful; data in the real world is messy and buggy. I got to take over our department's undergraduate statistics course a few years back. The first change I made was all homework, tests, and projects used real data set. I intentionally have them collect bad data (they don't know its bad before hand). First day of class we collect data using the board game operation...I give basic instructions and then halfway through ask everyone to stop and agree on how they are entering data for the variable of 'success or failure' of the surgery. Oops... In my experience teaching the course, the reason the students (engineers) find statistical reasoning hard is: * They have never been given anything 'broken', everything is curated to avoid things not working. The result is they think data has inherent meaning. A right answer. * Their entire learning experience has been stripped of context and the need to make decisions with information. They can give me a p value but are terrified (not unable, just unwilling) to interpret it or give it meaning. * They have never encountered the concept of variability...everything is presented as systems with exact inputs and outputs. When I work with postdocs, I sometimes (less frequently) encounter many of the same challenges. Data is treated as sacred and external and inherent. It's wild to me.