3 ms·
This kind of problem is rife in academia. Part of the problem, as a sibling mentioned, is that many people get into these kinds of fields from non-programming
by joshvm 10y ago
This kind of problem is rife in academia.
Part of the problem, as a sibling mentioned, is that many people get into these kinds of fields from non-programming or even computer-illiterate backgrounds. At an undergraduate level, even in heavily numerical disciplines like physics, there is relatively little coding. Even then, it's tedious crap like F90 and bulletproof C. So people get into a Masters or a PhD in a field they love and are suddenly confronted by data analysis, and they have no idea what to do.
I've spoken a lot to my girlfriend (an astrophysicist) about this, as she's in this position herself. It's not that she isn't smart, she just has little to no experience with data wrangling and she's (in my opinion) been inadequately trained. I've solved things with Python one-liners that she would have spent literally days doing manually. We've had conversations along the lines of:
"But why does this file have 700 lines of input data filenames hardcoded? You know you could write some code to grab them for you?"
"Yes, but in the time it would take me to write the code, I may as well do it by hand." [In the end I wrote a 5 line regex to do it]
So I can assure you, people question the manual process and they think you're magical when you show them a quicker way of doing things. However, often there isn't the motivation or confidence to try and be magical themselves.
- Agentlien 10y agoMy wife works at the virology lab of a local hospital. They do a lot of validation and quality controls. However, when she started there, all their statistics were calculated by hand using approximate equations which were supposed to make it doable using a simple calculator. Whoever had set this up lacked a strong understanding of statistics. Still, this procedure had remained unchanged since the eighties and no one knew how or why it was supposed to work. Going back in old records, my wife found that there were a lot of errors in the recorded results. She spent a while setting up a simple Excel sheet into which they enter all test results and now all the statistics are calculated automatically using proper methods.
- throwanem 10y agoIs this necessarily a bad thing entire? House wizard can be a fairly comfortable position.
- SixSigma 10y ago> This kind of problem is rife in academia. This kind of problem is rife everywhere. People don't know and are not interested in how to use computers to help them because "it's too complicated". Excel doesn't help.
- th0waway 10y agoThis is where I take the time to explain to everyone that "Yeah, while it may save you time to do it manually, if you do it THIS way instead, you never have to write it from scratch again, and your output goes waaay up". Every job I've ever had, I always look at how my predecessors do the job, then automate the crap that I don't want to deal with. :/
- joshvm 10y agoI'm in the middle of writing a retrospective on my PhD - the stuff I wish I'd done at the beginning. Perhaps unsurprisingly, it mostly boils down to automate and pipeline everything. It almost always saves time because inevitably you have to repeat work with parameters changed very slightly.