3 ms·
I want to up-vote your comment more than once. Just this week, my boss told me to aggregate and then analyze 90 spreadsheets worth of data along with associated
by djacobs 16y ago
I want to up-vote your comment more than once. Just this week, my boss told me to aggregate and then analyze 90 spreadsheets worth of data along with associated metadata scattered around in various (structured) files. The task is one I'm going to have to do once every two weeks. It took my predecessor three days every time she had to do it, three days during which she had no time for any other experiments.
My solution? I wrote a Ruby script in 2 hours, then an R script in 1 hour (to run the stats). They're both flexible, reusable, and efficient. They get my boss her report the same day as my raw data come to me. And since my source data comes from another computer program, the process is flawless and does not suffer from copy errors or improperly paired metadata.
People should not have to do repetitive tasks like aggregation or mapping data to metadata by hand. We should be teaching every high schooler basic scripting and (more generally) that we can outsource the mundane to a computer.