My solution started out as a 10-line Python script where I would manually clean the data we received, then process it. CEO: "Will this scale?" Me: "No, absolu

My solution started out as a 10-line Python script where I would manually clean the data we received, then process it. CEO: "Will this scale?" Me: "No, absolutely not, at some point we'll need to hire someone who knows what they're doing." As time passed and we got more data, I significantly improved the data cleaning portions so that most of it was automated, and the parts that weren't automated would be brought up as suggestions I could quickly handle. I learned the very basics of performance and why `eval` is bad, set up my script so I didn't need to hard-code the number of files to process each day, started storing data on a network drive and then eventually a db... I still don't know what I'm doing, but by the time I left it took maybe 5 minutes of manual data cleaning to handle thousands of jobs a day, and then the remainder could be done on a single machine.