3 ms·

I've encountered a few manual processes for producing data recently that amount to query this thing from here, export this CSV from here then plug it into this

I've encountered a few manual processes for producing data recently that amount to query this thing from here, export this CSV from here then plug it into this sheet and write down the number. It works, it produces the right number and probably only takes a few minutes of one person's time each month. I'm still for automating this end to end, because it makes it auditable and tamper proof. If the data is produced by some ETL job then any problems are bugs in the job, not because someone made a typo (or in the worst case falsified figures, etc - basically it's good to remove humans from the process)

agustif 5y ago
I made a code challenge which consisted basically on some simple ETL stuff, get data, parse geoJSON into a State, return data in json format in an api endpoint including pinging some random example api, which casually the day I sent the assignment and was testing it, would be crashing and giving nulls for the positions. It was because some CDN/DNS provider du jour (can't remember exactly which was down. So whatever was auto-generating random gps locations wasn't working it this small time window and would make my otherwise correct code and tests fail, I explained all this and got hired, but I could haven't had the luck to read about the down issue on HN and just be f***