3 ms·

One of the worst people I worked with was a guy who worked 12+ hours and his code was insanely complex for no good reason. He rewrote other people's code over n

One of the worst people I worked with was a guy who worked 12+ hours and his code was insanely complex for no good reason. He rewrote other people's code over night without any review. Because he was good friend with management he could get away with it. He created a 100.000 line ETL tool that I replaced with 1000 lines one, that also run 10-100 times faster. I did it while explained everything to all of the team members so everybody was onboard. I never work more than 8 hours and I am willing to be fired over this. I guess work more hours is not indication of any quality and people should not be judged based on this.

mgkimsal 8y ago
>He created a 100.000 line ETL tool that I replaced with 1000 lines one, that also run 10-100 times faster. I did it while explained everything to all of the team members so everybody was onboard. Data import processes often seem to get shuffled to people usually least prepared to understand how to handle them. 2 anecdotes: 1. worked someplace years ago who would take in customer data, transform it in to our structure, then put in to mysql (2004 mysql). larger projects... this ended up taking 24 hours in some cases. I spent some time (a few hours), and had the whole thing working in under an hour. Working with a couple other folks there they got it down to around 20-25 minutes. The big secret was .... dropping indexes, importing chunks in to memory (vs disk), then copying the memory table to disk after a chunk. This was first met with "that can't work... we've already optimized everything... etc". Until it was demonstrated. 2. Colleague worked at a company which was primarily email marketing. They'd take in email lists, "import" them (which created some related metadata records, etc). The guy who wrote this importer had 'rewritten it to be faster' a year earlier, and larger imports (we're talking like... 10k emails) were taking days. Onboarding a client of 30k emails would take 2 weeks. It was literally insane code. My colleague rewrote the whole thing to work the 30k emails in under an hour, documented the code, had tests demonstrating weird edge cases... "Can't trust it - original dev needs to review it first". He (my colleague) was part of a group of people who was let go a couple months later (and was also approaching 40). The phrase "big data" was also taking hold as a buzzword around the time of incident 2 (2 years ago maybe?). People think anything that can't fit in to excel is somehow 'big data'.
- mmt 8y ago
  > People think anything that can't fit in to excel is somehow 'big data'. To be fair, I'd say this is only true if "excel" is generalized to "the largest non-distributed-computing tool they're personally aware of". Granted, that may usually be a spreadsheet, but at least in the tech industry, people do know of DBMSes. They just happen to think an affordable single server is an order of magnitude smaller than is actually achievable, even if it's an order of magnitude larger than a spreadsheet.

mgkimsal 8y ago

>He created a 100.000 line ETL tool that I replaced with 1000 lines one, that also run 10-100 times faster. I did it while explained everything to all of the team members so everybody was onboard. Data import processes often seem to get shuffled to people usually least prepared to understand how to handle them. 2 anecdotes: 1. worked someplace years ago who would take in customer data, transform it in to our structure, then put in to mysql (2004 mysql). larger projects... this ended up taking 24 hours in some cases. I spent some time (a few hours), and had the whole thing working in under an hour. Working with a couple other folks there they got it down to around 20-25 minutes. The big secret was .... dropping indexes, importing chunks in to memory (vs disk), then copying the memory table to disk after a chunk. This was first met with "that can't work... we've already optimized everything... etc". Until it was demonstrated. 2. Colleague worked at a company which was primarily email marketing. They'd take in email lists, "import" them (which created some related metadata records, etc). The guy who wrote this importer had 'rewritten it to be faster' a year earlier, and larger imports (we're talking like... 10k emails) were taking days. Onboarding a client of 30k emails would take 2 weeks. It was literally insane code. My colleague rewrote the whole thing to work the 30k emails in under an hour, documented the code, had tests demonstrating weird edge cases... "Can't trust it - original dev needs to review it first". He (my colleague) was part of a group of people who was let go a couple months later (and was also approaching 40). The phrase "big data" was also taking hold as a buzzword around the time of incident 2 (2 years ago maybe?). People think anything that can't fit in to excel is somehow 'big data'.

mmt 8y ago
> People think anything that can't fit in to excel is somehow 'big data'. To be fair, I'd say this is only true if "excel" is generalized to "the largest non-distributed-computing tool they're personally aware of". Granted, that may usually be a spreadsheet, but at least in the tech industry, people do know of DBMSes. They just happen to think an affordable single server is an order of magnitude smaller than is actually achievable, even if it's an order of magnitude larger than a spreadsheet.