4 ms·

I work at a FAANG and it was absolutely astonishing to find out how often this happens. You can make a long, impactful career by just being "the guy who adds l

I work at a FAANG and it was absolutely astonishing to find out how often this happens. You can make a long, impactful career by just being "the guy who adds log statements throughout the codebase and reasons through it", doing this at even a simplistic level has always shown me an astonishing fix to some long-standing issue. n.b. It also attracts a ton of political fun. People's first order reaction is denial, and it only gets worse from there. Absolutely no one except 1-2 colleagues will see it as "oh we should fix that", and at least one person will make sure your boss' boss' boss is CCd on an email with a nice version of "no he's just insufficiently concerned about {concurrency, memory management, take your pick}" Just wait it out quietly when that happens, do not engage or complain. If nothing happens and you're never asked about it by leadership, but your peers ask, make plans to move onto another team.

jrochkind1 3y ago
A long impactful career, or a career of horrible frustration and alienation as everyone gets mad at you for pointing out their bugs? (or, from their point of view, making trouble insisting that something is a bug which isn't and is causing no problems)
sidfthec 3y ago
What FAANG have you seen this at? I've been at big tech companies for most of my career and I've never seen anyone deny the existence of a technical bug. I've seen plenty of teams mark a bug as lower priority and never fix it because other things are higher priority. But denying that the bug exists, especially after a detailed explanation? That doesn't resonate with my experiences.
- com2kid 3y ago
  I've told this story before! It used to be writing the outputs from the C/C++ preprocessor (.i files) to disk took forever (5+ minutes IIRC) with Microsoft's compilers. I asked one of the lead compiler developers why, and he waved me away saying it was just really complicated. Around that time a bunch of tools existed for GCC that worked with .i files, but none existed in the Microsoft ecosystem likely because writing .i files was so slow. I was on the compiler test team at the time and we did lots of stuff with .i files, our tests were distributed across a large cluster of test machines (see my post about that https://meanderingthoughts.hashnode.dev/how-microsoft-tested-compilers-circa-2006 https://meanderingthoughts.hashnode.dev/how-microsoft-tested...) so it wasn't a big deal, but it still annoyed me. One day I decided to find out what was going on, so I loaded up process monitor while outputting a .i file and watched what was happening. Much to my surprise, only 1 byte was being written at a time! No wonder writes were taking forever. A quick dive into the source code revealed a comment above the file write call that read to the effect // to work around a bug in windows 98 So anyway I opened a bug against the compiler saying we should probably fix that. :)
  - sidfthec 3y ago
    But that's not the type of story that's being claimed from the person I responded to. Of course the lead developer waved you off. You wondered why things took forever, and the lead developer knew it was a complicated system and figured it wasn't worth their time investigating. It happened to be incorrect, but the lead developer wasn't in denial. They just filtered the issue out because they can't afford to go down every rabbit-hole they come across. I'm sure once you found the actual bug, it was later fixed. The person I was responding to seems to think a large number of people are in denial when a bug is filed against them. That doesn't make sense, and isn't something I see. It'd be as if when you pointed out the actual bug, the lead developer continued to say it wasn't actually a bug (which is of course ridiculous and I bet didn't happen).
    - refulgentis 3y ago
      You are ascribing an absurdly maximalist viewpoint to me, one that would be obviously wrong at its face. I know it's not so confusing as to get that sort of interpretation, because of the score on the comment, and comments like the above that explain to you how this happens. As a result, I don't feel comfortable providing more detail publicly about my situation. That far off the mark tends to indicate an aggressive rather than curious interlocutor. I am comfortable building on their example. The particulars of the issue are quite similar in a very helpful way. I did the investigation, did a fix, worked it up to my manager and my managers manager. Elated, we work diligently for a couple weeks to document concisely, 3 page tech doc, briefest code deltas possible, one page + slides withs simple diagrams. It gets bogged down at managers managers coleads submanager for the platform team implicated. They basically say "reading the single byte at a time means its provably serial and thus has no concurrency bugs.", as indicated in my original comment.
    - bogwog 3y ago
      It's 2 anecdotes to your 1. Anecdotes are useless, but you are down by 1. I suggest you call for reinforcements, or make a hasty retreat.
- deleted 3y ago
  [deleted]
jcarrano 3y ago
I used to work on a large-ish open source project. When I was bored I used to go bug-picking (not hunting, picking). I'd go and browse the source wherever my intuition told me there was likely a bug and I indeed found a few using with such a "method".

jrochkind1 3y ago

A long impactful career, or a career of horrible frustration and alienation as everyone gets mad at you for pointing out their bugs? (or, from their point of view, making trouble insisting that something is a bug which isn't and is causing no problems)

sidfthec 3y ago

What FAANG have you seen this at? I've been at big tech companies for most of my career and I've never seen anyone deny the existence of a technical bug. I've seen plenty of teams mark a bug as lower priority and never fix it because other things are higher priority. But denying that the bug exists, especially after a detailed explanation? That doesn't resonate with my experiences.

com2kid 3y ago
I've told this story before! It used to be writing the outputs from the C/C++ preprocessor (.i files) to disk took forever (5+ minutes IIRC) with Microsoft's compilers. I asked one of the lead compiler developers why, and he waved me away saying it was just really complicated. Around that time a bunch of tools existed for GCC that worked with .i files, but none existed in the Microsoft ecosystem likely because writing .i files was so slow. I was on the compiler test team at the time and we did lots of stuff with .i files, our tests were distributed across a large cluster of test machines (see my post about that https://meanderingthoughts.hashnode.dev/how-microsoft-tested-compilers-circa-2006 https://meanderingthoughts.hashnode.dev/how-microsoft-tested...) so it wasn't a big deal, but it still annoyed me. One day I decided to find out what was going on, so I loaded up process monitor while outputting a .i file and watched what was happening. Much to my surprise, only 1 byte was being written at a time! No wonder writes were taking forever. A quick dive into the source code revealed a comment above the file write call that read to the effect // to work around a bug in windows 98 So anyway I opened a bug against the compiler saying we should probably fix that. :)
- sidfthec 3y ago
  But that's not the type of story that's being claimed from the person I responded to. Of course the lead developer waved you off. You wondered why things took forever, and the lead developer knew it was a complicated system and figured it wasn't worth their time investigating. It happened to be incorrect, but the lead developer wasn't in denial. They just filtered the issue out because they can't afford to go down every rabbit-hole they come across. I'm sure once you found the actual bug, it was later fixed. The person I was responding to seems to think a large number of people are in denial when a bug is filed against them. That doesn't make sense, and isn't something I see. It'd be as if when you pointed out the actual bug, the lead developer continued to say it wasn't actually a bug (which is of course ridiculous and I bet didn't happen).
  - refulgentis 3y ago
    You are ascribing an absurdly maximalist viewpoint to me, one that would be obviously wrong at its face. I know it's not so confusing as to get that sort of interpretation, because of the score on the comment, and comments like the above that explain to you how this happens. As a result, I don't feel comfortable providing more detail publicly about my situation. That far off the mark tends to indicate an aggressive rather than curious interlocutor. I am comfortable building on their example. The particulars of the issue are quite similar in a very helpful way. I did the investigation, did a fix, worked it up to my manager and my managers manager. Elated, we work diligently for a couple weeks to document concisely, 3 page tech doc, briefest code deltas possible, one page + slides withs simple diagrams. It gets bogged down at managers managers coleads submanager for the platform team implicated. They basically say "reading the single byte at a time means its provably serial and thus has no concurrency bugs.", as indicated in my original comment.
  - bogwog 3y ago
    It's 2 anecdotes to your 1. Anecdotes are useless, but you are down by 1. I suggest you call for reinforcements, or make a hasty retreat.
deleted 3y ago
[deleted]

jcarrano 3y ago

I used to work on a large-ish open source project. When I was bored I used to go bug-picking (not hunting, picking). I'd go and browse the source wherever my intuition told me there was likely a bug and I indeed found a few using with such a "method".