4 ms·

If management has no visibility into the project progress, that seems like a pretty major problem. It absolutely shouldn't be "binary." Any year-long project s

If management has no visibility into the project progress, that seems like a pretty major problem. It absolutely shouldn't be "binary." Any year-long project should have tractable progress metrics. Upper management doesn't need to be technical, but someone in the chain absolutely should be capable of translating progress into a digestible format.

roenxi 2y ago
It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. Under normal conditions the metrics you are talking to boil down to "the dev says we're making progress". The level of formality with which they say that changes, but not the underlying process of how the metric is generated. The only way to improve on that is for managers to go in to git and check what is happening. Technical managers can get deep into tracking progress. Non-technical managers cannot.
- morgante 2y ago
  > It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. That seems like a cop out. Knowing precisely how long a project will take is famously hard, but knowing the rough progress being made towards milestones is absolutely solvable. Git is the best place to look, but you don't need that level of detail to see new things being shipped. Everywhere I've worked (from small startups up to Google) it is absolutely the norm to have clear milestones to work towards on at least a quarterly basis.
  - roenxi 2y ago
    > knowing the rough progress being made towards milestones is absolutely solvable Yeah, but the process for knowing that is ask the dev and they will tell you. Or ask the PM who will ask the dev. Or ask a tech lead who will ask the relevant dev. Or read the spreadsheet entry that the dev updated. All roads lead through conversation with the dev working on a feature. There are occasional exceptions, but they are rare. If the dev is willing to lie, or honestly mistaken, or just too inexperienced to estimate how they are doing then it is close to impossible to gauge progress. > That seems like a cop out You would probably believe the number of people who say that.
    - skydhash 2y ago
      > If the dev is willing to lie Isn't that a firing offense? > or honestly mistaken Estimates can be wrong, but I think a battery of test is pretty objective. > or just too inexperienced to estimate how they are doing That's hiring the wrong people
      - roenxi 2y ago
        > Isn't that a firing offense? Yeah. But the dude in the article lied about progress so I have to include it. > Estimates can be wrong, but I think a battery of test is pretty objective. Yeah. But if 80% of the tests are passing, is the project 80% of the way through the timeline to completion? No. They are useful because they measure correctness but they don't measure progress. I also don't think it is reasonable to expect non-technical managers to interpret technical tests; even as a rough measure of progress.
        skydhash 2y ago
        If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. It may not help predict the end of the project, but I dare say that a manager would like to know that instead of a vague "We're still working on it, but we don't know when will be done". Even if it's a bug you're investigating, a more helpful report would be "I had three hypothesis, I managed to eliminate two" instead of "I'm still on this ticket".
        roenxi 2y ago
        > If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. Sure. But that is the sort of information you get by ... talking to the responsible dev. You can get the dev to describe their progress in any one of a myriad of ways. But you aren't escaping from the fundamental issue here of the dev self-reporting and management ultimately having to have confidence in the dev's ability to self-report. There is no more information for non-technical management in this metric than if the dev says "I think I'm about 60% done". I'd have more respect for a dev that estimated this way than one who just guessed at random, but it isn't making it easier for non-technical people to gauge progress. It is actually running a serious risk of backfiring, because on the date that the metric suggests the project should complete, the dev might quite reasonably discover something that needs another X amount of time to work on something that the test cases didn't cover. Then management may well feel deceived because it turns out the metric wasn't tracking progress accurately.
bruce511 2y ago
When you build a building, it starts by digging a hole and then filling it up. Then you see walls go up etc. Although progress is visible, visible can lie. Obviously the project had milestones. But it is in the nature of big conversions/updates that you will encounter things you don't know. We're in that stage now of killing the unknowns. It's not a linear process and the quantity of them is unknown. So at this point metrics become hard. (We hit all the targets for the "knowns", and we continue to squash the unknowns, so the dev team is confident, but management is nervous.) Progress is easy to digest. But (for now) there's no easy "finish line" to see, which makes their lives hard. There's no "quick fix" here where "better paperwork" would improve things. As much as you (and indeed management) would like that.

roenxi 2y ago

It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. Under normal conditions the metrics you are talking to boil down to "the dev says we're making progress". The level of formality with which they say that changes, but not the underlying process of how the metric is generated. The only way to improve on that is for managers to go in to git and check what is happening. Technical managers can get deep into tracking progress. Non-technical managers cannot.

morgante 2y ago
> It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. That seems like a cop out. Knowing precisely how long a project will take is famously hard, but knowing the rough progress being made towards milestones is absolutely solvable. Git is the best place to look, but you don't need that level of detail to see new things being shipped. Everywhere I've worked (from small startups up to Google) it is absolutely the norm to have clear milestones to work towards on at least a quarterly basis.
- roenxi 2y ago
  > knowing the rough progress being made towards milestones is absolutely solvable Yeah, but the process for knowing that is ask the dev and they will tell you. Or ask the PM who will ask the dev. Or ask a tech lead who will ask the relevant dev. Or read the spreadsheet entry that the dev updated. All roads lead through conversation with the dev working on a feature. There are occasional exceptions, but they are rare. If the dev is willing to lie, or honestly mistaken, or just too inexperienced to estimate how they are doing then it is close to impossible to gauge progress. > That seems like a cop out You would probably believe the number of people who say that.
  - skydhash 2y ago
    > If the dev is willing to lie Isn't that a firing offense? > or honestly mistaken Estimates can be wrong, but I think a battery of test is pretty objective. > or just too inexperienced to estimate how they are doing That's hiring the wrong people
    - roenxi 2y ago
      > Isn't that a firing offense? Yeah. But the dude in the article lied about progress so I have to include it. > Estimates can be wrong, but I think a battery of test is pretty objective. Yeah. But if 80% of the tests are passing, is the project 80% of the way through the timeline to completion? No. They are useful because they measure correctness but they don't measure progress. I also don't think it is reasonable to expect non-technical managers to interpret technical tests; even as a rough measure of progress.
      - skydhash 2y ago
        If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. It may not help predict the end of the project, but I dare say that a manager would like to know that instead of a vague "We're still working on it, but we don't know when will be done". Even if it's a bug you're investigating, a more helpful report would be "I had three hypothesis, I managed to eliminate two" instead of "I'm still on this ticket".
        roenxi 2y ago
        > If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. Sure. But that is the sort of information you get by ... talking to the responsible dev. You can get the dev to describe their progress in any one of a myriad of ways. But you aren't escaping from the fundamental issue here of the dev self-reporting and management ultimately having to have confidence in the dev's ability to self-report. There is no more information for non-technical management in this metric than if the dev says "I think I'm about 60% done". I'd have more respect for a dev that estimated this way than one who just guessed at random, but it isn't making it easier for non-technical people to gauge progress. It is actually running a serious risk of backfiring, because on the date that the metric suggests the project should complete, the dev might quite reasonably discover something that needs another X amount of time to work on something that the test cases didn't cover. Then management may well feel deceived because it turns out the metric wasn't tracking progress accurately.

bruce511 2y ago

When you build a building, it starts by digging a hole and then filling it up. Then you see walls go up etc. Although progress is visible, visible can lie. Obviously the project had milestones. But it is in the nature of big conversions/updates that you will encounter things you don't know. We're in that stage now of killing the unknowns. It's not a linear process and the quantity of them is unknown. So at this point metrics become hard. (We hit all the targets for the "knowns", and we continue to squash the unknowns, so the dev team is confident, but management is nervous.) Progress is easy to digest. But (for now) there's no easy "finish line" to see, which makes their lives hard. There's no "quick fix" here where "better paperwork" would improve things. As much as you (and indeed management) would like that.