4 ms·

It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable peop

by roenxi 3y ago

It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. Under normal conditions the metrics you are talking to boil down to "the dev says we're making progress". The level of formality with which they say that changes, but not the underlying process of how the metric is generated. The only way to improve on that is for managers to go in to git and check what is happening. Technical managers can get deep into tracking progress. Non-technical managers cannot.

morgante 3y ago
> It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. That seems like a cop out. Knowing precisely how long a project will take is famously hard, but knowing the rough progress being made towards milestones is absolutely solvable. Git is the best place to look, but you don't need that level of detail to see new things being shipped. Everywhere I've worked (from small startups up to Google) it is absolutely the norm to have clear milestones to work towards on at least a quarterly basis.
- roenxi 3y ago
  > knowing the rough progress being made towards milestones is absolutely solvable Yeah, but the process for knowing that is ask the dev and they will tell you. Or ask the PM who will ask the dev. Or ask a tech lead who will ask the relevant dev. Or read the spreadsheet entry that the dev updated. All roads lead through conversation with the dev working on a feature. There are occasional exceptions, but they are rare. If the dev is willing to lie, or honestly mistaken, or just too inexperienced to estimate how they are doing then it is close to impossible to gauge progress. > That seems like a cop out You would probably believe the number of people who say that.
  - skydhash 3y ago
    > If the dev is willing to lie Isn't that a firing offense? > or honestly mistaken Estimates can be wrong, but I think a battery of test is pretty objective. > or just too inexperienced to estimate how they are doing That's hiring the wrong people
    - roenxi 3y ago
      > Isn't that a firing offense? Yeah. But the dude in the article lied about progress so I have to include it. > Estimates can be wrong, but I think a battery of test is pretty objective. Yeah. But if 80% of the tests are passing, is the project 80% of the way through the timeline to completion? No. They are useful because they measure correctness but they don't measure progress. I also don't think it is reasonable to expect non-technical managers to interpret technical tests; even as a rough measure of progress.
      - skydhash 3y ago
        If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. It may not help predict the end of the project, but I dare say that a manager would like to know that instead of a vague "We're still working on it, but we don't know when will be done". Even if it's a bug you're investigating, a more helpful report would be "I had three hypothesis, I managed to eliminate two" instead of "I'm still on this ticket".
        roenxi 3y ago
        > If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. Sure. But that is the sort of information you get by ... talking to the responsible dev. You can get the dev to describe their progress in any one of a myriad of ways. But you aren't escaping from the fundamental issue here of the dev self-reporting and management ultimately having to have confidence in the dev's ability to self-report. There is no more information for non-technical management in this metric than if the dev says "I think I'm about 60% done". I'd have more respect for a dev that estimated this way than one who just guessed at random, but it isn't making it easier for non-technical people to gauge progress. It is actually running a serious risk of backfiring, because on the date that the metric suggests the project should complete, the dev might quite reasonably discover something that needs another X amount of time to work on something that the test cases didn't cover. Then management may well feel deceived because it turns out the metric wasn't tracking progress accurately.

by roenxi 3y ago

morgante 3y ago

> It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. That seems like a cop out. Knowing precisely how long a project will take is famously hard, but knowing the rough progress being made towards milestones is absolutely solvable. Git is the best place to look, but you don't need that level of detail to see new things being shipped. Everywhere I've worked (from small startups up to Google) it is absolutely the norm to have clear milestones to work towards on at least a quarterly basis.

roenxi 3y ago
> knowing the rough progress being made towards milestones is absolutely solvable Yeah, but the process for knowing that is ask the dev and they will tell you. Or ask the PM who will ask the dev. Or ask a tech lead who will ask the relevant dev. Or read the spreadsheet entry that the dev updated. All roads lead through conversation with the dev working on a feature. There are occasional exceptions, but they are rare. If the dev is willing to lie, or honestly mistaken, or just too inexperienced to estimate how they are doing then it is close to impossible to gauge progress. > That seems like a cop out You would probably believe the number of people who say that.
- skydhash 3y ago
  > If the dev is willing to lie Isn't that a firing offense? > or honestly mistaken Estimates can be wrong, but I think a battery of test is pretty objective. > or just too inexperienced to estimate how they are doing That's hiring the wrong people
  - roenxi 3y ago
    > Isn't that a firing offense? Yeah. But the dude in the article lied about progress so I have to include it. > Estimates can be wrong, but I think a battery of test is pretty objective. Yeah. But if 80% of the tests are passing, is the project 80% of the way through the timeline to completion? No. They are useful because they measure correctness but they don't measure progress. I also don't think it is reasonable to expect non-technical managers to interpret technical tests; even as a rough measure of progress.
    - skydhash 3y ago
      If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. It may not help predict the end of the project, but I dare say that a manager would like to know that instead of a vague "We're still working on it, but we don't know when will be done". Even if it's a bug you're investigating, a more helpful report would be "I had three hypothesis, I managed to eliminate two" instead of "I'm still on this ticket".
      - roenxi 3y ago
        > If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. Sure. But that is the sort of information you get by ... talking to the responsible dev. You can get the dev to describe their progress in any one of a myriad of ways. But you aren't escaping from the fundamental issue here of the dev self-reporting and management ultimately having to have confidence in the dev's ability to self-report. There is no more information for non-technical management in this metric than if the dev says "I think I'm about 60% done". I'd have more respect for a dev that estimated this way than one who just guessed at random, but it isn't making it easier for non-technical people to gauge progress. It is actually running a serious risk of backfiring, because on the date that the metric suggests the project should complete, the dev might quite reasonably discover something that needs another X amount of time to work on something that the test cases didn't cover. Then management may well feel deceived because it turns out the metric wasn't tracking progress accurately.