6 ms·

my own anicdata: Integrate a default proxy into our existing "choose proxy" workflow takes about 2 weeks, and I'm currently on my 3rd month of those 2 weeks :)

my own anicdata: Integrate a default proxy into our existing "choose proxy" workflow takes about 2 weeks, and I'm currently on my 3rd month of those 2 weeks :) Reasons for the delay: 1) Proxy is a stand-alone application. Needs it's own deployment and build configuration 2) Main app was running on Node 6.x. High time to upgrade to 10.x as 6.x LTS is running out. 3) Upgrade to 10.x breaks some modules we depend on (Google Cloud datastore) and that module has been deprecated in Node 8.x+.. time to refactor that.... 4) Main app is in a mono-repo with backend systems still running node 6.x Don't want to spend the time upgrading everything to node 10, so need to split Main app into it's own repo, decoupling from existing mono-repo codebase 5) Since we are upgrading main app to Node 10, Best to remove dependencies on old definitely-typed typescript typings (upgrade to npm @types definitions). And as such refactor to use latest version of all modules.... 6) Security: New Default Proxy should only allow the Main app. code a solution for this using Keypairs.... 7) Docs, sample code for users, announcement mail. thankfully i'm currently on step 7. Usually I expect to be off on my estimates by a factor of 4x. This is excessive :)

elemeno 7y ago
But in a case like yours if sounds like steps 2 through 5 aren't strictly relevant to the change you're making, so it's no wonder that your estimate is blown out the water. (Sorry if I'm wrong, you've obviously got more context on your own platform than I do!) It does make me wonder though if there is a trend of 'hiding' necessary maintenance work in larger tasks - if so that seems to me to be indicative of a larger problem around not being given/making the time to do those jobs as their own tasks.
- hinkley 7y ago
  Yes, there is. Probably the best four projects I worked on, there was collusion to spread out the cost of necessary maintenance across all work items. Because nobody will agree to doing maintenance stories (this is how Scrum hijacks developer ethics). Never ask permission to do something that must be done. Or to look at it another way, if no stories are maintenance stories, then all stories are maintenance stories.
  - novaleaf 7y ago
    I used to be all about jumping on the next fancy thing as soon as it was (pre)released. Having been burnt and learned my lesson, now I delay until it's blindingly obvious it needs to be done. But yeah, if it needs t be done, do it.
    - Sharlin 7y ago
      Version x.y.0 of most software is "public beta" quality. x.y.1 is "release candidate". x.y.2 might be production-ready.
Balgair 7y ago
My own 'magnification' index is as such: Take whatever time I think it should take. Then double that number and go up one time unit. Exp: I think it should take me 3 hours this afternoon to do this -> 6 days. I think it should take me 2 days to to this -> 4 weeks. I think it should take me 1 week to do this -> 2 months. I think this should take us one month to accomplish -> 2 years.
- zbigniewc 7y ago
  Such approach assumes that literally all tasks are minefields full of unpredictability. But are they really? In my experience such reasoning and 'magnification' applies only to minority of cases, while most of the time you hopefully deal with well-enough understood domain and codebase to accurately estimate the effort based on the initial judgement.
  - Balgair 7y ago
    For me, it's more of a guestimating rule. Sure, some tasks take less than I'd expect them too. Sure, some take a LOT longer than even my guesstimate. But this helps me 'plan for drought and pray for rain', so to speak.
    - yebyen 7y ago
      To clarify why this strategy is not as bad or egregious as it seems, one must only ask: what are the consequences of overestimated time for work (that gets delivered early?) And what are the consequences of underestimating work? Would you rather deliver a product with finished surfaces or cut corners? Proper design and good overall test coverage, or zero budget for maintenance? It also helps to understand that management will ultimately decide that we need to do the project in half the time, and with these extra features you didn't ever plan on building. My budgeting strategy is to always assume that for whatever scope you have planned, 50% of the complexity that you will need to deliver in order to reach the finish line, remains unknown at any given stage of planning. Compensate upward if there still remain known unknowns.
hinkley 7y ago
You need to get an agreement with the team that you're going to amortize future upgrade treadmill activities across all work that is done. You can't mortgage future productivity to get short term gains when your project is supposed to run for 5, 10, or 20 years. You'll spend most of your time working slowly if you work that way.
- novaleaf 7y ago
  yours and a lot of other comments seem to hint that this work may not be strictly required? I'm a bit surprised. At least in my case, as a public facing SaaS I don't think running Node 6.x past it's LTS window is a valid choice, so better to bite the bullet now. If I went about doing the minimum work, it just adds more code/complexity that needs to be refactored when upgrading Node.
  - hinkley 7y ago
    No, it has to be done. But some people won't believe it until they've had a couple tough upgrades that show the cost of laxity. On one project there were CERT advisories out, and we had deferred upgrades due to breaking changes in those upgrades. All of a sudden we had to deal with the upgrade and a security issue at the same time. It was ugly. After the second time, we started putting at least one upgrade story per month on the work queue. Sometimes we let the engineer pick what they wanted to upgrade (just upgrade something!). Other times we picked the engineer for the work that needed to be done.
  - Techonomicon 7y ago
    People are hinting that it is not strictly required to get what was initially desired done. And that seems 100% true. You do not have to do everything at one time. You can easily to one thing, and then right after that, literally right after, do another. You could have gotten the proxy up and going, and then upgraded node. It's not about "doing the minimum work" it's about understanding the problem space, what can be split up, and how to attack the surface area efficiently. You're also introducing other problems that can occur by doing multiple things at the same time. Generally speaking I advocate my developers to be really good at separating problem spaces and attacking accordingly. It's really hard to take on multiple issues at one time and actually have high confidence in not breaking many systems along the way.
stickfigure 7y ago
While all platforms have some measure of this phenomenon, the update treadmill problem does seem to be especially acute in the javascript ecosystem. I update Java dependencies maybe once a year; it's usually painless and incompatibilities are highlighted by the type system. If you let your JS dependences get stale by a few months, you're looking at hours or days of work. And that doesn't count the massive shifts in build systems and frameworks that seem to happen annually.
- novaleaf 7y ago
  I've spent the last 5 years on Node, and honestly most modules are fine. The only ones I've had serious problems with are those created by Google. They keep taking huge complex dependencies on things like Grpc that break between Node versions, and keep churning their other library signatures (or deprecating entire modules). Maybe in the last year they've settled down, maybe not. I learned my lesson and don't take dependencies on them anymore. Much easier.
- tracker1 7y ago
  I just spend maybe half a day a month updating everything to the latest and testing. There are the occasional gotchas, but it's not so bad... When you leave a codebase to rot for a few years, it doesn't surprise me that it's more painful.

elemeno 7y ago

But in a case like yours if sounds like steps 2 through 5 aren't strictly relevant to the change you're making, so it's no wonder that your estimate is blown out the water. (Sorry if I'm wrong, you've obviously got more context on your own platform than I do!) It does make me wonder though if there is a trend of 'hiding' necessary maintenance work in larger tasks - if so that seems to me to be indicative of a larger problem around not being given/making the time to do those jobs as their own tasks.

hinkley 7y ago
Yes, there is. Probably the best four projects I worked on, there was collusion to spread out the cost of necessary maintenance across all work items. Because nobody will agree to doing maintenance stories (this is how Scrum hijacks developer ethics). Never ask permission to do something that must be done. Or to look at it another way, if no stories are maintenance stories, then all stories are maintenance stories.
- novaleaf 7y ago
  I used to be all about jumping on the next fancy thing as soon as it was (pre)released. Having been burnt and learned my lesson, now I delay until it's blindingly obvious it needs to be done. But yeah, if it needs t be done, do it.
  - Sharlin 7y ago
    Version x.y.0 of most software is "public beta" quality. x.y.1 is "release candidate". x.y.2 might be production-ready.

Balgair 7y ago

My own 'magnification' index is as such: Take whatever time I think it should take. Then double that number and go up one time unit. Exp: I think it should take me 3 hours this afternoon to do this -> 6 days. I think it should take me 2 days to to this -> 4 weeks. I think it should take me 1 week to do this -> 2 months. I think this should take us one month to accomplish -> 2 years.

zbigniewc 7y ago
Such approach assumes that literally all tasks are minefields full of unpredictability. But are they really? In my experience such reasoning and 'magnification' applies only to minority of cases, while most of the time you hopefully deal with well-enough understood domain and codebase to accurately estimate the effort based on the initial judgement.
- Balgair 7y ago
  For me, it's more of a guestimating rule. Sure, some tasks take less than I'd expect them too. Sure, some take a LOT longer than even my guesstimate. But this helps me 'plan for drought and pray for rain', so to speak.
  - yebyen 7y ago
    To clarify why this strategy is not as bad or egregious as it seems, one must only ask: what are the consequences of overestimated time for work (that gets delivered early?) And what are the consequences of underestimating work? Would you rather deliver a product with finished surfaces or cut corners? Proper design and good overall test coverage, or zero budget for maintenance? It also helps to understand that management will ultimately decide that we need to do the project in half the time, and with these extra features you didn't ever plan on building. My budgeting strategy is to always assume that for whatever scope you have planned, 50% of the complexity that you will need to deliver in order to reach the finish line, remains unknown at any given stage of planning. Compensate upward if there still remain known unknowns.

hinkley 7y ago

You need to get an agreement with the team that you're going to amortize future upgrade treadmill activities across all work that is done. You can't mortgage future productivity to get short term gains when your project is supposed to run for 5, 10, or 20 years. You'll spend most of your time working slowly if you work that way.

novaleaf 7y ago
yours and a lot of other comments seem to hint that this work may not be strictly required? I'm a bit surprised. At least in my case, as a public facing SaaS I don't think running Node 6.x past it's LTS window is a valid choice, so better to bite the bullet now. If I went about doing the minimum work, it just adds more code/complexity that needs to be refactored when upgrading Node.
- hinkley 7y ago
  No, it has to be done. But some people won't believe it until they've had a couple tough upgrades that show the cost of laxity. On one project there were CERT advisories out, and we had deferred upgrades due to breaking changes in those upgrades. All of a sudden we had to deal with the upgrade and a security issue at the same time. It was ugly. After the second time, we started putting at least one upgrade story per month on the work queue. Sometimes we let the engineer pick what they wanted to upgrade (just upgrade something!). Other times we picked the engineer for the work that needed to be done.
- Techonomicon 7y ago
  People are hinting that it is not strictly required to get what was initially desired done. And that seems 100% true. You do not have to do everything at one time. You can easily to one thing, and then right after that, literally right after, do another. You could have gotten the proxy up and going, and then upgraded node. It's not about "doing the minimum work" it's about understanding the problem space, what can be split up, and how to attack the surface area efficiently. You're also introducing other problems that can occur by doing multiple things at the same time. Generally speaking I advocate my developers to be really good at separating problem spaces and attacking accordingly. It's really hard to take on multiple issues at one time and actually have high confidence in not breaking many systems along the way.

stickfigure 7y ago

While all platforms have some measure of this phenomenon, the update treadmill problem does seem to be especially acute in the javascript ecosystem. I update Java dependencies maybe once a year; it's usually painless and incompatibilities are highlighted by the type system. If you let your JS dependences get stale by a few months, you're looking at hours or days of work. And that doesn't count the massive shifts in build systems and frameworks that seem to happen annually.

novaleaf 7y ago
I've spent the last 5 years on Node, and honestly most modules are fine. The only ones I've had serious problems with are those created by Google. They keep taking huge complex dependencies on things like Grpc that break between Node versions, and keep churning their other library signatures (or deprecating entire modules). Maybe in the last year they've settled down, maybe not. I learned my lesson and don't take dependencies on them anymore. Much easier.
tracker1 7y ago
I just spend maybe half a day a month updating everything to the latest and testing. There are the occasional gotchas, but it's not so bad... When you leave a codebase to rot for a few years, it doesn't surprise me that it's more painful.