4 ms·

> nothing gets shipped unless it's perfect, and so nothing gets iterated on… That’s why a solid, painless, quick rollback story is absolutely vital but underra

by teeray 3y ago

> nothing gets shipped unless it's perfect, and so nothing gets iterated on… That’s why a solid, painless, quick rollback story is absolutely vital but underrated for agility. It’s not a big deal if you ship a regression but can roll back quickly. On the other hand, if every release is playing for keeps, you’ll slow down to a glacial pace to try to ship perfection. But perfection is impossible no matter how much process you put in front of it—you will ship a bug some day. So pretending like you can’t ship bugs because you have initial design reviews, multiple code reviews, sign-offs from multiple stakeholders, CI that takes 3 days to run, etc. just makes the day you do ship a big that much more painful.

cogman10 3y ago
The issue is that big messes can be made that are hard to clean up in one small release. For example, one of our teams had the requirement "Allow end users to programmatically configure widget xyz". Now, widget xyz has a janky design and a table with 50 columns associated with it. So what did this team do? They took that internal table definition, made a CRUD api that exposes the whole thing, and moved on. Tell me, how do you fix something like that like that? They followed the agile route, got an MVP out with hardly any effort on their part. But now we have this API that shows off 20 years of legacy that our end users now depend on. And to be clear, of those 50 fields, maybe 3 of them mattered to the end user and the rest could have been inferred. But doing that inference is a lot more work as it takes understanding the domain.
- brailsafe 3y ago
  Well I suppose that's the difference between thoughtful systems and process and zero systems and process, and also poor problem definition. There is some nuance between "ship literally anything to end users" and "ship an improvement" or something else to end users, or to a beta that roles out to end users and isolates the possible regression to a smaller surface area. It also seems like those developers might either suck (possible, but uncharitable), or that the system rewards the wrong things. But much of the time, better or worse laters of bureaucracy in the system come about from these mistakes which will eventually happen, so it's a very valid point, and I do think a lot of the time it's companies building up scar tissue and not being capable of getting the necessary surgery to fix it.
  - cogman10 3y ago
    > It also seems like those developers might either suck (possible, but uncharitable), or that the system rewards the wrong things. Yes, they do, but also it's an entire department problem and one that the upper bureaucracy doesn't see as an issue. I'm really not surprised this was the route that particular department took. Office politics are fun, though, so sucking at your job often doesn't mean anything if your managers are all buddies. But before the creation/evolution of that department we already had these sorts of problems. This may just be a "just my company" problem but I don't think it is. After all, from what I know of business management, this sort of "extra staff and rework is waste" mentality seems really pervasive.
    - brailsafe 3y ago
      Yep, that all sounds quite familiar, definitely not just your company, it's characteristic of trying to monkey-patch whatever management can point a finger at, because that's what they're rewarded for, and that's what their manager is rewarded for. No word of a lie, I ran into a colleague from the first company I saw this problem at (and was fired from) years later when I happened to revisit a restaurant I liked in the area. I asked him if he still worked there, and he laughed. Said that they eventually fired the entire department and he stayed on just as a consultant for a bit. Which is odd, because although I was a naive early-twenties kid at the time, I could tell everyone else in that department was quite good at their jobs.
- lmm 3y ago
  > Tell me, how do you fix something like that like that? A little at a time; continuous refactoring/improvement is a vital pillar of agile. You've got 50 redundant fields that can be inferred but you have to understand the domain to do that? Well, understand the domain a bit, and infer one of them. Then next time you come back to that part of the code do another one. (I mean, if the table was like that already then it sounds like the API didn't introduce a new problem, it just surfaced a problem you already had)
  - cogman10 3y ago
    > continuous refactoring/improvement is a vital pillar of agile. The problem with this pillar is it's often viewed as nothing more than waste by business. Try arguing with a PM "Hey, we need to actually take a bit of time to redo XYZ because it's really janky" and they'll come back with "Yes, but client PDQ is ready to give us money for this shiny new feature! We don't have time to waste redoing XYZ because it's not currently causing us any problems". The table going from 1, 2, 10, 20, 50 columns never actually hurt the system. It got nasty looking but ultimately not something you could sell to a manager "Hey, we should address this". So, by the time "Hey, let's expose this" rolled around the institutional will to want to correct anything had long since past. It's really easy to sell redoing something if it's actively causing harm to the systems, but stuff that's just a bit janky has next to 0 priority (often even negative priority because fixing it might break something).
    - lmm 3y ago
      There's a balance to be had - and frankly most code has a finite lifetime and developers often put too much priority on making it cleaner than it needs to be. The technical debt metaphor is a good one; a PM should understand that when code isn't clean it increases the cost of making changes to that code, and sometimes that's a price you want to pay and sometimes you don't.
- wmorgan 3y ago
  Api longa, sprinta brevis
- lelanthran 3y ago
  > For example, one of our teams had the requirement "Allow end users to programmatically configure widget xyz". Now, widget xyz has a janky design and a table with 50 columns associated with it. > So what did this team do? They took that internal table definition, made a CRUD api that exposes the whole thing, and moved on. > Tell me, how do you fix something like that like that? That's genius! You can now use your metrics from that API to determine how the users are actually using it to provide more value. Are they hooking into something they shouldn't? Well then they are missing a feature and that is an opportunity for you to build that feature for them.
- JackFr 3y ago
  > "Allow end users to programmatically configure widget xyz" That's an extremely poorly defined requirement, and I would say that's the ultimate source of the issue. Your team took that and made a crud api. Another team might have written a YAML based DSL. Another team might have created an even more complex scheme. A BA/PM should have pushed back on that and gotten much, much better specificity as to what the users needed to do. > And to be clear, of those 50 fields, maybe 3 of them mattered to the end user and the rest could have been inferred. But doing that inference is a lot more work as it takes understanding the domain. Exactly -- except the part about "doing that inference is a lot more work". Compared to recovering from the wrong path the software has taken?

by teeray 3y ago

cogman10 3y ago

The issue is that big messes can be made that are hard to clean up in one small release. For example, one of our teams had the requirement "Allow end users to programmatically configure widget xyz". Now, widget xyz has a janky design and a table with 50 columns associated with it. So what did this team do? They took that internal table definition, made a CRUD api that exposes the whole thing, and moved on. Tell me, how do you fix something like that like that? They followed the agile route, got an MVP out with hardly any effort on their part. But now we have this API that shows off 20 years of legacy that our end users now depend on. And to be clear, of those 50 fields, maybe 3 of them mattered to the end user and the rest could have been inferred. But doing that inference is a lot more work as it takes understanding the domain.

brailsafe 3y ago
Well I suppose that's the difference between thoughtful systems and process and zero systems and process, and also poor problem definition. There is some nuance between "ship literally anything to end users" and "ship an improvement" or something else to end users, or to a beta that roles out to end users and isolates the possible regression to a smaller surface area. It also seems like those developers might either suck (possible, but uncharitable), or that the system rewards the wrong things. But much of the time, better or worse laters of bureaucracy in the system come about from these mistakes which will eventually happen, so it's a very valid point, and I do think a lot of the time it's companies building up scar tissue and not being capable of getting the necessary surgery to fix it.
- cogman10 3y ago
  > It also seems like those developers might either suck (possible, but uncharitable), or that the system rewards the wrong things. Yes, they do, but also it's an entire department problem and one that the upper bureaucracy doesn't see as an issue. I'm really not surprised this was the route that particular department took. Office politics are fun, though, so sucking at your job often doesn't mean anything if your managers are all buddies. But before the creation/evolution of that department we already had these sorts of problems. This may just be a "just my company" problem but I don't think it is. After all, from what I know of business management, this sort of "extra staff and rework is waste" mentality seems really pervasive.
  - brailsafe 3y ago
    Yep, that all sounds quite familiar, definitely not just your company, it's characteristic of trying to monkey-patch whatever management can point a finger at, because that's what they're rewarded for, and that's what their manager is rewarded for. No word of a lie, I ran into a colleague from the first company I saw this problem at (and was fired from) years later when I happened to revisit a restaurant I liked in the area. I asked him if he still worked there, and he laughed. Said that they eventually fired the entire department and he stayed on just as a consultant for a bit. Which is odd, because although I was a naive early-twenties kid at the time, I could tell everyone else in that department was quite good at their jobs.
lmm 3y ago
> Tell me, how do you fix something like that like that? A little at a time; continuous refactoring/improvement is a vital pillar of agile. You've got 50 redundant fields that can be inferred but you have to understand the domain to do that? Well, understand the domain a bit, and infer one of them. Then next time you come back to that part of the code do another one. (I mean, if the table was like that already then it sounds like the API didn't introduce a new problem, it just surfaced a problem you already had)
- cogman10 3y ago
  > continuous refactoring/improvement is a vital pillar of agile. The problem with this pillar is it's often viewed as nothing more than waste by business. Try arguing with a PM "Hey, we need to actually take a bit of time to redo XYZ because it's really janky" and they'll come back with "Yes, but client PDQ is ready to give us money for this shiny new feature! We don't have time to waste redoing XYZ because it's not currently causing us any problems". The table going from 1, 2, 10, 20, 50 columns never actually hurt the system. It got nasty looking but ultimately not something you could sell to a manager "Hey, we should address this". So, by the time "Hey, let's expose this" rolled around the institutional will to want to correct anything had long since past. It's really easy to sell redoing something if it's actively causing harm to the systems, but stuff that's just a bit janky has next to 0 priority (often even negative priority because fixing it might break something).
  - lmm 3y ago
    There's a balance to be had - and frankly most code has a finite lifetime and developers often put too much priority on making it cleaner than it needs to be. The technical debt metaphor is a good one; a PM should understand that when code isn't clean it increases the cost of making changes to that code, and sometimes that's a price you want to pay and sometimes you don't.
wmorgan 3y ago
Api longa, sprinta brevis
lelanthran 3y ago
> For example, one of our teams had the requirement "Allow end users to programmatically configure widget xyz". Now, widget xyz has a janky design and a table with 50 columns associated with it. > So what did this team do? They took that internal table definition, made a CRUD api that exposes the whole thing, and moved on. > Tell me, how do you fix something like that like that? That's genius! You can now use your metrics from that API to determine how the users are actually using it to provide more value. Are they hooking into something they shouldn't? Well then they are missing a feature and that is an opportunity for you to build that feature for them.
JackFr 3y ago
> "Allow end users to programmatically configure widget xyz" That's an extremely poorly defined requirement, and I would say that's the ultimate source of the issue. Your team took that and made a crud api. Another team might have written a YAML based DSL. Another team might have created an even more complex scheme. A BA/PM should have pushed back on that and gotten much, much better specificity as to what the users needed to do. > And to be clear, of those 50 fields, maybe 3 of them mattered to the end user and the rest could have been inferred. But doing that inference is a lot more work as it takes understanding the domain. Exactly -- except the part about "doing that inference is a lot more work". Compared to recovering from the wrong path the software has taken?