4 ms·

Not having ever worked on anything like this, the death knell for these projects must be the inability to ever reduce functionality into a reasonable core set.

by zaroth 9y ago

Not having ever worked on anything like this, the death knell for these projects must be the inability to ever reduce functionality into a reasonable core set. Basically the 80/20 rule but where there are practically an infinite number of edge cases that stretch the project on indefinitely? You could serve 80% of the payroll burden with software that cost $10m. 90% coverage costs $100m. 98% costs $1,000m. And 100% costs $∞

microcolonel 9y ago
Also there's a good chance that the decision to go with IBM as a vendor, and the conduct thereafter, was corrupt from the start. That would go a long way toward explaining why so little due diligence was observed in assessing whether or not to continue the project with them at any of the stages it could've been. Yes, people do make huge blunders in powering forward with overpowered, unnecessary solutions to problems they largely do not have; but blunders this big usually don't take this long to suss out.
- saas_co_de 9y ago
  > blunders this big usually don't take this long to suss out This is actually pretty minor league compared to U.S. government failures. e.g. https://www.wired.com/story/us-border-patrol-hasnt-validated-e-passport-data-for-years/ https://www.wired.com/story/us-border-patrol-hasnt-validated...
  - microcolonel 9y ago
    Idunno, that doesn't really seem like such a big deal. The whole idea was rather pointless (in the face of all the bigger fish they have to fry) to begin with. With such headed, small minded planning, it may have been a blessing if they never bothered to implement it.
- AndyNemmity 9y ago
  At the same time, "Nobody got fired for choosing IBM" is a statement for a reason. If you hire someone else, people will say, "Why didn't you hire the best?" If you hire IBM, you're avoiding that reasoning, even if on any given project IBM or other large enterprise vendors may not be the best.
trhway 9y ago
>You could serve 80% of the payroll burden with software that cost $10m. 90% coverage costs $100m. 98% costs $1,000m. And 100% costs $∞ this is where crucial distinction and choice comes - enterprise software which supposedly have enough flexibility and configurability to cover all the cases or the software which allows and requires a bunch of custom code. The former is a monster and the latter requires good tech chops. And choosing the software with the right balance between what can be configured and what would have to be developed requires deep tech wisdom and experience on the part of the leadership which is usually a very tall order...
chime 9y ago
Having worked on similar things on a smaller scale but higher up the chain, I can tell you the list of possible issues is a mile long. Everything from lack of leadership and conflicting goals to ever-changing scopes and insufficient resources is a candidate for the big blame. A lot of people, organizations, and systems have to work well together to succeed. As techies, we think the technical solution is the most superior solution but that is not always the case. Here is an example of something that just happened to me 2 hours ago today. User called and said osTicket system I manage treats '0' as blank. So when they fill out a form with question 'How many X happened today?' and they enter '0', the system just treats the field as unfilled and does not print '0'. That's a big problem for auditing because they absolutely need to be able to show that it was '0'. I started digging into the source code to see where '0' int was treated as a blank. osTicket being PHP where 0, '', null, and false can all be treated as FALSE, it could've been 8+ hours worth of work digging into the right piece of code to figure out the fix. So I called the user up and we discussed the best way to handle this. In the end, I added a new question above this one: 'Did one or more X happen today? YES/NO'. If it's 'NO', it doesn't matter the 'How many X happened today?' with '0' doesn't get printed. To make a major implementation work, this kind of discussion needs to happen 10000x a day among 1000+ people, many of whom may not have the authority to just decide things on the fly.
- AndyNemmity 9y ago
  And, often times that discussion leads to a decision maker saying - that shouldn't be the case, it's your problem, not ours that you can't handle that. So you end up assigning someone to 8+ hours of work to do it, you also have to end up having 2-4 meetings on the topic, and on and on.
  - cm2187 9y ago
    8 hours of work to fix a small issue like this one is just bad design. In this case using php.
    - AndyNemmity 9y ago
      You're right, but at the same time, it's a totally unhelpful statement in a project. Everything is bad design everywhere, at least when you're a consultant. You're generally not brought into projects that have good design, and everything's roses. I've been on a few projects with amazing design, and it was a very different experience. I didn't even argue for more work for them, because they had it all under control. They essentially, wanted to be confirmed how great they were. I don't know if the CIO was curious if it was true, or if they all just wanted pats on the back, but we taught them a few really detailed things they didn't know, helped on a few really tricky edge cases, and said you're golden, keep killing it.
rhizome 9y ago
Basically the 80/20 rule but where there are practically an infinite number of edge cases that stretch the project on indefinitely? Zeno's Featureset
- jamiepenney 9y ago
  That's the perfect way to put it! Definitely going to steal this for later use.
mseebach 9y ago
> You could serve 80% of the payroll burden with software that cost $10m. 90% coverage costs $100m. 98% costs $1,000m. And 100% costs $∞ I read something in a trade journal probably probably about 15 years ago, that really stuck with me: When businesses buy big software solutions (I think the article was about ERP systems), they should prepare to adjust and change their processes (within reason, of course) and not just insist that the software be modified endlessly to suit every little ossified process and preference. Not just because it's very expensive, but because the resulting system will be brittle and incompatible and taking advantage of further developments. Too many buyers of software would benefit immensely from taking a good, hard look at those remaining 20% and making some hard decisions about what is actually important. Sure, for a government payroll system, there is limited room for movement, but I'd suspect that the majority of the long tail are people on weird, custom contracts and the correct solution is actually to hire a room full of clerks to handle these in Excel - and/or applying political pressure on the hiring parties to get their contracts adjusted to fit standard payroll schemes.
- 1024core 9y ago
  > Not just because it's very expensive, but because the resulting system will be brittle and incompatible and taking advantage of further developments. But that's where the lure of the IBMs of this world comes in: they claim no change will be required, it's all free. Of course, we know that eventually it'll be a big fuckup.
- greenshackle3 9y ago
  Having worked on ERP software for medium businesses, that sounds spot-on to me; in my experience the biggest challenge was not so much technical as getting clients to change their processes so they can be systematized in a sane way. There is a lot of resistance from employees not willing to change the way they work. Even seemingly simple things like standardizing language / glossary between departments so we can have a single representation in the system can be an uphill battle.
  - azernik 9y ago
    Same for me, working on MRP-ish software for small businesses.

by zaroth 9y ago

microcolonel 9y ago

Also there's a good chance that the decision to go with IBM as a vendor, and the conduct thereafter, was corrupt from the start. That would go a long way toward explaining why so little due diligence was observed in assessing whether or not to continue the project with them at any of the stages it could've been. Yes, people do make huge blunders in powering forward with overpowered, unnecessary solutions to problems they largely do not have; but blunders this big usually don't take this long to suss out.

saas_co_de 9y ago
> blunders this big usually don't take this long to suss out This is actually pretty minor league compared to U.S. government failures. e.g. https://www.wired.com/story/us-border-patrol-hasnt-validated-e-passport-data-for-years/ https://www.wired.com/story/us-border-patrol-hasnt-validated...
- microcolonel 9y ago
  Idunno, that doesn't really seem like such a big deal. The whole idea was rather pointless (in the face of all the bigger fish they have to fry) to begin with. With such headed, small minded planning, it may have been a blessing if they never bothered to implement it.
AndyNemmity 9y ago
At the same time, "Nobody got fired for choosing IBM" is a statement for a reason. If you hire someone else, people will say, "Why didn't you hire the best?" If you hire IBM, you're avoiding that reasoning, even if on any given project IBM or other large enterprise vendors may not be the best.

trhway 9y ago

>You could serve 80% of the payroll burden with software that cost $10m. 90% coverage costs $100m. 98% costs $1,000m. And 100% costs $∞ this is where crucial distinction and choice comes - enterprise software which supposedly have enough flexibility and configurability to cover all the cases or the software which allows and requires a bunch of custom code. The former is a monster and the latter requires good tech chops. And choosing the software with the right balance between what can be configured and what would have to be developed requires deep tech wisdom and experience on the part of the leadership which is usually a very tall order...

chime 9y ago

Having worked on similar things on a smaller scale but higher up the chain, I can tell you the list of possible issues is a mile long. Everything from lack of leadership and conflicting goals to ever-changing scopes and insufficient resources is a candidate for the big blame. A lot of people, organizations, and systems have to work well together to succeed. As techies, we think the technical solution is the most superior solution but that is not always the case. Here is an example of something that just happened to me 2 hours ago today. User called and said osTicket system I manage treats '0' as blank. So when they fill out a form with question 'How many X happened today?' and they enter '0', the system just treats the field as unfilled and does not print '0'. That's a big problem for auditing because they absolutely need to be able to show that it was '0'. I started digging into the source code to see where '0' int was treated as a blank. osTicket being PHP where 0, '', null, and false can all be treated as FALSE, it could've been 8+ hours worth of work digging into the right piece of code to figure out the fix. So I called the user up and we discussed the best way to handle this. In the end, I added a new question above this one: 'Did one or more X happen today? YES/NO'. If it's 'NO', it doesn't matter the 'How many X happened today?' with '0' doesn't get printed. To make a major implementation work, this kind of discussion needs to happen 10000x a day among 1000+ people, many of whom may not have the authority to just decide things on the fly.

AndyNemmity 9y ago
And, often times that discussion leads to a decision maker saying - that shouldn't be the case, it's your problem, not ours that you can't handle that. So you end up assigning someone to 8+ hours of work to do it, you also have to end up having 2-4 meetings on the topic, and on and on.
- cm2187 9y ago
  8 hours of work to fix a small issue like this one is just bad design. In this case using php.
  - AndyNemmity 9y ago
    You're right, but at the same time, it's a totally unhelpful statement in a project. Everything is bad design everywhere, at least when you're a consultant. You're generally not brought into projects that have good design, and everything's roses. I've been on a few projects with amazing design, and it was a very different experience. I didn't even argue for more work for them, because they had it all under control. They essentially, wanted to be confirmed how great they were. I don't know if the CIO was curious if it was true, or if they all just wanted pats on the back, but we taught them a few really detailed things they didn't know, helped on a few really tricky edge cases, and said you're golden, keep killing it.

rhizome 9y ago

Basically the 80/20 rule but where there are practically an infinite number of edge cases that stretch the project on indefinitely? Zeno's Featureset

jamiepenney 9y ago
That's the perfect way to put it! Definitely going to steal this for later use.

mseebach 9y ago

> You could serve 80% of the payroll burden with software that cost $10m. 90% coverage costs $100m. 98% costs $1,000m. And 100% costs $∞ I read something in a trade journal probably probably about 15 years ago, that really stuck with me: When businesses buy big software solutions (I think the article was about ERP systems), they should prepare to adjust and change their processes (within reason, of course) and not just insist that the software be modified endlessly to suit every little ossified process and preference. Not just because it's very expensive, but because the resulting system will be brittle and incompatible and taking advantage of further developments. Too many buyers of software would benefit immensely from taking a good, hard look at those remaining 20% and making some hard decisions about what is actually important. Sure, for a government payroll system, there is limited room for movement, but I'd suspect that the majority of the long tail are people on weird, custom contracts and the correct solution is actually to hire a room full of clerks to handle these in Excel - and/or applying political pressure on the hiring parties to get their contracts adjusted to fit standard payroll schemes.

1024core 9y ago
> Not just because it's very expensive, but because the resulting system will be brittle and incompatible and taking advantage of further developments. But that's where the lure of the IBMs of this world comes in: they claim no change will be required, it's all free. Of course, we know that eventually it'll be a big fuckup.
greenshackle3 9y ago
Having worked on ERP software for medium businesses, that sounds spot-on to me; in my experience the biggest challenge was not so much technical as getting clients to change their processes so they can be systematized in a sane way. There is a lot of resistance from employees not willing to change the way they work. Even seemingly simple things like standardizing language / glossary between departments so we can have a single representation in the system can be an uphill battle.
- azernik 9y ago
  Same for me, working on MRP-ish software for small businesses.