5 ms·

So, what makes a payroll system at this scale so difficult? On the face of it, there are salary employees and there are hourly employees. How many taxing bodies

So, what makes a payroll system at this scale so difficult? On the face of it, there are salary employees and there are hourly employees. How many taxing bodies does Canada have? Federal, provincial, and municipal? Is that a few dozen or a few hundred? Is Canadian tax code crazy complicated that doesn't allow for a rules based system? I'm honestly curious as to how a "simple" payroll system can go so far off the rails.

threeseed 9y ago
FTA: ".. had yet to be encoded with many of the payroll complexities faced by a massive and diverse public service, such as unique shift rules for Coast Guard officers or prison guards."
BoorishBears 9y ago
I remember thinking “Why does this simple state Jury Duty site need to be down from 9PM to 9AM” “I could write a site that did this job with 99.9% reliability in a weekend” Turns out disability laws require a real person be available to assist with the site. Imagine bidding on a website project and accidentally promising 24/7 live support. That’s government tech in a nutshell.
Demoneeri 9y ago
There are multiple rules based on collective bargaining (union). "The software platform itself, a heavily modified version of Oracle’s PeopleSoft program, had yet to be encoded with many of the payroll complexities faced by a massive and diverse public service, such as unique shift rules for Coast Guard officers or prison guards."
deleted 9y ago
[deleted]
mkohler 9y ago
http://wiki.c2.com/?WhyIsPayrollHard http://wiki.c2.com/?WhyIsPayrollHard
- vidanay 9y ago
  That's.....amazing It almost gives the impression that an air traffic control system is actually easier (although also failed.)
  - fruzz 9y ago
    Both are incredibly complex (I am a developer on an air traffic control system.) There are thousands of requirements for voice communications alone.
- raverbashing 9y ago
  Interesting In the end, it seems a bit like "why is $social_network hard" I mean, why is it hard to show my my social graph? And the answer is: it isn't. The issue is building a system that deals with everybody's social graph. Because you have in your social feed one person that has few friends (easy), that other person that is followed by several people and they leave lots of comments (and you are loading their information when the post is shown - and of course there's no way of sharding that so your info can be read from one shard exclusively) and everybody has their list of blocked persons, other permissions than on top of that you have things like "algorithmic reordering", prioritization, paid promotions, etc
- emmelaich 9y ago
  Similarly, Gay marriage: the database engineering perspective https://qntm.org/gay https://qntm.org/gay
  - jessaustin 9y ago
    That article doesn't seem to exist at the moment. For the sake of discussion, let's imagine there was some DB schema somewhere that kept two separate WOMEN and MEN tables. Can we not agree that was always wrong? Gay marriage cannot possibly be the first situation in which that sort of decision caused extra work or inconvenience.
dragonwriter 9y ago
> So, what makes a payroll system at this scale so difficult? Complex pre-existing business rules of a large, sui generis employer (governments, especially sovereign ones rather than localities, often are because the laws that apply to other employees don't apply to them, and the rules that do apply often aren't coherent across-the-board policies determined by a central org focussed on HR, but often are set in special case laws applied to individual subunits of the government by a legislature then operationalized by the HR staff in that subunit) trying to be put in a COTS system make it hard, but probably what makes it most out of control is that the contractor is optimized in maximizing value extracted not efficient delivery and that governments generally suck at overseeing IT contracts (sometimes this is corruption, but it's often just lacking the knowledge and skills to do it effectively at the direct contract management staff level, often complicated by mandated IT contracting frameworks that are based on the underlying assumption that software engineering is basically structurally the same as civil engineering.)
- x0x0 9y ago
  It seems like lots of the payroll was being carried out by humans in local units. Humans are very good at holding lots of exceptions in our heads and, in the absence of clear rules, doing reasonable things. Computers are shit at both of the above. So things like Bob negotiated with his boss to leave early on Thursday to cover childcare and, in exchange, is coming in early on Tuesdays... but got asked to come in Saturday and now has overtime and what does he get paid based on province or local rules? Does OT end at midnight or not? Do employees get paid non-required OT as a courtesy? And frankly, lots of payroll probably sometimes violates the rules in minor ways, but as long as the employee is happy and the org is happy no-one is upset. Computers are not good at letting small things like that go. And that's a pretty trivial case. What about eg janitorial that is paid by multiple government orgs on some cost sharing arrangement because the orgs share office space, but is covering an event for a 3rd org in the 2nd org's space? And that janitor is on a particular contract, one of perhaps 5 or 6? If that has to get encoded into a central system, have fun with that. God only knows how many exceptions there are across a quarter of a million employees.
  - 9y ago
deleted 9y ago
[deleted]
j45 9y ago
Decades of different payment rules are anything but simple. Imagine people hired over the last 50 years. Each at a different rate, pay scale, union, non-union, some people in a position have certain perks, some don't, and it all varies from region to region, province to province, department to department of the government. I recently oversaw a payroll transformation project in Canada for an energy co at a smaller, but most complex scale according to the payroll vendor. A few key problem makers add to the payroll mess; - Accountants and their spreadsheets who think adding one little tweak here or there isn't a big deal. They create so much technical debt. It's not realized, until 10 or 20 simple tweaks on top of each other, and then have to interpret them relative to labor or union rules of each province. Accountants are not a vision of innovation. You will hear many just talk about having to do "manipulations" of data to get it how it should be and quickly losing vision that computers are good calculators and followers of instruction. - HR / Payroll practitioners do not have enough digital skills on average. Either they can't do the excel work and feel threatened, or do have the excel skills and feel threatened. Either way, change management is a constant threat. - No one knows how to successfully procure, implement and roll out software as a non-technical manager. Whether it's accounting, hr, payroll, sales, every position is becoming digital first, and the powers that be are not. It's a good case as to why CTO's are needed reporting to CEO's who get the big picture. Payroll othherwise, is a neat problem on it's own. In my case, the project ended up implementing a larger vendor of payroll that did a decent job that was saved by preparations we made at the CTO level. Before selecting a vendor, a payroll script capturing the rules for all employees in all regions to calculate everything was developed in-house. It took months of hardening at a few payrolls per month. Using this in-house payroll tool, and not wishing to update it, it was used to hold the screws to the payroll vendor if their output didn't match. Hilarity inevitably ensued. If someone built the stripe of payroll, it would be something.
- AndyNemmity 9y ago
  Let alone all the country specific rules. Just that is impossible to deal with properly.
  - j45 9y ago
xexers 9y ago
The unions have crazy complex pay rules. For example, in the private sector, if my boss goes away on a 2 week vacation, usually the boss might delegate someone to answer questions and possibly run meetings... but that delegated person doesn't get paid more. However, the public sector unions have negotiated that the delegated person get a pay bump for those 2 weeks only while the boss is away.
smnrchrds 9y ago
> How many taxing bodies does Canada have? Federal, provincial, and municipal? For payroll tax, only federal and provincial. Municipalities can only collect property tax, not sales or payroll tax. So, all in all, 14 tax jurisdictions (1 federal, 10 provincial, 3 territorial). If they were also handling payroll for diplomats and other Canadian employees in other countries, this number could be significantly higher.
draw_down 9y ago
I could build it in a weekend.
uhhhhhhh 9y ago
My understanding is its not taxing, its shift work and the thousands of edge cases across the various gov departments that have evolved their pay systems, contracts/union agreements separately
mikekchar 9y ago
The headline is misleading. According to the content of the article, the project has been going for 10 years and has spend $460 million. The government has pledged another $431 million in the future to deal with problems going forward. $460 million over 10 years is $46 million a year, which is still a lot of money, but if you break it down it's not nearly so mystifying. If all the money is being spent by contractors, we could assume that the loaded labour rate of the contractors is about $200K per year. It's been more than a decade since I was in Canada, but I'm guessing this is not an outrageous rate. That breaks down to 230 people working on the project. In truth, the payroll system seems to have been operational for the last 2 years and the system employs 500 employees (it was supposed to replace 1200 employees, thus saving money, but there is no indication if those 1200 were, indeed, replaced). I'm guessing that the $460 million includes those 500 employees for at least 2 years and may also include capital costs to house them (the reporter may not be interested in depreciating capital assets in their report). So it's hard to say exactly how many contractors worked on this, but it's probably safe to say that there were more than 100. A government project with more than 100 contractors == not possible to succeed in my experience. Just the communication issues alone would be insurmountable. As time wore on and pressure to put something into use grew, the mistakes would have piled up causing absolute mayhem. Unfortunately this is an all too common occurrence. If you go to any meetup in Ottawa and talk to government contractors I'm sure there will be a couple wiping their brow and saying, "This is a stroke of luck because it will take some of the heat off our our project that is burning even more money".

threeseed 9y ago

FTA: ".. had yet to be encoded with many of the payroll complexities faced by a massive and diverse public service, such as unique shift rules for Coast Guard officers or prison guards."

BoorishBears 9y ago

I remember thinking “Why does this simple state Jury Duty site need to be down from 9PM to 9AM” “I could write a site that did this job with 99.9% reliability in a weekend” Turns out disability laws require a real person be available to assist with the site. Imagine bidding on a website project and accidentally promising 24/7 live support. That’s government tech in a nutshell.

Demoneeri 9y ago

There are multiple rules based on collective bargaining (union). "The software platform itself, a heavily modified version of Oracle’s PeopleSoft program, had yet to be encoded with many of the payroll complexities faced by a massive and diverse public service, such as unique shift rules for Coast Guard officers or prison guards."

deleted 9y ago

[deleted]

mkohler 9y ago

http://wiki.c2.com/?WhyIsPayrollHard http://wiki.c2.com/?WhyIsPayrollHard

vidanay 9y ago
That's.....amazing It almost gives the impression that an air traffic control system is actually easier (although also failed.)
- fruzz 9y ago
  Both are incredibly complex (I am a developer on an air traffic control system.) There are thousands of requirements for voice communications alone.
raverbashing 9y ago
Interesting In the end, it seems a bit like "why is $social_network hard" I mean, why is it hard to show my my social graph? And the answer is: it isn't. The issue is building a system that deals with everybody's social graph. Because you have in your social feed one person that has few friends (easy), that other person that is followed by several people and they leave lots of comments (and you are loading their information when the post is shown - and of course there's no way of sharding that so your info can be read from one shard exclusively) and everybody has their list of blocked persons, other permissions than on top of that you have things like "algorithmic reordering", prioritization, paid promotions, etc
emmelaich 9y ago
Similarly, Gay marriage: the database engineering perspective https://qntm.org/gay https://qntm.org/gay
- jessaustin 9y ago
  That article doesn't seem to exist at the moment. For the sake of discussion, let's imagine there was some DB schema somewhere that kept two separate WOMEN and MEN tables. Can we not agree that was always wrong? Gay marriage cannot possibly be the first situation in which that sort of decision caused extra work or inconvenience.

dragonwriter 9y ago

> So, what makes a payroll system at this scale so difficult? Complex pre-existing business rules of a large, sui generis employer (governments, especially sovereign ones rather than localities, often are because the laws that apply to other employees don't apply to them, and the rules that do apply often aren't coherent across-the-board policies determined by a central org focussed on HR, but often are set in special case laws applied to individual subunits of the government by a legislature then operationalized by the HR staff in that subunit) trying to be put in a COTS system make it hard, but probably what makes it most out of control is that the contractor is optimized in maximizing value extracted not efficient delivery and that governments generally suck at overseeing IT contracts (sometimes this is corruption, but it's often just lacking the knowledge and skills to do it effectively at the direct contract management staff level, often complicated by mandated IT contracting frameworks that are based on the underlying assumption that software engineering is basically structurally the same as civil engineering.)

x0x0 9y ago
It seems like lots of the payroll was being carried out by humans in local units. Humans are very good at holding lots of exceptions in our heads and, in the absence of clear rules, doing reasonable things. Computers are shit at both of the above. So things like Bob negotiated with his boss to leave early on Thursday to cover childcare and, in exchange, is coming in early on Tuesdays... but got asked to come in Saturday and now has overtime and what does he get paid based on province or local rules? Does OT end at midnight or not? Do employees get paid non-required OT as a courtesy? And frankly, lots of payroll probably sometimes violates the rules in minor ways, but as long as the employee is happy and the org is happy no-one is upset. Computers are not good at letting small things like that go. And that's a pretty trivial case. What about eg janitorial that is paid by multiple government orgs on some cost sharing arrangement because the orgs share office space, but is covering an event for a 3rd org in the 2nd org's space? And that janitor is on a particular contract, one of perhaps 5 or 6? If that has to get encoded into a central system, have fun with that. God only knows how many exceptions there are across a quarter of a million employees.
- 9y ago

deleted 9y ago

[deleted]

j45 9y ago

Decades of different payment rules are anything but simple. Imagine people hired over the last 50 years. Each at a different rate, pay scale, union, non-union, some people in a position have certain perks, some don't, and it all varies from region to region, province to province, department to department of the government. I recently oversaw a payroll transformation project in Canada for an energy co at a smaller, but most complex scale according to the payroll vendor. A few key problem makers add to the payroll mess; - Accountants and their spreadsheets who think adding one little tweak here or there isn't a big deal. They create so much technical debt. It's not realized, until 10 or 20 simple tweaks on top of each other, and then have to interpret them relative to labor or union rules of each province. Accountants are not a vision of innovation. You will hear many just talk about having to do "manipulations" of data to get it how it should be and quickly losing vision that computers are good calculators and followers of instruction. - HR / Payroll practitioners do not have enough digital skills on average. Either they can't do the excel work and feel threatened, or do have the excel skills and feel threatened. Either way, change management is a constant threat. - No one knows how to successfully procure, implement and roll out software as a non-technical manager. Whether it's accounting, hr, payroll, sales, every position is becoming digital first, and the powers that be are not. It's a good case as to why CTO's are needed reporting to CEO's who get the big picture. Payroll othherwise, is a neat problem on it's own. In my case, the project ended up implementing a larger vendor of payroll that did a decent job that was saved by preparations we made at the CTO level. Before selecting a vendor, a payroll script capturing the rules for all employees in all regions to calculate everything was developed in-house. It took months of hardening at a few payrolls per month. Using this in-house payroll tool, and not wishing to update it, it was used to hold the screws to the payroll vendor if their output didn't match. Hilarity inevitably ensued. If someone built the stripe of payroll, it would be something.

AndyNemmity 9y ago
Let alone all the country specific rules. Just that is impossible to deal with properly.
- j45 9y ago

xexers 9y ago

The unions have crazy complex pay rules. For example, in the private sector, if my boss goes away on a 2 week vacation, usually the boss might delegate someone to answer questions and possibly run meetings... but that delegated person doesn't get paid more. However, the public sector unions have negotiated that the delegated person get a pay bump for those 2 weeks only while the boss is away.

smnrchrds 9y ago

> How many taxing bodies does Canada have? Federal, provincial, and municipal? For payroll tax, only federal and provincial. Municipalities can only collect property tax, not sales or payroll tax. So, all in all, 14 tax jurisdictions (1 federal, 10 provincial, 3 territorial). If they were also handling payroll for diplomats and other Canadian employees in other countries, this number could be significantly higher.

draw_down 9y ago

I could build it in a weekend.

uhhhhhhh 9y ago

My understanding is its not taxing, its shift work and the thousands of edge cases across the various gov departments that have evolved their pay systems, contracts/union agreements separately

mikekchar 9y ago

The headline is misleading. According to the content of the article, the project has been going for 10 years and has spend $460 million. The government has pledged another $431 million in the future to deal with problems going forward. $460 million over 10 years is $46 million a year, which is still a lot of money, but if you break it down it's not nearly so mystifying. If all the money is being spent by contractors, we could assume that the loaded labour rate of the contractors is about $200K per year. It's been more than a decade since I was in Canada, but I'm guessing this is not an outrageous rate. That breaks down to 230 people working on the project. In truth, the payroll system seems to have been operational for the last 2 years and the system employs 500 employees (it was supposed to replace 1200 employees, thus saving money, but there is no indication if those 1200 were, indeed, replaced). I'm guessing that the $460 million includes those 500 employees for at least 2 years and may also include capital costs to house them (the reporter may not be interested in depreciating capital assets in their report). So it's hard to say exactly how many contractors worked on this, but it's probably safe to say that there were more than 100. A government project with more than 100 contractors == not possible to succeed in my experience. Just the communication issues alone would be insurmountable. As time wore on and pressure to put something into use grew, the mistakes would have piled up causing absolute mayhem. Unfortunately this is an all too common occurrence. If you go to any meetup in Ottawa and talk to government contractors I'm sure there will be a couple wiping their brow and saying, "This is a stroke of luck because it will take some of the heat off our our project that is burning even more money".