7 ms·

We know exactly how to fix it. Our cowardly politicians and toothless regulatory agencies are not up for the challenge. For every piece of software that can di

We know exactly how to fix it. Our cowardly politicians and toothless regulatory agencies are not up for the challenge. For every piece of software that can directly and materially harm someone's life like this, there should be a chain of responsibility. And within that chain, there should be legal recourse and, in most cases, penal consequences, especially in the case of inadequate software quality/testing/validation, should the software fail to perform its task correctly. Bonus side effect, software quality will go up across the board in the industry.

bendbro 6y ago
Nobody will take that contract. Any mistake in the government specifying what they want you to build can be politically manipulated into a legal matter with some probability you'll end up in prison.
- AdrianB1 6y ago
  Not at all: any mistake in the government specifications is the government's problem. If you have requirements and a requirement tracing matrix to deliverables and you deliver what is required, you are good. But usually in most parts of the world these government contracts are intentionally made ambiguous for so many reasons including corruption and incompetence of the people writing requirements.
frompdx 6y ago
My cynical take is that the lack of accountability is exactly what makes software enabled bureaucracy so appealing. If this true, there is no incentive to change.
- rbanffy 6y ago
  It's a cost/benefit analysis. They'll be taken to court by a few inmates, have to pay them or settle out of court and bill the government for their stays. Since everything is a matter of money, what the company will do is what brings them the most profit after they paid their legal bills. It seems keeping inmates longer pays better than releasing them on time.
- spaetzleesser 6y ago
  This is part of it. Otherwise it’s just way cheaper to run a faceless bureaucracy that sometimes throws people under the bus when a mistake has been made.
kingaillas 6y ago
Yeah but the cost of that chain will also rise. If I'm (or my company is) personally on the hook for bugs, then I'm going to adopt a NASA-like software quality regimen, pushing up the cost of the product. Every single part of the software stack below me, from hardware, OS, compiler toolchain, disavows responsibility so if I have to absorb all the risk, the product is going to be mind bogglingly expensive.
- ryanong 6y ago
  that doesn't sound bad to me if we are dealing with people's life or people's freedoms
- Draiken 6y ago
  I have to say that sounds exactly how this kind of software should be built. We're not talking about the newest social media hype. This software actually matters. Specially since today most of these bureaucratic processes can't be done without these softwares.
  - throwaway0a5e 6y ago
    >I have to say that sounds exactly how this kind of software should be built. You see this in every topic. Every "muh pride in muh trade" person says something like this about the relevant trade but the fact of the matter is that the world runs on off-brand duct tape, harbor freight tools, walmart jeans, economy tires, and all sorts of other "value" solutions and the race to the bottom is what has given us much of the modern world that we take for granted. A balance needs to be struck. And it generally needs to be struck further toward the "quickly and cheaply build it like crap but make it easy to override or reset" portion of the available solution space than anyone pontificating about quality on the internet will readily admit.
    - handoflixue 6y ago
      We don't say "perfection is impossible" when it comes to bridges collapsing. We understand that yes, on rare occasion a bridge WILL collapse, but we go and find the people responsible, and we still hold them accountable. This is a level of accountability that basically every other field of engineering is held to, and they've all risen to the challenge and left the "off-brand duct tape" behind. Even within programming, planes don't fall out of the sky daily, so I feel safe assume the aerospace programmers are comfortable working with a high degree of responsibility. High speed traders are dealing with million-dollar stakes and a single mistake can make the news. I'd expect they've got a very accountable culture where people get fired when that happens. There are costs, yes, but there's also costs to keep 733 people illegally imprisoned - we're talking two man-years of peoples lives lost every DAY this goes on.
varenc 6y ago
If there’s penal consequences for bad software, you can bet that the development cost will easily 10x overnight.
- deleted 6y ago
  [deleted]
- oftenwrong 6y ago
  Is there a better option? This is software that has to 100% work in order to be trusted. If the software cannot be trusted, additional process has to fill in to verify the decisions the software is making. There's no way to deliver a good solution cheaply.
  - TedDoesntTalk 6y ago
    All software has bugs. This is the mantra we are taught and for a good reason. The answer is human oversight.
- TedDoesntTalk 6y ago
  I wouldn’t work on such software.
  - dimitrios1 6y ago
    You would for 10x the salary.
JustSomeNobody 6y ago
> Our cowardly politicians and toothless regulatory agencies are not up for the challenge. Because their constituents want people to be punished and if the inmates have to suffer a little extra so be it, "they shouldn't have committed a crime." Our society is severely lacking in empathy.
P_I_Staker 6y ago
Or things don't get much better, and management finds a way to make it someone else's problem. Do you think it's the CEO or the people making these decisions that will be locked up? We'll just be arresting whatever yes men show up to be the pawn in the stupid game someone else architected. More people working with a gun to their head. I'd rather the gun be pointed at the person who already has a gun pointed at me, instead of both barrels facing in my direction.
rjurney 6y ago
There is a chain. There is legal recourse. And there are considerations in government IT that you would not believe and they are incredibly difficult to deal with on minimal resources. It has to be harder than the private sector and this application isn't any different than buggy mainframe software run by major banks. It sits and gets crufty.
- foerbert 6y ago
  This is a new system that replaced a previous one not that long ago. This isn't some crazy old thing running COBAL on a VAX somewhere that nobody understands anymore.
  - Spooky23 6y ago
    The old COBOL crap is more likely to have been implemented by someone with a clue. The “new” systems are usually aping the old system behavior. In one case, I ran into a system where some company converted COBOL transactions into Java with some sort of automated tool to put the legacy system “on the internet”.
  - aksss 6y ago
    I have no idea what that means, for newer is not necessarily more supportable. Who knows what the system is - maybe they had a multi-million dollar SAP implementation to manage prisons, and now you’re looking for functional support of that platform after it was customized all to hell, you need a-team SAP resources, not the new kid at Wipro... I can only imagine what’s behind that curtain. It’s the government so I imagine the worst.
RHSeeger 6y ago
It will never be the case that software will be perfect. We can get closer and closer, but the closer we are the more expensive the next step in closing the gap is. While I do agree that making software better/more reliable is a good goal, I believe we would be better off making the system as a whole more robust; the system that includes humans. For every situation where a piece of software has control of something that effects society (individual, group, etc), there should always be a clear and direct means of appealing / pushing back on the decision that was made. Those means should involve a human reviewing the information and making a decision based on that information, not on what the computer said. There's thread after thread of us saying the exact same thing about companies like Google and Facebook; it should apply as a general rule.
- rowanG077 6y ago
  No one is arguing that software must be perfect. But we aren't really even trying. Most software is written in extremely error prone languages without adequate testing. You don't hear anyone saying we should throw out finite-element analyses and other computational verification methods when designing bridges because bridges can never be perfectly secure. Yet that is exactly the sentiment I often hear on software.
  - Aerroon 6y ago
    Are bridges and software comparable in complexity though? How many engineers work on designing a bridge compared to software developers that are writing a program? There are more than an order of magnitude more software developers in the US than civil engineers. Now think about the state of US infrastructure. Does it inspire confidence for the future?
    - rowanG077 6y ago
      I'd say a lot of software is comparable in complexity to the large bridges that exist. Of course there are massive software projects that dwarf any bridge ever build. But a lot of software is only moderately complex. I don't know about infrastructure in the US. I don't live there. I'm happy with the infrastructure in west Europe though. I wish that much care was put in to the software I use every day.
x86_64Ubuntu 6y ago
The politicians and toothlessness of the regulatory agencies is a direct result of the electorate. The electorate likes these sorts of outcomes, and any politician that goes against them will find themselves either primaried or drummed out of office.
daanlo 6y ago
Imho that wouldn‘t cause quality to go up. It would just make it more expensive to develop and to fix bugs. Even more cover your ass would go on. Or at least a huge share of that burden needs to be on the client so that they define and then test and control the SW they receive properly. The problems with the software sound like typical big software project problems. Trying to cover a huge breadth of use cases with lots of very important tiny details and released in a big bang (one migration). It sounds like more of a project mgmt problem than a software problem to me. But maybe I am just a hammer and see nails everywhere.
throwaway8581 6y ago
It’s not the software makers who are committing the crimes. It’s the people abdicating responsibility to software. You can’t wipe your hands of releasing a prisoner on schedule by delegation that to software. The software can help you with your task, but if it’s brought to your attention that there’s a mistake, your failure to promptly fix it is on you.
- Tepix 6y ago
  It's both. Look at the amount of unnecessary waste created by the abysmal Android update policy.
  - throwaway8581 6y ago
    Most of the time, liability for defective software should be civil liability in tort or contract. In most cases where something bad has occurred involving software, it’s going to be hard for the developers of the software to have the sort of mental state and unattenuated causal connection with the occurred harm that we typically require for criminal liability for reckless conduct. I’m not saying it can’t happen, but it would be very unusual circumstances, especially since there’s usually an operator of the software sitting between the developer and the person harmed by the software.
  - EE84M3i 6y ago
    Which abysmal android update policy? What waste?
pc86 6y ago
At what point do you feel the developers - the ones who actually wrote the code - should be held legally responsible for that code's execution?
- deleted 6y ago
  [deleted]
- platinumrad 6y ago
  This headlining issue is a specification change and it is an administrative rather than an engineering failure to knowingly rely on outdated software. The article also refers to other software problems but in scenarios like this the people who write the code (as well as the people who operate it, such as corrections department IT staff) tend to have no decision making power whatsoever: >“It was Thanksgiving weekend,” one source recalled. “We were killing ourselves working on it, but every person associated with the software rollout begged (Deputy Director) Profiri not to go live.”
raymondh 6y ago
> And within that chain, there should be legal > recourse and, in most cases, penal consequences, Wikipedia says, "Under common law, false imprisonment is both a crime and a tort".
rectang 6y ago
> cowardly politicians They aren't cowardly; they are responding rationally to a constituency that hates "criminals". Prioritizing fixing discriminatory systems (such as this software, or "stop and frisk", or the death penalty) is bad electoral politics for "tough on crime" politicians.
rsj_hn 6y ago
> We know exactly how to fix it. Our cowardly politicians and toothless regulatory agencies are not up for the challenge. For every piece of software that can directly and materially harm someone's life like this, there should be a chain of responsibility. No, you know how to blame people and punish people, but that doesn't mean you know how to deliver custom bespoke software for a price that the various government agencies can afford which doesn't have bugs that severely hurt peoples lives. In fact, punishing people is not going to accomplish that. That's the problem with a legislature that thinks it can pass any law it wants - let's take into account this new variable X that our software has no way of collecting or measuring - without looking at the feasibility of actually implementing the law given the infrastructure available, and without approving a corresponding budget for software upgrades to actually enact the law, and taking into account how much time it would take to write, test, deploy, and then train people to use the new software instead of just issuing streams of mandates like Emperor Norton and expecting the mandates to materialize into existence like the morning dew. And if said morning dew does not appear, then we can punish and sue the people in charge when they tell us there is no way they can do what we are asking them. Of course there is blame on the prison leadership for covering things up and that leadership should be fired, but you can punish and sue people all day long and it's not going to result in any good code being written. Punish enough people, and it will just result in the Law being repealed. The problem with this type of bespoke code is that it has exactly 1 customer, so it's going to be horrendously expensive while also being buggy and quickly thrown together compared to software whose development costs are leveraged over millions of customers. And then what happens next year when some crusader decides that they need to take some other new variable into account? Constantly changing requirements, underspecified projects, one-off projects whose schedules are impossible to estimate, and cash strapped local governments. Yeah, that's a recipe for success. This is why everyone hates enterprise software, but even enterprise software has tens of thousands of customers. Bespoke software for the Arizona prison system -- forget it.
carlmr 6y ago
Maybe every contract like this should be programmed to the same API twice. Then you could at least compare it the two pieces of software agree. You check the disagrees and get the companies to fix them (should be part of the contract). And don't tell me you can't buy two CRUD applications for 24 million dollars. It's a silly amount of money for such a buggy application.

bendbro 6y ago

Nobody will take that contract. Any mistake in the government specifying what they want you to build can be politically manipulated into a legal matter with some probability you'll end up in prison.

AdrianB1 6y ago
Not at all: any mistake in the government specifications is the government's problem. If you have requirements and a requirement tracing matrix to deliverables and you deliver what is required, you are good. But usually in most parts of the world these government contracts are intentionally made ambiguous for so many reasons including corruption and incompetence of the people writing requirements.

frompdx 6y ago

My cynical take is that the lack of accountability is exactly what makes software enabled bureaucracy so appealing. If this true, there is no incentive to change.

rbanffy 6y ago
It's a cost/benefit analysis. They'll be taken to court by a few inmates, have to pay them or settle out of court and bill the government for their stays. Since everything is a matter of money, what the company will do is what brings them the most profit after they paid their legal bills. It seems keeping inmates longer pays better than releasing them on time.
spaetzleesser 6y ago
This is part of it. Otherwise it’s just way cheaper to run a faceless bureaucracy that sometimes throws people under the bus when a mistake has been made.

kingaillas 6y ago

Yeah but the cost of that chain will also rise. If I'm (or my company is) personally on the hook for bugs, then I'm going to adopt a NASA-like software quality regimen, pushing up the cost of the product. Every single part of the software stack below me, from hardware, OS, compiler toolchain, disavows responsibility so if I have to absorb all the risk, the product is going to be mind bogglingly expensive.

ryanong 6y ago
that doesn't sound bad to me if we are dealing with people's life or people's freedoms
Draiken 6y ago
I have to say that sounds exactly how this kind of software should be built. We're not talking about the newest social media hype. This software actually matters. Specially since today most of these bureaucratic processes can't be done without these softwares.
- throwaway0a5e 6y ago
  >I have to say that sounds exactly how this kind of software should be built. You see this in every topic. Every "muh pride in muh trade" person says something like this about the relevant trade but the fact of the matter is that the world runs on off-brand duct tape, harbor freight tools, walmart jeans, economy tires, and all sorts of other "value" solutions and the race to the bottom is what has given us much of the modern world that we take for granted. A balance needs to be struck. And it generally needs to be struck further toward the "quickly and cheaply build it like crap but make it easy to override or reset" portion of the available solution space than anyone pontificating about quality on the internet will readily admit.
  - handoflixue 6y ago
    We don't say "perfection is impossible" when it comes to bridges collapsing. We understand that yes, on rare occasion a bridge WILL collapse, but we go and find the people responsible, and we still hold them accountable. This is a level of accountability that basically every other field of engineering is held to, and they've all risen to the challenge and left the "off-brand duct tape" behind. Even within programming, planes don't fall out of the sky daily, so I feel safe assume the aerospace programmers are comfortable working with a high degree of responsibility. High speed traders are dealing with million-dollar stakes and a single mistake can make the news. I'd expect they've got a very accountable culture where people get fired when that happens. There are costs, yes, but there's also costs to keep 733 people illegally imprisoned - we're talking two man-years of peoples lives lost every DAY this goes on.

varenc 6y ago

If there’s penal consequences for bad software, you can bet that the development cost will easily 10x overnight.

deleted 6y ago
[deleted]
oftenwrong 6y ago
Is there a better option? This is software that has to 100% work in order to be trusted. If the software cannot be trusted, additional process has to fill in to verify the decisions the software is making. There's no way to deliver a good solution cheaply.
- TedDoesntTalk 6y ago
  All software has bugs. This is the mantra we are taught and for a good reason. The answer is human oversight.
TedDoesntTalk 6y ago
I wouldn’t work on such software.
- dimitrios1 6y ago
  You would for 10x the salary.

JustSomeNobody 6y ago

> Our cowardly politicians and toothless regulatory agencies are not up for the challenge. Because their constituents want people to be punished and if the inmates have to suffer a little extra so be it, "they shouldn't have committed a crime." Our society is severely lacking in empathy.

P_I_Staker 6y ago

Or things don't get much better, and management finds a way to make it someone else's problem. Do you think it's the CEO or the people making these decisions that will be locked up? We'll just be arresting whatever yes men show up to be the pawn in the stupid game someone else architected. More people working with a gun to their head. I'd rather the gun be pointed at the person who already has a gun pointed at me, instead of both barrels facing in my direction.

rjurney 6y ago

There is a chain. There is legal recourse. And there are considerations in government IT that you would not believe and they are incredibly difficult to deal with on minimal resources. It has to be harder than the private sector and this application isn't any different than buggy mainframe software run by major banks. It sits and gets crufty.

foerbert 6y ago
This is a new system that replaced a previous one not that long ago. This isn't some crazy old thing running COBAL on a VAX somewhere that nobody understands anymore.
- Spooky23 6y ago
  The old COBOL crap is more likely to have been implemented by someone with a clue. The “new” systems are usually aping the old system behavior. In one case, I ran into a system where some company converted COBOL transactions into Java with some sort of automated tool to put the legacy system “on the internet”.
- aksss 6y ago
  I have no idea what that means, for newer is not necessarily more supportable. Who knows what the system is - maybe they had a multi-million dollar SAP implementation to manage prisons, and now you’re looking for functional support of that platform after it was customized all to hell, you need a-team SAP resources, not the new kid at Wipro... I can only imagine what’s behind that curtain. It’s the government so I imagine the worst.

RHSeeger 6y ago

It will never be the case that software will be perfect. We can get closer and closer, but the closer we are the more expensive the next step in closing the gap is. While I do agree that making software better/more reliable is a good goal, I believe we would be better off making the system as a whole more robust; the system that includes humans. For every situation where a piece of software has control of something that effects society (individual, group, etc), there should always be a clear and direct means of appealing / pushing back on the decision that was made. Those means should involve a human reviewing the information and making a decision based on that information, not on what the computer said. There's thread after thread of us saying the exact same thing about companies like Google and Facebook; it should apply as a general rule.

rowanG077 6y ago
No one is arguing that software must be perfect. But we aren't really even trying. Most software is written in extremely error prone languages without adequate testing. You don't hear anyone saying we should throw out finite-element analyses and other computational verification methods when designing bridges because bridges can never be perfectly secure. Yet that is exactly the sentiment I often hear on software.
- Aerroon 6y ago
  Are bridges and software comparable in complexity though? How many engineers work on designing a bridge compared to software developers that are writing a program? There are more than an order of magnitude more software developers in the US than civil engineers. Now think about the state of US infrastructure. Does it inspire confidence for the future?
  - rowanG077 6y ago
    I'd say a lot of software is comparable in complexity to the large bridges that exist. Of course there are massive software projects that dwarf any bridge ever build. But a lot of software is only moderately complex. I don't know about infrastructure in the US. I don't live there. I'm happy with the infrastructure in west Europe though. I wish that much care was put in to the software I use every day.

x86_64Ubuntu 6y ago

The politicians and toothlessness of the regulatory agencies is a direct result of the electorate. The electorate likes these sorts of outcomes, and any politician that goes against them will find themselves either primaried or drummed out of office.

daanlo 6y ago

Imho that wouldn‘t cause quality to go up. It would just make it more expensive to develop and to fix bugs. Even more cover your ass would go on. Or at least a huge share of that burden needs to be on the client so that they define and then test and control the SW they receive properly. The problems with the software sound like typical big software project problems. Trying to cover a huge breadth of use cases with lots of very important tiny details and released in a big bang (one migration). It sounds like more of a project mgmt problem than a software problem to me. But maybe I am just a hammer and see nails everywhere.

throwaway8581 6y ago

It’s not the software makers who are committing the crimes. It’s the people abdicating responsibility to software. You can’t wipe your hands of releasing a prisoner on schedule by delegation that to software. The software can help you with your task, but if it’s brought to your attention that there’s a mistake, your failure to promptly fix it is on you.

Tepix 6y ago
It's both. Look at the amount of unnecessary waste created by the abysmal Android update policy.
- throwaway8581 6y ago
  Most of the time, liability for defective software should be civil liability in tort or contract. In most cases where something bad has occurred involving software, it’s going to be hard for the developers of the software to have the sort of mental state and unattenuated causal connection with the occurred harm that we typically require for criminal liability for reckless conduct. I’m not saying it can’t happen, but it would be very unusual circumstances, especially since there’s usually an operator of the software sitting between the developer and the person harmed by the software.
- EE84M3i 6y ago
  Which abysmal android update policy? What waste?

pc86 6y ago

At what point do you feel the developers - the ones who actually wrote the code - should be held legally responsible for that code's execution?

deleted 6y ago
[deleted]
platinumrad 6y ago
This headlining issue is a specification change and it is an administrative rather than an engineering failure to knowingly rely on outdated software. The article also refers to other software problems but in scenarios like this the people who write the code (as well as the people who operate it, such as corrections department IT staff) tend to have no decision making power whatsoever: >“It was Thanksgiving weekend,” one source recalled. “We were killing ourselves working on it, but every person associated with the software rollout begged (Deputy Director) Profiri not to go live.”

raymondh 6y ago

> And within that chain, there should be legal > recourse and, in most cases, penal consequences, Wikipedia says, "Under common law, false imprisonment is both a crime and a tort".

rectang 6y ago

> cowardly politicians They aren't cowardly; they are responding rationally to a constituency that hates "criminals". Prioritizing fixing discriminatory systems (such as this software, or "stop and frisk", or the death penalty) is bad electoral politics for "tough on crime" politicians.

rsj_hn 6y ago

> We know exactly how to fix it. Our cowardly politicians and toothless regulatory agencies are not up for the challenge. For every piece of software that can directly and materially harm someone's life like this, there should be a chain of responsibility. No, you know how to blame people and punish people, but that doesn't mean you know how to deliver custom bespoke software for a price that the various government agencies can afford which doesn't have bugs that severely hurt peoples lives. In fact, punishing people is not going to accomplish that. That's the problem with a legislature that thinks it can pass any law it wants - let's take into account this new variable X that our software has no way of collecting or measuring - without looking at the feasibility of actually implementing the law given the infrastructure available, and without approving a corresponding budget for software upgrades to actually enact the law, and taking into account how much time it would take to write, test, deploy, and then train people to use the new software instead of just issuing streams of mandates like Emperor Norton and expecting the mandates to materialize into existence like the morning dew. And if said morning dew does not appear, then we can punish and sue the people in charge when they tell us there is no way they can do what we are asking them. Of course there is blame on the prison leadership for covering things up and that leadership should be fired, but you can punish and sue people all day long and it's not going to result in any good code being written. Punish enough people, and it will just result in the Law being repealed. The problem with this type of bespoke code is that it has exactly 1 customer, so it's going to be horrendously expensive while also being buggy and quickly thrown together compared to software whose development costs are leveraged over millions of customers. And then what happens next year when some crusader decides that they need to take some other new variable into account? Constantly changing requirements, underspecified projects, one-off projects whose schedules are impossible to estimate, and cash strapped local governments. Yeah, that's a recipe for success. This is why everyone hates enterprise software, but even enterprise software has tens of thousands of customers. Bespoke software for the Arizona prison system -- forget it.

carlmr 6y ago

Maybe every contract like this should be programmed to the same API twice. Then you could at least compare it the two pieces of software agree. You check the disagrees and get the companies to fix them (should be part of the contract). And don't tell me you can't buy two CRUD applications for 24 million dollars. It's a silly amount of money for such a buggy application.