5 ms·

Worse - you may not know the software has this power over you. I remember a specially egregious case that one of the rental companies was guilty of: automated

Worse - you may not know the software has this power over you. I remember a specially egregious case that one of the rental companies was guilty of: automated criminality The software in question automated the reporting of grand theft auto to authorities. The rental company used it to streamline reporting of rentals to police. The problem: false positives. The software would report drivers as criminals, when in fact cars had been exchanged midrental due to car failures etc, and the original car hadnt been returned under the normal workflow, which the software failed to account for. Then that police report would chase innocent people for literal years, coming up to unsuspecting customers during routine speeding stops. I always wonder whether contributor names to closed sourced software should always be transparent. Excellent engineers should be able to take credit on their work and there would be more skin in the game. Particularly for employers that get suspect resources from to solve for budget constrains artificially imposed by business managers with no stake in the product functionality, to critical software processes like SREs etc

novok 6y ago
In that case although, you don't know which system or team is responsible for the 'cop reporting' part. I could be making a very innocent rental state tracking system, and then someone else just accesses the same table without my knowledge and makes a criminal reporting system after the fact. After a certain point, we recognize as a society that just because you make chairs, your not at fault if someone then takes a chair you made and then bashes someone on the head with a chair. It's the basher's fault, not yours.
- bryanrasmussen 6y ago
  >you don't know which system or team is responsible for the 'cop reporting' part. you also don't know if they have any information given regarding possible workflow deviations, or if it was made when they had one workflow and workflow was changed without paying for changing the system. If that were the case it would be an example of shitty management gets people harassed by the police and potentially killed because getting harassed by the police at the wrong moment, or looking like the wrong person, in the wrong country can get you killed.
- deleted 6y ago
  [deleted]
skybrian 6y ago
Open source projects disclaim all warranties because the authors have no control over how the software might be used. It seems like there are similar issues for code written for a business? It would be foolish to take liability when you can’t stop the business from continuing to use the software and causing more damage.
hedora 6y ago
The false positive in this case is filing a false police report. That’s illegal in the US, and probably exposes the rental car company to civil liabilities (generally, contracts that allow one party to commit crimes are unenforceable, so I’d hope that extends to eliminating the binding arbitration clause, though I am not a lawyer). The police should pursue it cases like this, and arrest the decision maker in the organization that caused this to happen. If it’s been happening repeatedly, the person’s sentence should be increased accordingly.
- spockz 6y ago
  Be that as it may, it will do you very little good when you get pulled over for something routine and then you get arrested or worse for something you don’t even have the faintest idea about. Which is why bad software can hurt people a lot.
  - routerl 6y ago
    I agree completely, and have never understood people who seem to think this would never happen to them. The OP's point is easily generalizable to so many domains; anyone who has written enough commercial software has seen bugs like these, though perhaps not with as severe legal consequences.
- m463 6y ago
  I would imagine that normal people don't have as much legal experience or representation as a large corporation (that has experience in exactly this). I had a friend - who happened to be pretty well-versed AND interested in legal matters - who was sued by someone for some car accident. The thing was, the identification came from a license plate, and he lived hundreds of miles away and drove a completely different car. He finally got in touch with the attorney - who didn't care - and made him care. Took time and effort. Would have cost a normal person a pretty penny.
- gameswithgo 6y ago
  haha, police pursuing white collar crime! too busy choking unlicensed street vendors to death
mmglr 6y ago
> I always wonder whether contributor names to closed sourced software should always be transparent. Are you implying that the developers that created this back office tool should be liable for the false police reports? If so then then counterpoint based on having made back office apps: the individual contributor (i.e. developer) may either (1) have little to no say in what the app is doing as they are simply implementing the requirements as provided and/or (2) have no knowledge of how it might fail (e.g. when the app is integrated as part of larger business processes)
- HenryBemis 6y ago
  The developer's actions are 'best effort'. Didn't the company run extensive tests to cover all possible scenarios? If you build a system that orders coffee capsules, you want to make sure you don't order x100. But when you report 'thieves' to the authorities you open yourself up for plenty of lawsuits.. imagine the poor people being harassed by the authorities for some crime they didn't do, and will take plenty of time/money/effort to clean-up, all because company XYZ does not have a thorough UAT..
  - IG_Semmelweiss 6y ago
    "imagine the poor people being harassed by the authorities" Unfortunately we don't have to imagine this. This is exactly what happened (Hertz was the culprit, see link downtread) You may not even need that word "poor" inserted in there in fact, the average person in middle america may not have the resources to to avoid a night in county jail, due to a wrongful imprisonment / accusation.
    - HenryBemis 6y ago
      Apologies, the word "poor" didn't have to do with fanancial status. We use the word "poor" also metaphorically. Someone breaks their leg, we say "poor guy".. In also know that in Italy someone may say "povre" in the meaning pity, not poverty.
- 908B64B197 6y ago
bengale 6y ago
Makes me think of this scandal in the UK: https://en.m.wikipedia.org/wiki/Horizon_(IT_system) https://en.m.wikipedia.org/wiki/Horizon_(IT_system)
- NikolaNovak 6y ago
  Geebus. That's a terrifying read and a 15 year nightmare! I take it post office is a private entity in UK? The last paragraph where the institution disband the oversight and destroys results day before report is due is infuriating :O
spaced-out 6y ago
>I always wonder whether contributor names to closed sourced software should always be transparent. At the very least, for this to be fair, that list should include management, up to the CEO.
tedunangst 6y ago
Related: https://www.inquirer.com/business/retail/hertz-stolen-car-grand-theft-auto-malofiy-bankruptcy-lawsuit-20200803.html https://www.inquirer.com/business/retail/hertz-stolen-car-gr...
- IG_Semmelweiss 6y ago
  ^ this is it.
cmiles74 6y ago
> I always wonder whether contributor names to closed sourced software should always be transparent. In my opinion, absolutely not! The company or corporation holds all the cards: they manage the budget, the schedule, the hiring firing and promoting. The company is responsible to marketing and what sales people are telling customers. In most cases every bit of code the developers produce is wholly and entirely the property of the company. Blaming the developer, to my mind, only empowers companies making software like this to make more of it and to sell it more widely. I would argue that we should see harsher penalties for organizations that produce and market (or mis-market) software that is causing harm.

novok 6y ago

In that case although, you don't know which system or team is responsible for the 'cop reporting' part. I could be making a very innocent rental state tracking system, and then someone else just accesses the same table without my knowledge and makes a criminal reporting system after the fact. After a certain point, we recognize as a society that just because you make chairs, your not at fault if someone then takes a chair you made and then bashes someone on the head with a chair. It's the basher's fault, not yours.

bryanrasmussen 6y ago
>you don't know which system or team is responsible for the 'cop reporting' part. you also don't know if they have any information given regarding possible workflow deviations, or if it was made when they had one workflow and workflow was changed without paying for changing the system. If that were the case it would be an example of shitty management gets people harassed by the police and potentially killed because getting harassed by the police at the wrong moment, or looking like the wrong person, in the wrong country can get you killed.
deleted 6y ago
[deleted]

skybrian 6y ago

Open source projects disclaim all warranties because the authors have no control over how the software might be used. It seems like there are similar issues for code written for a business? It would be foolish to take liability when you can’t stop the business from continuing to use the software and causing more damage.

hedora 6y ago

The false positive in this case is filing a false police report. That’s illegal in the US, and probably exposes the rental car company to civil liabilities (generally, contracts that allow one party to commit crimes are unenforceable, so I’d hope that extends to eliminating the binding arbitration clause, though I am not a lawyer). The police should pursue it cases like this, and arrest the decision maker in the organization that caused this to happen. If it’s been happening repeatedly, the person’s sentence should be increased accordingly.

spockz 6y ago
Be that as it may, it will do you very little good when you get pulled over for something routine and then you get arrested or worse for something you don’t even have the faintest idea about. Which is why bad software can hurt people a lot.
- routerl 6y ago
  I agree completely, and have never understood people who seem to think this would never happen to them. The OP's point is easily generalizable to so many domains; anyone who has written enough commercial software has seen bugs like these, though perhaps not with as severe legal consequences.
m463 6y ago
I would imagine that normal people don't have as much legal experience or representation as a large corporation (that has experience in exactly this). I had a friend - who happened to be pretty well-versed AND interested in legal matters - who was sued by someone for some car accident. The thing was, the identification came from a license plate, and he lived hundreds of miles away and drove a completely different car. He finally got in touch with the attorney - who didn't care - and made him care. Took time and effort. Would have cost a normal person a pretty penny.
gameswithgo 6y ago
haha, police pursuing white collar crime! too busy choking unlicensed street vendors to death

mmglr 6y ago

> I always wonder whether contributor names to closed sourced software should always be transparent. Are you implying that the developers that created this back office tool should be liable for the false police reports? If so then then counterpoint based on having made back office apps: the individual contributor (i.e. developer) may either (1) have little to no say in what the app is doing as they are simply implementing the requirements as provided and/or (2) have no knowledge of how it might fail (e.g. when the app is integrated as part of larger business processes)

HenryBemis 6y ago
The developer's actions are 'best effort'. Didn't the company run extensive tests to cover all possible scenarios? If you build a system that orders coffee capsules, you want to make sure you don't order x100. But when you report 'thieves' to the authorities you open yourself up for plenty of lawsuits.. imagine the poor people being harassed by the authorities for some crime they didn't do, and will take plenty of time/money/effort to clean-up, all because company XYZ does not have a thorough UAT..
- IG_Semmelweiss 6y ago
  "imagine the poor people being harassed by the authorities" Unfortunately we don't have to imagine this. This is exactly what happened (Hertz was the culprit, see link downtread) You may not even need that word "poor" inserted in there in fact, the average person in middle america may not have the resources to to avoid a night in county jail, due to a wrongful imprisonment / accusation.
  - HenryBemis 6y ago
    Apologies, the word "poor" didn't have to do with fanancial status. We use the word "poor" also metaphorically. Someone breaks their leg, we say "poor guy".. In also know that in Italy someone may say "povre" in the meaning pity, not poverty.
908B64B197 6y ago

bengale 6y ago

Makes me think of this scandal in the UK: https://en.m.wikipedia.org/wiki/Horizon_(IT_system) https://en.m.wikipedia.org/wiki/Horizon_(IT_system)

NikolaNovak 6y ago
Geebus. That's a terrifying read and a 15 year nightmare! I take it post office is a private entity in UK? The last paragraph where the institution disband the oversight and destroys results day before report is due is infuriating :O

spaced-out 6y ago

>I always wonder whether contributor names to closed sourced software should always be transparent. At the very least, for this to be fair, that list should include management, up to the CEO.

tedunangst 6y ago

Related: https://www.inquirer.com/business/retail/hertz-stolen-car-grand-theft-auto-malofiy-bankruptcy-lawsuit-20200803.html https://www.inquirer.com/business/retail/hertz-stolen-car-gr...

IG_Semmelweiss 6y ago
^ this is it.

cmiles74 6y ago

> I always wonder whether contributor names to closed sourced software should always be transparent. In my opinion, absolutely not! The company or corporation holds all the cards: they manage the budget, the schedule, the hiring firing and promoting. The company is responsible to marketing and what sales people are telling customers. In most cases every bit of code the developers produce is wholly and entirely the property of the company. Blaming the developer, to my mind, only empowers companies making software like this to make more of it and to sell it more widely. I would argue that we should see harsher penalties for organizations that produce and market (or mis-market) software that is causing harm.