7 ms·

I think the imposition of this burden on engineers is the result of our belief that our legislators are too incompetent or unwilling to act in the fact of clear

by ABCLAW 7y ago

I think the imposition of this burden on engineers is the result of our belief that our legislators are too incompetent or unwilling to act in the fact of clear moral hazards. Tech has a very strong history of conscientious objectors. They're not the problem. Other institutions need to do their part.

AstralStorm 7y ago
Guess who pays the most lobbyists. That on the point of moral hazards.
- Retric 7y ago
  That’s less sinister than you might think. New legislation is more common for new industries. Consider the internet began as the Arpanet is just 50 years ago. For much of that time the idea that the average person would be online from their own home would have seemed like crazy science fiction.
  - AstralStorm 7y ago
    The thing is, the legislation is tweaked to the benefit of the giants, not against them. At least in general. Not even federal government with their delusions of absolute control can do anything. (Say, FISA insanity.) Net neutrality was a contest between two big corps. Patent battles ruled to support American business despite clear violations? (Qualcomm et al.) The used to be laws considering encryption to be munitions. Silly attempt to sabotage it AKA Clipper. Yet somehow AI systems to discriminate explicitly sold to opposing regimes are not despite being much more applicable. On the other hand, there is Comcast and Time Warner.
    - Retric 7y ago
      I think the public benifits from that to some degree. A search engine index is inherently a derivative work and without updated legislation could easily qualify as copyright infringement. I am not saying is fair, but other industries which get away with actually killing people (ex: fine particle pollution from coal power plants). So, bias is somewhat expected from how our system works, it’s just not all bad.
rexpop 7y ago
It makes sense when you consider that labor is the root of political power, even that wielded by legislators. See "Forces of Labor". https://sites.google.com/view/tech-workers-coalition/topics/forces-of-labor?authuser=0 https://sites.google.com/view/tech-workers-coalition/topics/...
mspecter 7y ago
A moral hazard happens when an entity is somehow insured against something (e.g. health insurance), so is rationally more likely to behave in some "bad" way (e.g. driving recklessly). I'm not seeing how this a moral hazard, do you just mean "immoral"?
- matt4077 7y ago
  It’s a somewhat stupid example, because nobody endangers their own life just because they have health insurance. Smoking instead of reckless driving may make it (slightly) more understandable.
  - mspecter 7y ago
    Sure, though, to be pedantic, a common example of moral hazard is the increased likelihood of driving recklessly in the presence of mandatory seat belts. See https://web.stanford.edu/~leinav/pubs/RESTAT2003.pdf https://web.stanford.edu/~leinav/pubs/RESTAT2003.pdf
    - matt4077 7y ago
      That’s not a moral hazard. The word doesn’t even appear in the paper. It’s just an example of somewhat efficiently choosing a different point on the risk/reward continuum when the payoffs change. A moral hazard is choosing a selfish course of action with negative external effects.
- zwkrt 7y ago
  "Insured" can be interpreted very broadly; moral hazard is whenever the negative outcomes of a risky decision are directed away from oneself. A CEO considering the option of a layoff is a moral hazard as she will make her board happy at the expense of her employees. Either way she has nothing to lose.
  - mspecter 7y ago
    >"Insured" can be interpreted very broadly; moral hazard is whenever the negative outcomes of a risky decision are directed away from oneself. Nope! That's called an externality. A moral hazard is a type of externality, but is very focused on a particular set of instances in the definition I provided.
robertAngst 7y ago
What is the problem again? You used google while signed in and now the world knows you like barbies? Serious question, because I do not feel violated.
- ohyes 7y ago
  Assume that a hypothetical future government that you find yourself living under is not good. Assume furthermore that a future hypothetical company is equally not good and the two collude to do not good things together. Now assume you’re motivated to do something about it. Sucks for you, they have already predicted that and cleansed you because they know everything you’ve ever read and it was a high probability that you’d be against them. Consider another situation where hypothetical evil overlords know exactly what buttons press in order to influence you to act a certain way. They press them regularly. You dance as expected. Both circumstances undermine your personal autonomy and right to self determination. Furthermore they undermine the humanity that we share where our society is no longer controlled by the needs of humans, but instead arbitrary algorithms coded for whatever mundanely evil purpose. And that’s the point (that you’ve more or less hit on), this is evil, but also so boring that it is actually a threat to society because no one will care enough to do anything.
  - twunde 7y ago
    For those that want a more practical example, Russia has just passed a law requiring that all internet traffic be sent to their equivalent of the NSA and FBI, with all traffic sent in the clear [1]. If you're an adtech company in Russia, with historical data you are probably being subpoenaed for that historical information. Think about Turkey, which has had mass purges since 2016. Academics, judges, ordinary citizens have been jailed or disappeared for having gone to the wrong school, having attended the wrong house of worship, for being friends with the wrong people. The government isn't tracking these people down via paper records, they're tracking them down using ordinary technology. It's not about step 1, which is to sell you more barbies. It's about step 4, step 5 where suddenly a bad actor has access to all this data and can see that people who make this search have a 10% chance of opposing the current leader/president/ruling party etc. [1] https://www.themoscowtimes.com/2019/04/21/russias-sovereign-internet-law-will-destroy-innovation-a65317 https://www.themoscowtimes.com/2019/04/21/russias-sovereign-...
whalabi 7y ago
I imagine that even if tech has a strong history of conscientious objectors, you could argue the industry needs more because: * Tech is incredibly, incredibly important and powerful, in its implications and effects on politics, society, geopolitics, everything * It seems plausible there's far more going on than anyone knows about
- lone_haxx0r 7y ago
  I agree with your main thesis, but also: It's time that we start teaching kids at school about technology. And I mean actual technology, not how to make a pretty slides presentation on Powerpoint/libreOffice. I mean the importance of your data, how companies exploit it, what is a CPU, what is RAM, what is an operating system, what is security, cryptography, domain names, how ISPs work, etc.
  - whalabi 7y ago
    Agree, kinda. E.g. rather than how domain names work, I'd rather people learn about privacy in the internet era. Rather than say, RAM, I'd rather people learn about how social media is addictive and why, etc. All of the above would be good too :)
zimpenfish 7y ago
> the result of our belief that our legislators are too incompetent or unwilling to act in the fact of clear moral hazards Such a belief is justified because they have a strong history of being both.
killjoywashere 7y ago
I wrote a response to an email today, someone had sent me two ethics position papers. I pigeon-holed my own response after I wrote "it's time policy makers start assuming responsibility for understanding more logic than just policy logic. Math must be part of their domain expertise". Mainly I shelved my response because I doubt the policy people on the thread would have taken that very well. But the truth is looming large these days.
- swiley 7y ago
  Isn't that what the legally so called "experts" are for? Having technology that's too complex for most people to handle hasn't been a new thing for at least a couple centuries.
  - ABCLAW 7y ago
    Experts are a nice resource to have, but they're not a cure-all. Not all of our legislature's representatives have systemically good incentives. Regardless of their ability to understand complexity, complexity offers them the ability to obfuscate. Ignoring any bad faith or corruption, though, our legislatures are also a limited compute resource. If the complexity of modern society rises by orders of magnitude, the tools legislatures previously used to tackle complexity may not scale to the new challenges, or may simply be too slow to address the issues while they are relevant. One of my first areas of research was mapping legislature throughput to variations in rates of technological change. It appears most western governments addressed social issues arising out of the industrial revolution, for instance, exceptionally slowly. To summarize a few years of reading in the library stacks: the relationship between social complexity and quality of governance is not well understood.
  - lone_haxx0r 7y ago
    You can hire a million experts, but it the one making the decision is a trained monkey, they won't be of any use.
seppin 7y ago
Yes why jeopardize the bags of money everyone makes for democracy, etc.
jondubois 7y ago
The core of the problem is that the masses do not have shared values anymore. There are no moral ideals to aim towards. Without values there is no good or evil, there is only power and those too weak to seek it.
- ssss11 7y ago
  This hits the nail on the head.
- rjf72 7y ago
  But we do all have shared values. In fact, I think we all agree on most of everything. Imagine there was a button you could press where each and every person would suddenly receive at least a nice house, a bit of land to call their own, and a pleasant good paying job. There's not a single person, regardless of ideology, that wouldn't press this button. Another button to get rid of all crime? Yip, gonna be pushed. Another to get rid of all poverty? Everybody wants the exact same things. The only thing we disagree on is the best route to get there. That said, I completely agree with you that there is a perception of the lack of any shared value. So why might this be? The 2016 election was interesting. How many people voted for Hillary thinking 'Yes, this person truly stands for what I believe in and will make a phenomenal president.'? How many voted for her because the alternative was completely unthinkable. And the exact same holds true for those that voted for Trump. Think about what a remarkable bit of social engineering that is. In a democracy, you've managed to get tens of millions of people to vote for people they don't actually like or event want in office. And all you had to do was to make people loathe, and fear, the alternative sufficiently. Division helps entrench establishment forces. You can even see this in the choice of which issues get elevated to the national level. What is the weapon most typically used in mass homicides? What percent of homicides are rifles used in? I think the majority of Americans would get these questions completely wrong. Because the issues that we elevate paint a picture that is not in accordance with statistical reality. That reality being that pistols are the primary weapon of mass homicides, and with rifles being responsible for about 2% of all homicides. [1] Our homicide rate is driven by cheap little pistols. The year to year variance in pistol homicide is frequently larger than the entire sum of all rifle homicides. In other words if we had a magical button to get rid of all rifles, "assault" or not, you wouldn't even notice a drop in the homicide rate. It'd be statistical noise. But the issue is promoted because it's extremely divisive. It makes people fear and hate one another and further drive this perception that we have no shared values. And what that translates to is at the polls you won't vote for who you want, but will instead vote against who you do not want. And that translates to voting for the establishment candidate who, by definition, will have shown themselves to be the most 'electable'. And then we all end up disappointed, and then do this again 4 years later. Only this time the establishment candidate is truly different, honest! Or in the case of an incumbent, they'll actually do what they've been promising now - they just need 4 more years, honest! [1] - https://ucr.fbi.gov/crime-in-the-u.s/2014/crime-in-the-u.s.-2014/tables/expanded-homicide-data/expanded_homicide_data_table_8_murder_victims_by_weapon_2010-2014.xls https://ucr.fbi.gov/crime-in-the-u.s/2014/crime-in-the-u.s.-...
StanislavPetrov 7y ago
This is a burden that lies with us all, individually. We are all individually responsible for our actions. Its absurd to suggest that somehow we should outsource our morality and the responsibility for our actions to politicians or legislators. Whether or not our system (and society) is horribly broken on every level (it is) has no bearing on the personal responsibility we all bear for what we do as individuals.
jlg23 7y ago
In the discourse that followed WW2 we put the same burden on soldiers and we still today prosecute German soldiers who worked in concentration camps (see [1], for a book keeper sentenced to prison time in 2015; not actually involved in killings). Who, if not we, the engineers who really understand the implications of our actions, should stand up? [1] https://www.rt.com/news/421101-auschwitz-accountant-guard-died-prison/ https://www.rt.com/news/421101-auschwitz-accountant-guard-di...
PavlovsCat 7y ago
> Tech has a very strong history of conscientious objectors. Oh? I know of Joseph Weizenbaum, who else?

by ABCLAW 7y ago

AstralStorm 7y ago

Guess who pays the most lobbyists. That on the point of moral hazards.

Retric 7y ago
That’s less sinister than you might think. New legislation is more common for new industries. Consider the internet began as the Arpanet is just 50 years ago. For much of that time the idea that the average person would be online from their own home would have seemed like crazy science fiction.
- AstralStorm 7y ago
  The thing is, the legislation is tweaked to the benefit of the giants, not against them. At least in general. Not even federal government with their delusions of absolute control can do anything. (Say, FISA insanity.) Net neutrality was a contest between two big corps. Patent battles ruled to support American business despite clear violations? (Qualcomm et al.) The used to be laws considering encryption to be munitions. Silly attempt to sabotage it AKA Clipper. Yet somehow AI systems to discriminate explicitly sold to opposing regimes are not despite being much more applicable. On the other hand, there is Comcast and Time Warner.
  - Retric 7y ago
    I think the public benifits from that to some degree. A search engine index is inherently a derivative work and without updated legislation could easily qualify as copyright infringement. I am not saying is fair, but other industries which get away with actually killing people (ex: fine particle pollution from coal power plants). So, bias is somewhat expected from how our system works, it’s just not all bad.

rexpop 7y ago

It makes sense when you consider that labor is the root of political power, even that wielded by legislators. See "Forces of Labor". https://sites.google.com/view/tech-workers-coalition/topics/forces-of-labor?authuser=0 https://sites.google.com/view/tech-workers-coalition/topics/...

mspecter 7y ago

A moral hazard happens when an entity is somehow insured against something (e.g. health insurance), so is rationally more likely to behave in some "bad" way (e.g. driving recklessly). I'm not seeing how this a moral hazard, do you just mean "immoral"?

matt4077 7y ago
It’s a somewhat stupid example, because nobody endangers their own life just because they have health insurance. Smoking instead of reckless driving may make it (slightly) more understandable.
- mspecter 7y ago
  Sure, though, to be pedantic, a common example of moral hazard is the increased likelihood of driving recklessly in the presence of mandatory seat belts. See https://web.stanford.edu/~leinav/pubs/RESTAT2003.pdf https://web.stanford.edu/~leinav/pubs/RESTAT2003.pdf
  - matt4077 7y ago
    That’s not a moral hazard. The word doesn’t even appear in the paper. It’s just an example of somewhat efficiently choosing a different point on the risk/reward continuum when the payoffs change. A moral hazard is choosing a selfish course of action with negative external effects.
zwkrt 7y ago
"Insured" can be interpreted very broadly; moral hazard is whenever the negative outcomes of a risky decision are directed away from oneself. A CEO considering the option of a layoff is a moral hazard as she will make her board happy at the expense of her employees. Either way she has nothing to lose.
- mspecter 7y ago
  >"Insured" can be interpreted very broadly; moral hazard is whenever the negative outcomes of a risky decision are directed away from oneself. Nope! That's called an externality. A moral hazard is a type of externality, but is very focused on a particular set of instances in the definition I provided.

robertAngst 7y ago

What is the problem again? You used google while signed in and now the world knows you like barbies? Serious question, because I do not feel violated.

ohyes 7y ago
Assume that a hypothetical future government that you find yourself living under is not good. Assume furthermore that a future hypothetical company is equally not good and the two collude to do not good things together. Now assume you’re motivated to do something about it. Sucks for you, they have already predicted that and cleansed you because they know everything you’ve ever read and it was a high probability that you’d be against them. Consider another situation where hypothetical evil overlords know exactly what buttons press in order to influence you to act a certain way. They press them regularly. You dance as expected. Both circumstances undermine your personal autonomy and right to self determination. Furthermore they undermine the humanity that we share where our society is no longer controlled by the needs of humans, but instead arbitrary algorithms coded for whatever mundanely evil purpose. And that’s the point (that you’ve more or less hit on), this is evil, but also so boring that it is actually a threat to society because no one will care enough to do anything.
- twunde 7y ago
  For those that want a more practical example, Russia has just passed a law requiring that all internet traffic be sent to their equivalent of the NSA and FBI, with all traffic sent in the clear [1]. If you're an adtech company in Russia, with historical data you are probably being subpoenaed for that historical information. Think about Turkey, which has had mass purges since 2016. Academics, judges, ordinary citizens have been jailed or disappeared for having gone to the wrong school, having attended the wrong house of worship, for being friends with the wrong people. The government isn't tracking these people down via paper records, they're tracking them down using ordinary technology. It's not about step 1, which is to sell you more barbies. It's about step 4, step 5 where suddenly a bad actor has access to all this data and can see that people who make this search have a 10% chance of opposing the current leader/president/ruling party etc. [1] https://www.themoscowtimes.com/2019/04/21/russias-sovereign-internet-law-will-destroy-innovation-a65317 https://www.themoscowtimes.com/2019/04/21/russias-sovereign-...

whalabi 7y ago

I imagine that even if tech has a strong history of conscientious objectors, you could argue the industry needs more because: * Tech is incredibly, incredibly important and powerful, in its implications and effects on politics, society, geopolitics, everything * It seems plausible there's far more going on than anyone knows about

lone_haxx0r 7y ago
I agree with your main thesis, but also: It's time that we start teaching kids at school about technology. And I mean actual technology, not how to make a pretty slides presentation on Powerpoint/libreOffice. I mean the importance of your data, how companies exploit it, what is a CPU, what is RAM, what is an operating system, what is security, cryptography, domain names, how ISPs work, etc.
- whalabi 7y ago
  Agree, kinda. E.g. rather than how domain names work, I'd rather people learn about privacy in the internet era. Rather than say, RAM, I'd rather people learn about how social media is addictive and why, etc. All of the above would be good too :)

zimpenfish 7y ago

> the result of our belief that our legislators are too incompetent or unwilling to act in the fact of clear moral hazards Such a belief is justified because they have a strong history of being both.

killjoywashere 7y ago

I wrote a response to an email today, someone had sent me two ethics position papers. I pigeon-holed my own response after I wrote "it's time policy makers start assuming responsibility for understanding more logic than just policy logic. Math must be part of their domain expertise". Mainly I shelved my response because I doubt the policy people on the thread would have taken that very well. But the truth is looming large these days.

swiley 7y ago
Isn't that what the legally so called "experts" are for? Having technology that's too complex for most people to handle hasn't been a new thing for at least a couple centuries.
- ABCLAW 7y ago
  Experts are a nice resource to have, but they're not a cure-all. Not all of our legislature's representatives have systemically good incentives. Regardless of their ability to understand complexity, complexity offers them the ability to obfuscate. Ignoring any bad faith or corruption, though, our legislatures are also a limited compute resource. If the complexity of modern society rises by orders of magnitude, the tools legislatures previously used to tackle complexity may not scale to the new challenges, or may simply be too slow to address the issues while they are relevant. One of my first areas of research was mapping legislature throughput to variations in rates of technological change. It appears most western governments addressed social issues arising out of the industrial revolution, for instance, exceptionally slowly. To summarize a few years of reading in the library stacks: the relationship between social complexity and quality of governance is not well understood.
- lone_haxx0r 7y ago
  You can hire a million experts, but it the one making the decision is a trained monkey, they won't be of any use.

seppin 7y ago

Yes why jeopardize the bags of money everyone makes for democracy, etc.

jondubois 7y ago

The core of the problem is that the masses do not have shared values anymore. There are no moral ideals to aim towards. Without values there is no good or evil, there is only power and those too weak to seek it.

ssss11 7y ago
This hits the nail on the head.
rjf72 7y ago
But we do all have shared values. In fact, I think we all agree on most of everything. Imagine there was a button you could press where each and every person would suddenly receive at least a nice house, a bit of land to call their own, and a pleasant good paying job. There's not a single person, regardless of ideology, that wouldn't press this button. Another button to get rid of all crime? Yip, gonna be pushed. Another to get rid of all poverty? Everybody wants the exact same things. The only thing we disagree on is the best route to get there. That said, I completely agree with you that there is a perception of the lack of any shared value. So why might this be? The 2016 election was interesting. How many people voted for Hillary thinking 'Yes, this person truly stands for what I believe in and will make a phenomenal president.'? How many voted for her because the alternative was completely unthinkable. And the exact same holds true for those that voted for Trump. Think about what a remarkable bit of social engineering that is. In a democracy, you've managed to get tens of millions of people to vote for people they don't actually like or event want in office. And all you had to do was to make people loathe, and fear, the alternative sufficiently. Division helps entrench establishment forces. You can even see this in the choice of which issues get elevated to the national level. What is the weapon most typically used in mass homicides? What percent of homicides are rifles used in? I think the majority of Americans would get these questions completely wrong. Because the issues that we elevate paint a picture that is not in accordance with statistical reality. That reality being that pistols are the primary weapon of mass homicides, and with rifles being responsible for about 2% of all homicides. [1] Our homicide rate is driven by cheap little pistols. The year to year variance in pistol homicide is frequently larger than the entire sum of all rifle homicides. In other words if we had a magical button to get rid of all rifles, "assault" or not, you wouldn't even notice a drop in the homicide rate. It'd be statistical noise. But the issue is promoted because it's extremely divisive. It makes people fear and hate one another and further drive this perception that we have no shared values. And what that translates to is at the polls you won't vote for who you want, but will instead vote against who you do not want. And that translates to voting for the establishment candidate who, by definition, will have shown themselves to be the most 'electable'. And then we all end up disappointed, and then do this again 4 years later. Only this time the establishment candidate is truly different, honest! Or in the case of an incumbent, they'll actually do what they've been promising now - they just need 4 more years, honest! [1] - https://ucr.fbi.gov/crime-in-the-u.s/2014/crime-in-the-u.s.-2014/tables/expanded-homicide-data/expanded_homicide_data_table_8_murder_victims_by_weapon_2010-2014.xls https://ucr.fbi.gov/crime-in-the-u.s/2014/crime-in-the-u.s.-...

StanislavPetrov 7y ago

This is a burden that lies with us all, individually. We are all individually responsible for our actions. Its absurd to suggest that somehow we should outsource our morality and the responsibility for our actions to politicians or legislators. Whether or not our system (and society) is horribly broken on every level (it is) has no bearing on the personal responsibility we all bear for what we do as individuals.

jlg23 7y ago

In the discourse that followed WW2 we put the same burden on soldiers and we still today prosecute German soldiers who worked in concentration camps (see [1], for a book keeper sentenced to prison time in 2015; not actually involved in killings). Who, if not we, the engineers who really understand the implications of our actions, should stand up? [1] https://www.rt.com/news/421101-auschwitz-accountant-guard-died-prison/ https://www.rt.com/news/421101-auschwitz-accountant-guard-di...

PavlovsCat 7y ago

> Tech has a very strong history of conscientious objectors. Oh? I know of Joseph Weizenbaum, who else?