2 ms·

This same kind of thought process--that one should strive not for "do no evil" but "can't do evil"--really applies everywhere, as it is somewhat general: amassi

by saurik 5y ago

This same kind of thought process--that one should strive not for "do no evil" but "can't do evil"--really applies everywhere, as it is somewhat general: amassing control over other people and their resources (money, data, whatever) is always going to be dangerous. Maybe you are good today, but in the future you might start to be swayed by changing incentives or situations due to forces such as "absolutely power corrupts absolutely". Or maybe you manage to always be good, but--as humans have fixed life spans--eventually retire or die or simply move on and are replaced by someone who is less good than you are. Or maybe you are good but the power you manage to concentrate gets stolen by someone (in the digital world, maybe you get hacked) and used without your permission to do bad things. Or maybe you want to be good, but your power is seen as an asset for something external--such as a government--and you end up being required to do bad things that make you sad. We see all of these issues play out constantly with large tech companies, with control techniques such as curated application markets getting abused as anti-competitive measures, or getting regulated by authoritarian governments as a tool for their regime. In 2017, I gave a talk at Mozilla Privacy Lab that looked at many of these issues, citing tons of situations--every slide is a screenshot of a news source, as somehow people always want to believe these situations are far-fetched--where having control has gone badly: https://youtu.be/vsazo-Gs7ms https://youtu.be/vsazo-Gs7ms

meheleventyone 5y ago
But there is also “can only do evil” where you accidentally set up a system you can’t fix.
- saurik 5y ago
  If users agree that the thing you have built is broken and want to opt into using a new less-broken thing, they can always do that, as would be the case (say) with software they download from you and are running on their computer (rather than software you host on your server, that you can change at will): you don't need the power to "reach your grubby mitts"--to put it bluntly--into their lives and fix what you built on their behalf in order for broken things to be fixed, and your definition of what is "broken" can easily be at odds with the user's preferences or even needs (which of course begs the entire question of how to avoid the incentive to be evil in the first place).
  - DarylZero 5y ago
    > If users ... want to opt into using a new less-broken thing, they can always do that Users don't necessarily have that much agency. The most successful and powerful businesses don't need to satisfy user preferences because the users are FORCED to conform THEMSELVES to the constraints imposed by the business.
    - tata71 5y ago
      > Users don't necessarily have that much agency. It's time to fix that. Web 1.0: Read Web 2.0: Read/Write Web 3.0: Read/Write/Own
      - CrazyStat 5y ago
        Based on the weekly reports of $XX million being stolen from $DEFI_ENTITY, web3 seems more like read/write/pwn to me.
        tata71 5y ago
        Have you opened a Twitter account with that wit? The fact is that you cannot verifiably compare those numbers to traditional finance hacks, because they're simply not transparently reported/cleaned up before the damage can be noticed. Don't fault a good in-built bug bounty for bringing about the next generation of strong, distributed infrastructure.
        viraptor 5y ago
        Traditional finance hacks usually abuse tax / debt of large enough companies. Web3 finance hacks take all available money directly out of your personal account. That's a much worse situation for a normal user, because with a bank you can often say "this is wrong, fix this" and the bank is insured enough that they will. Compare to the situation in Venezuela where the in-built bug bounty currently takes a significant amount of people's savings and they have no way to revert it.
saurik 5y ago
Ugh. I just barely missed the window to fix that "absolutely power corrupts absolutely" typo, and in the process was breaking that paragraph into two, as I think this better explains my vague "changing incentives or situations". Here, improved: > Maybe you are good today, but--as "absolute power corrupts absolutely"--you might find yourself incrementally abusing your power or even "selling out" in the future without even feeling yourself change. > Or maybe you always do good things, but due to a tragic change of circumstances--such as an expensive health issue with a family member--it becomes difficult to decide what being good means for you. > Or maybe you manage to be good "forever", ... > Or maybe even your heirs are all truly good, but the power you managed to concentrate ...

by saurik 5y ago

meheleventyone 5y ago

But there is also “can only do evil” where you accidentally set up a system you can’t fix.

saurik 5y ago
If users agree that the thing you have built is broken and want to opt into using a new less-broken thing, they can always do that, as would be the case (say) with software they download from you and are running on their computer (rather than software you host on your server, that you can change at will): you don't need the power to "reach your grubby mitts"--to put it bluntly--into their lives and fix what you built on their behalf in order for broken things to be fixed, and your definition of what is "broken" can easily be at odds with the user's preferences or even needs (which of course begs the entire question of how to avoid the incentive to be evil in the first place).
- DarylZero 5y ago
  > If users ... want to opt into using a new less-broken thing, they can always do that Users don't necessarily have that much agency. The most successful and powerful businesses don't need to satisfy user preferences because the users are FORCED to conform THEMSELVES to the constraints imposed by the business.
  - tata71 5y ago
    > Users don't necessarily have that much agency. It's time to fix that. Web 1.0: Read Web 2.0: Read/Write Web 3.0: Read/Write/Own
    - CrazyStat 5y ago
      Based on the weekly reports of $XX million being stolen from $DEFI_ENTITY, web3 seems more like read/write/pwn to me.
      - tata71 5y ago
        Have you opened a Twitter account with that wit? The fact is that you cannot verifiably compare those numbers to traditional finance hacks, because they're simply not transparently reported/cleaned up before the damage can be noticed. Don't fault a good in-built bug bounty for bringing about the next generation of strong, distributed infrastructure.
        viraptor 5y ago
        Traditional finance hacks usually abuse tax / debt of large enough companies. Web3 finance hacks take all available money directly out of your personal account. That's a much worse situation for a normal user, because with a bank you can often say "this is wrong, fix this" and the bank is insured enough that they will. Compare to the situation in Venezuela where the in-built bug bounty currently takes a significant amount of people's savings and they have no way to revert it.

saurik 5y ago

Ugh. I just barely missed the window to fix that "absolutely power corrupts absolutely" typo, and in the process was breaking that paragraph into two, as I think this better explains my vague "changing incentives or situations". Here, improved: > Maybe you are good today, but--as "absolute power corrupts absolutely"--you might find yourself incrementally abusing your power or even "selling out" in the future without even feeling yourself change. > Or maybe you always do good things, but due to a tragic change of circumstances--such as an expensive health issue with a family member--it becomes difficult to decide what being good means for you. > Or maybe you manage to be good "forever", ... > Or maybe even your heirs are all truly good, but the power you managed to concentrate ...