5 ms·

If I'm not mis-remembering, there's an interesting section in Dawkins' The Selfish Gene about cooperative grooming behavior (I think it was in some sort of wate

If I'm not mis-remembering, there's an interesting section in Dawkins' The Selfish Gene about cooperative grooming behavior (I think it was in some sort of water fowl) and how the birds deal with cheating behavior (individuals that accept grooming without reciprocating). My takeaway was roughly: Any exchange of value based on trust is exploitable. The simplest cure is excluding the exploiter, but this doesn't scale well. The exploiter can skate on anonymity if the community is large enough to continually prey on someone new. Spreading news of an exploiter's behavior to others can greatly improve how well this scales, but this behavior also requires trust. I think the more direct problem is with scale, and that the internet is at the nexus of many trust issues only because it has ramped up the scale and scope of many interactions. I'm not super optimistic on solving this intrinsic problem of trust in social exchanges, but I do see this framing as a silver lining. It seems at least plausible to iterate offline at significantly smaller scales on mechanisms for building and maintaining trust--and rectifying its breaches--in ways that do actually scale.

Sushi-san 7y ago
So, should we just try to ignore the larger-scale exploitation of trust?
- abathur 7y ago
  No, we shouldn't ignore it. But I think there's a risk of sending a lot of bright, productive people off to tilt at technological/interface/regulatory windmills in a way that leaves fundamental trust issues out of scope. Even if Amazon, Twitter, Facebook, etc. could wave a magic wand and remove every fake review, counterfeit product, scam, personal threat, or piece of false/fraudulent media, there would still be a trust issue. We still have to trust that they did what they said, that whatever disappeared was correctly identified, and that they didn't wrongly remove many legitimate items in the process. Even if the magic wand that makes these decisions has the utmost ethical and logical integrity, there will still be some mix of skeptics, cynics, malign actors, competitors, bots, etc. who chum the waters with accusations to the contrary. We'd still have to choose whether or not to trust this process and the actors behind it. So, I think it might be productive to focus on some smaller questions first. To be semi-arbitrary: can we find a protocol for reliably building a 50-person trust network with a limited scope/focus (maybe identifying reliable providers of a single service), where each participant knows a small fraction of the network, which is capable of meeting both its purpose and is capable of detecting and reforming or ejecting exploiters? If the first is tractable, can you expand the scope/focus of the networks and retain these properties? Can you compose a higher-order network that retains these properties?
  - petermcneeley 7y ago
    These networks would be scalable by using derived trust values. In simple terms you trust others to provide you with the trust value of people you have never met. I'm sure one could mathematically prove certain implementations to be unexploitable.
    - felipemnoa 7y ago
      >>In simple terms you trust others to provide you with the trust value of people you have never met. References? Credit Score?
      - AnthonyMouse 7y ago
        > References? Credit Score? Both of those are easily gamed in the absence of trust. For references the mechanism is obvious. They list co-conspirators as references who then lie about their titles. For credit reporting it seems like you could do the same thing over time. Create fictitious entities to make positive filings with credit reporting agencies. I assume the reason that isn't more popular is that it's even faster to do it the other way and steal the identity of somebody with good credit.
  - rlucas 7y ago
    One of the things the internet has been really great at is allowing those small (n~50) networks with high trust for e.g. identifying trusted vendors, but for really niche things where it’s hard to get an organic local network. Think new programming languages, or to crib from a recent HN topic, building tube headphone amps. These kind of emergent niche communities can only exist when the broader network gets sufficient scale to have a critical mass in the niche. So in limiting scale overall let’s take care not to throw out the baby etc.
    - abathur 7y ago
      I'm not sure I see hard top-down scale limits as a goal. I don't think this would be in the best interest of most people with a hand on the steering wheel of large social networks. I'm imagining something more along the lines of: can we learn how to turn what small/niche networks do well into building blocks, and does that knowledge teach us anything we could use to (I guess adversarially) re-structure/reform some environments that are currently trust bonfires.
nixarian 7y ago
That's interesting. How equally dangerous would it be if, for example, someone spread misinformation about a party in an attempt to label them as untrustworthy. You could control opinion that way, by labeling anyone who disagrees with you as an outcast.
m463 7y ago
> Any exchange of value based on trust is exploitable I was thinking "money screws things up" and its corollary "free screws things up", then read your comment. It gets to the root of things. With trust, trade is unlimited, so I wonder how to architect things to stand up to the problems we have.
- orpep90nxkfo 7y ago
  The obvious is a robust welfare state. I mean it seems pretty clear to now that inequality goes up, social stability decreases, major overhaul, a generation of babysitting it... like just stop pandering to the meme we can’t afford universal healthcare and college education. Cause we also cannot afford leashing the next generation to soon to dead men’s gambling debts. But that effectively ends the point in aristocracy, the meme that we must cater to all these rugged individuals living off grandpas old money. So that’ll never fly. Violent revolt it is! Front row to the apocalypse! /s
tylerjwilk00 7y ago
My favorite concept from the book as well! The name of the behavior you're looking for is Evolutionarily Stable Strategy [1] often abbreviated to just ESS. [1] https://en.m.wikipedia.org/wiki/Evolutionarily_stable_strategy https://en.m.wikipedia.org/wiki/Evolutionarily_stable_strate...
schaefer 7y ago
It's true that "The Selfish Gene" has been a science-based classic of evolutionary biology for over 40 years. But it's also true that in those 40 years a lot of studies have taken aim at the central arguments of the book and in my opinion cast serious doubt on the accuracy of the books conclusions (Or perhaps limit the scope of those views as being a projection of western culture - but not of all cultures on earth, particularly not eastern philosophy, and not the animal kingdom at large) If you have never read any of these counterpoints, but find that the conclusions of Selfish Gene have shaped your world view, please consider reading some of these counterpoints and seeing if they persuade you to consider new perspectives. A particularly thorough book detailing these counterpoints is Matthieu Ricard's book: Altruism: The Power of Compassion to Change Yourself and the World
- andreilys 7y ago
  Any chance you can post some of these counterpoints? At first glance I’m not sure how a book on altruism/compassion refutes the gene-centred view of evolution, which holds that evolution is best viewed as acting on genes and that selection at the level of organisms or populations almost never overrides selection based on genes.
- crazygringo 7y ago
  The book you're recommending seems to be about psychological and spiritual relationships between human beings, from what I can gather from a quick skim on Amazon. But not a single one of the reviews actually describes a single argument the book is making (strange), so it's hard to judge. Dawkins' "The Selfish Gene" is a scientific work about evolutionary selection at the gene level -- I don't recall him touching on psychology at all. (He provides evolutionary explanations for certain altruistic behaviors, but I don't recall him even starting on how they might be expressed via a psychological mechanism or at any conscious level.) I can't quite imagine what either have to do with each other -- they seem to be such different topics. Or what Dawkins' work has to do with "western culture", or culture at all. At heart it's a quite mathematical/statistical argument. I'm curious, what exactly do you see being refuted?
  - 7y ago
raxxorrax 7y ago
I think it is just a formalization of the thoughts of egoists of the 19th century. Trust is a resource and is (only) used up if exploited. There are countless examples how this happened on the net. Remember the latest disclaimer you agreed to and consequently had your data stolen or abused? Don't even get me started on penis enlargement spam. Then additionally, people pretending to have identified the abusers very frequently become the largest abusers themselves. Lastly, trust needs room to grow, so trying to enforce it through surveillance and advertising will only strengthen the individuals ego. So... > can greatly improve how well this scales, but this behavior also requires trust. can backfire immensely. People have already tried and failed. There is no intrinsic problem with trust. On the net you are potentially connected to everyone on the planet. Nobody can claim the trust of everyone. And I firmly believe achieving this is the wrong goal. People need to simple not let their trust be abused and that is very possible. For example not letting ad companies handle your biases or believing the next random spam mail you get. Anonymity is a tool to shield you from abuse. Not the only one, but for non-public personas, nobody has found something better yet. If you keep looking for new predators, everybody will look like one at some point. Because you have let your trust be abused and have none left.
- abathur 7y ago
  > There is no intrinsic problem with trust. On the net you are potentially connected to everyone on the planet. Nobody can claim the trust of everyone. And I firmly believe achieving this is the wrong goal. > People need to simple not let their trust be abused and that is very possible. For example not letting ad companies handle your biases or believing the next random spam mail you get. > Anonymity is a tool to shield you from abuse. Not the only one, but for non-public personas, nobody has found something better yet. I'm not sure it's possible to distinguish the world you describe from the one we already live in.
SomeOldThrow 7y ago
Maybe I’m reading this wrong, but isn’t this just describing an echo chamber? Sure that’s nice with concepts like justice, but if the internet shows anything it’s that we thought we agreed on things far more than we actually do.
AnimalMuppet 7y ago
Interestingly, one could view China's social credit score system as a way of dealing with this trust issue. Note well: I am not advocating either a social credit score system, nor China's use of it. Instead, I am warning that any national-level attempt to solve the trust issue will likely be open to the same abuses as China's system.
- AstralStorm 7y ago
  That only works until enough people exploit the system in some way. (Both good and bad score...) Specifically, once it becomes a means of political control it ceases to be useful as a system for trustworthiness.

Sushi-san 7y ago

So, should we just try to ignore the larger-scale exploitation of trust?

abathur 7y ago
No, we shouldn't ignore it. But I think there's a risk of sending a lot of bright, productive people off to tilt at technological/interface/regulatory windmills in a way that leaves fundamental trust issues out of scope. Even if Amazon, Twitter, Facebook, etc. could wave a magic wand and remove every fake review, counterfeit product, scam, personal threat, or piece of false/fraudulent media, there would still be a trust issue. We still have to trust that they did what they said, that whatever disappeared was correctly identified, and that they didn't wrongly remove many legitimate items in the process. Even if the magic wand that makes these decisions has the utmost ethical and logical integrity, there will still be some mix of skeptics, cynics, malign actors, competitors, bots, etc. who chum the waters with accusations to the contrary. We'd still have to choose whether or not to trust this process and the actors behind it. So, I think it might be productive to focus on some smaller questions first. To be semi-arbitrary: can we find a protocol for reliably building a 50-person trust network with a limited scope/focus (maybe identifying reliable providers of a single service), where each participant knows a small fraction of the network, which is capable of meeting both its purpose and is capable of detecting and reforming or ejecting exploiters? If the first is tractable, can you expand the scope/focus of the networks and retain these properties? Can you compose a higher-order network that retains these properties?
- petermcneeley 7y ago
  These networks would be scalable by using derived trust values. In simple terms you trust others to provide you with the trust value of people you have never met. I'm sure one could mathematically prove certain implementations to be unexploitable.
  - felipemnoa 7y ago
    >>In simple terms you trust others to provide you with the trust value of people you have never met. References? Credit Score?
    - AnthonyMouse 7y ago
      > References? Credit Score? Both of those are easily gamed in the absence of trust. For references the mechanism is obvious. They list co-conspirators as references who then lie about their titles. For credit reporting it seems like you could do the same thing over time. Create fictitious entities to make positive filings with credit reporting agencies. I assume the reason that isn't more popular is that it's even faster to do it the other way and steal the identity of somebody with good credit.
- rlucas 7y ago
  One of the things the internet has been really great at is allowing those small (n~50) networks with high trust for e.g. identifying trusted vendors, but for really niche things where it’s hard to get an organic local network. Think new programming languages, or to crib from a recent HN topic, building tube headphone amps. These kind of emergent niche communities can only exist when the broader network gets sufficient scale to have a critical mass in the niche. So in limiting scale overall let’s take care not to throw out the baby etc.
  - abathur 7y ago
    I'm not sure I see hard top-down scale limits as a goal. I don't think this would be in the best interest of most people with a hand on the steering wheel of large social networks. I'm imagining something more along the lines of: can we learn how to turn what small/niche networks do well into building blocks, and does that knowledge teach us anything we could use to (I guess adversarially) re-structure/reform some environments that are currently trust bonfires.

nixarian 7y ago

That's interesting. How equally dangerous would it be if, for example, someone spread misinformation about a party in an attempt to label them as untrustworthy. You could control opinion that way, by labeling anyone who disagrees with you as an outcast.

m463 7y ago

> Any exchange of value based on trust is exploitable I was thinking "money screws things up" and its corollary "free screws things up", then read your comment. It gets to the root of things. With trust, trade is unlimited, so I wonder how to architect things to stand up to the problems we have.

orpep90nxkfo 7y ago
The obvious is a robust welfare state. I mean it seems pretty clear to now that inequality goes up, social stability decreases, major overhaul, a generation of babysitting it... like just stop pandering to the meme we can’t afford universal healthcare and college education. Cause we also cannot afford leashing the next generation to soon to dead men’s gambling debts. But that effectively ends the point in aristocracy, the meme that we must cater to all these rugged individuals living off grandpas old money. So that’ll never fly. Violent revolt it is! Front row to the apocalypse! /s

tylerjwilk00 7y ago

My favorite concept from the book as well! The name of the behavior you're looking for is Evolutionarily Stable Strategy [1] often abbreviated to just ESS. [1] https://en.m.wikipedia.org/wiki/Evolutionarily_stable_strategy https://en.m.wikipedia.org/wiki/Evolutionarily_stable_strate...

schaefer 7y ago

It's true that "The Selfish Gene" has been a science-based classic of evolutionary biology for over 40 years. But it's also true that in those 40 years a lot of studies have taken aim at the central arguments of the book and in my opinion cast serious doubt on the accuracy of the books conclusions (Or perhaps limit the scope of those views as being a projection of western culture - but not of all cultures on earth, particularly not eastern philosophy, and not the animal kingdom at large) If you have never read any of these counterpoints, but find that the conclusions of Selfish Gene have shaped your world view, please consider reading some of these counterpoints and seeing if they persuade you to consider new perspectives. A particularly thorough book detailing these counterpoints is Matthieu Ricard's book: Altruism: The Power of Compassion to Change Yourself and the World

andreilys 7y ago
Any chance you can post some of these counterpoints? At first glance I’m not sure how a book on altruism/compassion refutes the gene-centred view of evolution, which holds that evolution is best viewed as acting on genes and that selection at the level of organisms or populations almost never overrides selection based on genes.
crazygringo 7y ago
The book you're recommending seems to be about psychological and spiritual relationships between human beings, from what I can gather from a quick skim on Amazon. But not a single one of the reviews actually describes a single argument the book is making (strange), so it's hard to judge. Dawkins' "The Selfish Gene" is a scientific work about evolutionary selection at the gene level -- I don't recall him touching on psychology at all. (He provides evolutionary explanations for certain altruistic behaviors, but I don't recall him even starting on how they might be expressed via a psychological mechanism or at any conscious level.) I can't quite imagine what either have to do with each other -- they seem to be such different topics. Or what Dawkins' work has to do with "western culture", or culture at all. At heart it's a quite mathematical/statistical argument. I'm curious, what exactly do you see being refuted?
- 7y ago

raxxorrax 7y ago

I think it is just a formalization of the thoughts of egoists of the 19th century. Trust is a resource and is (only) used up if exploited. There are countless examples how this happened on the net. Remember the latest disclaimer you agreed to and consequently had your data stolen or abused? Don't even get me started on penis enlargement spam. Then additionally, people pretending to have identified the abusers very frequently become the largest abusers themselves. Lastly, trust needs room to grow, so trying to enforce it through surveillance and advertising will only strengthen the individuals ego. So... > can greatly improve how well this scales, but this behavior also requires trust. can backfire immensely. People have already tried and failed. There is no intrinsic problem with trust. On the net you are potentially connected to everyone on the planet. Nobody can claim the trust of everyone. And I firmly believe achieving this is the wrong goal. People need to simple not let their trust be abused and that is very possible. For example not letting ad companies handle your biases or believing the next random spam mail you get. Anonymity is a tool to shield you from abuse. Not the only one, but for non-public personas, nobody has found something better yet. If you keep looking for new predators, everybody will look like one at some point. Because you have let your trust be abused and have none left.

abathur 7y ago
> There is no intrinsic problem with trust. On the net you are potentially connected to everyone on the planet. Nobody can claim the trust of everyone. And I firmly believe achieving this is the wrong goal. > People need to simple not let their trust be abused and that is very possible. For example not letting ad companies handle your biases or believing the next random spam mail you get. > Anonymity is a tool to shield you from abuse. Not the only one, but for non-public personas, nobody has found something better yet. I'm not sure it's possible to distinguish the world you describe from the one we already live in.

SomeOldThrow 7y ago

Maybe I’m reading this wrong, but isn’t this just describing an echo chamber? Sure that’s nice with concepts like justice, but if the internet shows anything it’s that we thought we agreed on things far more than we actually do.

AnimalMuppet 7y ago

Interestingly, one could view China's social credit score system as a way of dealing with this trust issue. Note well: I am not advocating either a social credit score system, nor China's use of it. Instead, I am warning that any national-level attempt to solve the trust issue will likely be open to the same abuses as China's system.

AstralStorm 7y ago
That only works until enough people exploit the system in some way. (Both good and bad score...) Specifically, once it becomes a means of political control it ceases to be useful as a system for trustworthiness.