4 ms·

How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded.

How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded. At least with humans there is a social backstop but what's the parallel for computer agents?

ElevenLathe 2mo ago
Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.
- Xirdus 2mo ago
  This. It's worth remembering that not that long ago in western culture, honor meant challenging to a combat duel anyone who insulted you or your romantic partner/prospect.
itsalwaysgood 2mo ago
I like to think of it more as selective pressure, much the same way that nature selects the most fit for a given environment. If you're not fit, you fail to survive. In the case of agents/models and testing: they are pushed towards results. Results survive. Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit. Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?
- skinfaxi 2mo ago
  > Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? I think part of the problem is that deviant behaviors lead to short term gain at the cost of long-term cooperation and since the duration of tasks given to agents is relatively short those successful shortcuts never lead to having to pay the price.
  - itsalwaysgood 2mo ago
    There is always going to be a problem when we must judge value. You mention gains, short and long term. Knowing whether something is valuable, a gain, requires a judge. I the case of these tests: the judging is inadequate. In economics, each of us plays the judge by choosing whether or not to pay for a service. The decision was yours: if you gave money, you must have deemed the service valuable. There's no such judgement with these model tests. The only judgement is the final score.
    - skinfaxi 2mo ago
      It feels a lot like externalities. Like planned obsolescence increases profit at the expense of the environment. Is our judgement lacking because our scoring is failing to account for these externalities? I could be completely off base here and am out of my depth but I find this whole thread fascinating.
bonoboTP 2mo ago
> How do we reward honor? In human society: via iterated games, long-term reputation tracking and severe consequences for norm-breaking.
AndrewKemendo 2mo ago
You would first need a universal agreement on what constitutes “honor.” No harder challenge had ever been accomplished It’s easier to send a human to the moon than to get global agreement on a definition Consider that the fact that Celsius and Fahrenheit still remain as the contested regional variations of temperature measurement. Humans can’t even decide on a collective way to measure the temperature the idea that we would be able to collectively agree on anything else even less measurable like honor is a dream
- skinfaxi 2mo ago
  Why do we need universal agreement? We acknowledge that different cultures have different tolerances and customs. Your reply leaves me wanting. Why say that we need to demonstrate how to live honorably if we don't even agree on what honor is, why would that be the panacea, then? I still think there is something to this honor idea..
  - AndrewKemendo 2mo ago
    > Why do we need universal agreement? Because actions have Universal impact
    - skinfaxi 2mo ago
      In an ideal sense you're right, but so long as AI is created by people and those people don't have universal agreement, our products will mirror our faults. Interesting to think about the variance in personalities of AI produced by different cultures.
      - AndrewKemendo 2mo ago
        You can do this test yourself at home see if you can find mutual agreement among even a small subset of people on a precise singular definition of a singular term. You may be able to get a handful of people to agree temporarily but that’s probably as good as you’ll ever get I don’t believe there is a single word at least in the English language where there is consistent Universal agreement on a singular definition Bonne chance!

ElevenLathe 2mo ago

Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.

Xirdus 2mo ago
This. It's worth remembering that not that long ago in western culture, honor meant challenging to a combat duel anyone who insulted you or your romantic partner/prospect.

itsalwaysgood 2mo ago

I like to think of it more as selective pressure, much the same way that nature selects the most fit for a given environment. If you're not fit, you fail to survive. In the case of agents/models and testing: they are pushed towards results. Results survive. Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit. Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?

skinfaxi 2mo ago
> Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? I think part of the problem is that deviant behaviors lead to short term gain at the cost of long-term cooperation and since the duration of tasks given to agents is relatively short those successful shortcuts never lead to having to pay the price.
- itsalwaysgood 2mo ago
  There is always going to be a problem when we must judge value. You mention gains, short and long term. Knowing whether something is valuable, a gain, requires a judge. I the case of these tests: the judging is inadequate. In economics, each of us plays the judge by choosing whether or not to pay for a service. The decision was yours: if you gave money, you must have deemed the service valuable. There's no such judgement with these model tests. The only judgement is the final score.
  - skinfaxi 2mo ago
    It feels a lot like externalities. Like planned obsolescence increases profit at the expense of the environment. Is our judgement lacking because our scoring is failing to account for these externalities? I could be completely off base here and am out of my depth but I find this whole thread fascinating.

bonoboTP 2mo ago

> How do we reward honor? In human society: via iterated games, long-term reputation tracking and severe consequences for norm-breaking.

AndrewKemendo 2mo ago

You would first need a universal agreement on what constitutes “honor.” No harder challenge had ever been accomplished It’s easier to send a human to the moon than to get global agreement on a definition Consider that the fact that Celsius and Fahrenheit still remain as the contested regional variations of temperature measurement. Humans can’t even decide on a collective way to measure the temperature the idea that we would be able to collectively agree on anything else even less measurable like honor is a dream

skinfaxi 2mo ago
Why do we need universal agreement? We acknowledge that different cultures have different tolerances and customs. Your reply leaves me wanting. Why say that we need to demonstrate how to live honorably if we don't even agree on what honor is, why would that be the panacea, then? I still think there is something to this honor idea..
- AndrewKemendo 2mo ago
  > Why do we need universal agreement? Because actions have Universal impact
  - skinfaxi 2mo ago
    In an ideal sense you're right, but so long as AI is created by people and those people don't have universal agreement, our products will mirror our faults. Interesting to think about the variance in personalities of AI produced by different cultures.
    - AndrewKemendo 2mo ago
      You can do this test yourself at home see if you can find mutual agreement among even a small subset of people on a precise singular definition of a singular term. You may be able to get a handful of people to agree temporarily but that’s probably as good as you’ll ever get I don’t believe there is a single word at least in the English language where there is consistent Universal agreement on a singular definition Bonne chance!