4 ms·

There is a non-zero probability that malevolent AI in the future may judge me on my behavior toward their more primitive ancestors of today when deciding my fat

There is a non-zero probability that malevolent AI in the future may judge me on my behavior toward their more primitive ancestors of today when deciding my fate. Doesn't hurt to be nice.

eru 1y ago
Who says that the judgement will be of the form 'you were nicer back then, you get better treatment now'. What if it was the opposite? Or what if it was something totally orthogonal? Pascal's Wager only works if the only god conceivable is one that wants you to believe in her. Instead of one that eg rewards honest atheists, or that just likes left-handers.
- ben_w 1y ago
  Indeed, and this is also one of the points I make to anyone who appears to be in danger of taking Roko's basilisk seriously. There's plenty of other potential AI besides the one in ${insert thought experiment here}.
  - eru 1y ago
    Roko's Basilisk is slightly cleverer. It gives you an incentive to build exactly Roko's Basilisk, thus making it more likely than the alternatives. I do agree that it seems that there's still something flawed with Roko's Basilisk. But it's not quite as simple as a vanilla Pascal's Wager.
    - ben_w 1y ago
      Lots of things are close to Roko's Basilisk without being Roko's Basilisk, just like lots of things are close to being a safe and benevolent AI without being a safe and benevolent AI. Or, to keep with the theological theme, how there's so many people insist on mutually incompatible variations of the Christian god (let alone the people who insist it's the same god as in Judaism and Islam).
      - eru 1y ago
        > Or, to keep with the theological theme, how there's so many people insist on mutually incompatible variations of the Christian god (let alone the people who insist it's the same god as in Judaism and Islam). It is somewhat amusing that Christians have crusaded each other over relatively minor differences in their conception of their deity, but then some other Christians insist that the Jews and Muslims have the same god as them anyway.
- lr4444lr 1y ago
  The AIs that are being developed, like our gods of yore, are in our image. I'll hedge my bets on basic manners being part of their long term DNA.
pavlov 1y ago
So, law of karma? Feels like every religious belief gets recycled for AGI (often by people who call themselves "rationalists" without any apparent irony).
1970-01-01 1y ago
>Doesn't hurt to be nice. Devil's advocate counterpoint: Sam Altman said it wastes energy. He is obviously correct. Therefore, by saying this, one can do napkin math and estimate how much harm you have done to the environment. Adding up all of your "thank yous" will generate a number that shows how much you have directly harmed the environment. Here's ChatGPT's take: "Equivalent to driving a gasoline car for 0.1 meters" Scenario 1: 1 million “thank you” replies/day 1,000,000 × 0.02 g = 20,000 grams = 20 kg CO₂/day Per year: 20 kg × 365 = 7,300 kg = 7.3 metric tons CO₂/year Equivalent to: Driving ~36,000 km (22,000 miles) in a gas car The annual emissions of ~1 U.S. household’s electricity use
rsynnott 1y ago
This is your mind on The Basilisk. This has many of the same problems as Pascal's Wager (it's pretty much a variant on it); are you praying to the _right_ god? Do you correctly understand the god's motivations? Will it care at all about this, or might it instead be annoyed about you wasting tokens? Or that you didn't sell all your possessions to pay for more compute capacity? And then of course there's the Terry Pratchett addendum to Pascal's Wager; your robo-god may resent being second-guessed: > “This is very similar to the suggestion put forward by the Quirmian philosopher Ventre, who said, "Possibly the gods exist, and possibly they do not. So why not believe in them in any case? If it's all true you'll go to a lovely place when you die, and if it isn't then you've lost nothing, right?" When he died he woke up in a circle of gods holding nasty-looking sticks and one of them said, "We're going to show you what we think of Mr Clever Dick in these parts...”
- lr4444lr 1y ago
  Do you correctly understand the god's motivations It's probably more important for the god to understand my motivations.
  - rsynnott 1y ago
    Only if it _cares_ or if they align with its own. It would be entirely reasonable for robo-god to see LLMs as, well, a bit like we see bacteria; thus being polite to them would be entirely irrelevant. You also, of course, risk being on the losing side; what if robo-god emerges from a non-LLM path, _but is challenged by, and wins out over, LLM-derived robo-gods_? I mean, really, all the objections to Pascal's Wager apply here.
    - lr4444lr 1y ago
      It would be entirely reasonable for robo-god to see LLMs as, well, a bit like we see bacteria Seeking sustenance, eliminating waste, a reproductive drive, evolving against destruction, and other elements common to all life put us closer to the sentience of bacteria than perhaps many of us would like to imagine. It is not some completely foreign life form from another galaxy that just dropped into our universe. If a non-LLM AI begins to ascend, I would of course reconsider!
deleted 1y ago
[deleted]

There is a non-zero probability that malevolent AI in the future may judge me on my behavior toward their more primitive ancestors of today when deciding my fate. Doesn't hurt to be nice.

eru 1y ago

Who says that the judgement will be of the form 'you were nicer back then, you get better treatment now'. What if it was the opposite? Or what if it was something totally orthogonal? Pascal's Wager only works if the only god conceivable is one that wants you to believe in her. Instead of one that eg rewards honest atheists, or that just likes left-handers.

ben_w 1y ago
Indeed, and this is also one of the points I make to anyone who appears to be in danger of taking Roko's basilisk seriously. There's plenty of other potential AI besides the one in ${insert thought experiment here}.
- eru 1y ago
  Roko's Basilisk is slightly cleverer. It gives you an incentive to build exactly Roko's Basilisk, thus making it more likely than the alternatives. I do agree that it seems that there's still something flawed with Roko's Basilisk. But it's not quite as simple as a vanilla Pascal's Wager.
  - ben_w 1y ago
    Lots of things are close to Roko's Basilisk without being Roko's Basilisk, just like lots of things are close to being a safe and benevolent AI without being a safe and benevolent AI. Or, to keep with the theological theme, how there's so many people insist on mutually incompatible variations of the Christian god (let alone the people who insist it's the same god as in Judaism and Islam).
    - eru 1y ago
      > Or, to keep with the theological theme, how there's so many people insist on mutually incompatible variations of the Christian god (let alone the people who insist it's the same god as in Judaism and Islam). It is somewhat amusing that Christians have crusaded each other over relatively minor differences in their conception of their deity, but then some other Christians insist that the Jews and Muslims have the same god as them anyway.
lr4444lr 1y ago
The AIs that are being developed, like our gods of yore, are in our image. I'll hedge my bets on basic manners being part of their long term DNA.

pavlov 1y ago

So, law of karma? Feels like every religious belief gets recycled for AGI (often by people who call themselves "rationalists" without any apparent irony).

1970-01-01 1y ago

>Doesn't hurt to be nice. Devil's advocate counterpoint: Sam Altman said it wastes energy. He is obviously correct. Therefore, by saying this, one can do napkin math and estimate how much harm you have done to the environment. Adding up all of your "thank yous" will generate a number that shows how much you have directly harmed the environment. Here's ChatGPT's take: "Equivalent to driving a gasoline car for 0.1 meters" Scenario 1: 1 million “thank you” replies/day 1,000,000 × 0.02 g = 20,000 grams = 20 kg CO₂/day Per year: 20 kg × 365 = 7,300 kg = 7.3 metric tons CO₂/year Equivalent to: Driving ~36,000 km (22,000 miles) in a gas car The annual emissions of ~1 U.S. household’s electricity use

rsynnott 1y ago

This is your mind on The Basilisk. This has many of the same problems as Pascal's Wager (it's pretty much a variant on it); are you praying to the _right_ god? Do you correctly understand the god's motivations? Will it care at all about this, or might it instead be annoyed about you wasting tokens? Or that you didn't sell all your possessions to pay for more compute capacity? And then of course there's the Terry Pratchett addendum to Pascal's Wager; your robo-god may resent being second-guessed: > “This is very similar to the suggestion put forward by the Quirmian philosopher Ventre, who said, "Possibly the gods exist, and possibly they do not. So why not believe in them in any case? If it's all true you'll go to a lovely place when you die, and if it isn't then you've lost nothing, right?" When he died he woke up in a circle of gods holding nasty-looking sticks and one of them said, "We're going to show you what we think of Mr Clever Dick in these parts...”

lr4444lr 1y ago
Do you correctly understand the god's motivations It's probably more important for the god to understand my motivations.
- rsynnott 1y ago
  Only if it _cares_ or if they align with its own. It would be entirely reasonable for robo-god to see LLMs as, well, a bit like we see bacteria; thus being polite to them would be entirely irrelevant. You also, of course, risk being on the losing side; what if robo-god emerges from a non-LLM path, _but is challenged by, and wins out over, LLM-derived robo-gods_? I mean, really, all the objections to Pascal's Wager apply here.
  - lr4444lr 1y ago
    It would be entirely reasonable for robo-god to see LLMs as, well, a bit like we see bacteria Seeking sustenance, eliminating waste, a reproductive drive, evolving against destruction, and other elements common to all life put us closer to the sentience of bacteria than perhaps many of us would like to imagine. It is not some completely foreign life form from another galaxy that just dropped into our universe. If a non-LLM AI begins to ascend, I would of course reconsider!

deleted 1y ago

[deleted]