10 ms·
Pascal's Mugging (2009) [pdf]
- dang 7y agoRelated from 2011: https://news.ycombinator.com/item?id=2932157 https://news.ycombinator.com/item?id=2932157
- samspenc 7y agoThis is likely trending because John Carmack's recent post saying he was moving to AI references Pascal's Mugging: https://www.facebook.com/100006735798590/posts/2547632585471243/ https://www.facebook.com/100006735798590/posts/2547632585471... His Facebook post was also discussed in detail on Hacker News here: https://news.ycombinator.com/item?id=21530860 https://news.ycombinator.com/item?id=21530860
- Erlich_Bachman 7y agoI really love how this is could be seen as a demonstration that HN (or any other similar interest community, or probably the internet in general) functions like a hive mind, a collective consciousness. I've read that post by Carmack and I wondered exactly that "wonder what Pascal's Mugging is, interesting". Sometimes you go around and wonder various things, but don't look them up or do them in the moment, and then sometime later your subconscious mind serves you up with an answer, perhaps when you are more relaxed you just think up of the answer, or it happens to come up in a certain context, the subconscious lights it up there. It might be a word that you see randomly in a newspaper, or you think of a person that was related when you met them etc. And here the subconscious did the same wonderous thing, except it wasn't even strictly my personal subconsciousness, it was the group subconscious that found the information and presented it.
- peterwwillis 7y agoBut how useful is it? Pascal's Mugging was submitted to HN and discussed 8 years ago. If the collective consciousness keeps needing reminders of what it once knew, this is probably still inferior to a single intelligent person who reads a lot and remembers it all.
- Erlich_Bachman 7y agoAs a somewhat intelligent person, I still also do need reminders often. The trick is to forget mostly the unneeded stuff and maximally retain the useful stuff, and accurately discern between the two. The goal is not to remember everything. Also it would be strange to expect this group consciousness to never need reminders when it continuously has new people added to it, who are unfamiliar with old things. I think consciousness is more about active living rehashing of information and reupdating it, reupdating the worldview to adjust to constantly changing environment - not so much about building one single model that would somehow know everything. Such models tend to be stale or abstract and philosophical to the point of uselessness.
- ncmncm 7y agoThe population using HN takes in new members continuously, and the fraction who read everything posted eight years ago is very small.
- brian_herman__ 7y agoIt’s new to me :)
- pennaMan 7y agoI have noticed this pattern before: Someone mentions a topic deep inside a thread on HN and next thing you know that topic is on the front page.
- maest 7y agoIt could also be confirmation bias, though. The Baader-Meinhof thing.
- jon_akimbo 7y agoLooking forward to seeing the Baader-Mainhof phenomenon on the front page later today or tomorrow.
- asudosandwich 7y agoI only had to wait for the next comment.
- roenxi 7y agoThis can be easily resolved by considering that the victim of the mugging has finite resources. This is a usual remedy to problems were expected value alone gives stupid results (such as Pascal's Mugging). Something similar happens in lotteries where even if the expected value of buying a ticket is positive it is still not rational for an ordinary person to buy a ticket. If I have $400 dollars I can't afford to take 1:1000000 risks that cost $200 each. I will go bankrupt with an enormous likelihood whatever the payoff. There is a minimum cutoff involving cost/probability below which it does not make sense to take up the opportunity. There are links to similar theoretical ideas from the Pascale's Mugging wiki page - although from the casinos perspective not the gambler's - https://en.wikipedia.org/wiki/St._Petersburg_paradox#Finite_St._Petersburg_lotteries https://en.wikipedia.org/wiki/St._Petersburg_paradox#Finite_... and then https://en.wikipedia.org/wiki/Gambler%27s_ruin https://en.wikipedia.org/wiki/Gambler%27s_ruin for example. Most people will not take an 99% risk of going bankrupt in a game that will consume all their resource reserves; expected value as a statistic does not meaningfully capture the risk. Positive expectation, losing strategy.
- joe_the_user 7y agoThat's a good argument against seemingly plausible but risky approaches. This particular situation, of someone just pulling "it could be true!" out of their arse, can also be solved by framing things as "the more utility you claim, the less likely it seems and disproportionately so". IE, if the chance of getting X from the scoundrel is less than 1/(X^2*C), even integral of all the offers together winds up very small.
- petters 7y agoYes, but it is still not trivial to formalize mathematically without running into trouble: https://www.lesswrong.com/posts/a5JAiTdytou3Jg749/pascal-s-mugging-tiny-probabilities-of-vast-utilities https://www.lesswrong.com/posts/a5JAiTdytou3Jg749/pascal-s-m...
- roenxi 7y agoIt is pretty simple to formalise - if there is a pool of people who routinely accept positive-expected-value gambles where there is a P = 99.999999999% chance of going broke then we expect everyone in that pool will be broke unless the size of the pool is comparable to 1/(1-P). Tighten up the bounds on 'comparable' a bit and that is formalised. The mistake is accepting uncritically that expected value is the best metric to optimise. Nobody ever proved that expected value is a strategically superior metric. In fact it would be quite hard to prove that since it is not true. It leaves people vulnerable to making very stupid decisions as illustrated in Pascals Mugging. Optimium strategy involves at a minimum considering your available opportunities and available resources. Opportunity alone is not enough.
- praptak 7y agoParadoxes such as this one point at some rarely discussed assumptions about the utility function. The major one is that it exists. Or exists as something more than a model which is only good for a range of payoffs.
- adamchalmers 7y agoThe point of paradoxes like this is to demonstrate that even in our incredibly simple toy models of agents, we still run into issues. These paradoxes help point out weaknesses of the models and act as new desiridata for better models.
- zenon 7y agoI hereby declare that I will expose everyone that gives in to Pascal's mugger to a negative utility so great compared to whatever the mugger promises that it is always best to keep the wallet. I cold be telling the truth. You're welcome.
- ivan_ah 7y agoIn case someone is not familiar with Pascal's wager, here is a table that shows the expected values: https://en.wikipedia.org/wiki/Pascal%27s_wager#Analysis_with_decision_theory https://en.wikipedia.org/wiki/Pascal%27s_wager#Analysis_with... (the appearance of infinity breaks the decision theory...)
- MaxBarraclough 7y agoPascal's Wager fails for the simpler reason that it ignores the possibility that believing in god could send you to hell. I see a similar problem here. Pascal should consider the possibility that the mugger will use his (Pascal's) silly action as the basis for punishment, in place of the promised reward. Slim probability, potentially very high cost. The fact that this risk has gone unstated doesn't mean it isn't there.
- adamchalmers 7y agoI agree. This problem is still useful though, because to analyse a lot of counterfactual muggers, you must be able to analyse one mugger!
- MaxBarraclough 7y agoI agree that both problems are worth analysing, even if they're both absurd. Same goes for the Hangman's Paradox, a personal favourite. [0] I gather that Pascal had never intended Pascal's Wager to be a watertight argument, it was intended more as a plaything for showing how unlikely it was anyone could make a watertight case for the existence of God. [0] https://en.wikipedia.org/wiki/Hangman%27s_Paradox https://en.wikipedia.org/wiki/Hangman%27s_Paradox
- skrebbel 7y agoWow, I always thought of Pascal's Mugging as a satirical illustration of how stupid it is to take enormous (or extremely small) numbers seriously. Turns out people like John Carmack (see other comments in this thread) aren't picking up on the satire. Or am I reading him wrong? I think he's smarter than me, so what am I missing? Given Bostrom's general love for mixing tiny probabilities with enormous outcomes, what's the point of this article? It seems delightfully self-critical. How can the conclusion be anything other than that we should not be taking the AI/singularity crowd too seriously, as doing so would be akin to voluntarily handig over a wallet to a mugger?
- Erlich_Bachman 7y agoSounds more like you are taking the argument too seriously and are trying to read more into it than what it is. It is just a philosophical story created to provide a certain line of reasoning, as certain possible structure of an argument. People are free to apply this argument however they want, it doesn't prove anything by itself, it doesn't say anything about the world, it's up to the user of it. It does not make any conclusions, it's just a story. Carmack used it to illustrate his own beliefs (which are therefore: AI is possible and the payout for the AI is exteremely high, even if probability for it during the next couple of years is low). Carmack did not mean that you should believe or not believe in AI or anything else based on this argument. He just used it to illustrate what he himself chose to do. He did not base it on this argument alone, he did not just hear the argument and suddenly decide "now because of that I have to work on AI", he based it on his experience and knowledge of the actual field. The mugging argument is just a cute way of quickly explaining it. > I think he's smarter than me, so what am I missing? Wisdom and context.
- YeGoblynQueenne 7y agoI don't know the background to this but if I understand correctly, it kind of pivots on the probability that the mugger is indeed an Operator from the Seventh Dimension, that Pascal places at 1 in a quadrillion. In that case, I have to wonder where this estimate comes from? I get that it's just an arbitrary number and that any number would do, as long as it wasn't zero, but that's exactly the point: why can't Pascal place the probability of his mugger being an Operator from the Seventh Dimension at zero? Is there any evidence at all to support the mugger's claim? Is there any evidence at all that there is such a thing as a "Seventh Dimension" for which the only thing we know is that its "Operators" have magickal, utility-maximising powers? And does the whole thing only work if we assume that the probability that there is such a place and such people is more than 0?
- Strilanc 7y agoIf you set the probability at zero, you won't be convinced when they actually are an operator from the seventh dimension. That is to say, you run into the opposite problem of being Pascal's Muggle [1]. 1: https://www.lesswrong.com/posts/Ap4KfkHyxjYPDiqh2/pascal-s-muggle-infinitesimal-priors-and-strong-evidence https://www.lesswrong.com/posts/Ap4KfkHyxjYPDiqh2/pascal-s-m... > A wind begins to blow about the alley, whipping the Mugger's loose clothes about him as they shift from ill-fitting shirt and jeans into robes of infinite blackness, within whose depths tiny galaxies and stranger things seem to twinkle. In the sky above, a gap edged by blue fire opens with a horrendous tearing sound - you can hear people on the nearby street yelling in sudden shock and terror, implying that they can see it too - and displays the image of the Mugger himself, wearing the same robes that now adorn his body, seated before a keyboard and a monitor. > [...] "Unfortunately, you haven't offered me enough evidence," you explain.
- lidHanteyk 7y agoIn no particular order: * Helping a googleplex of people immediately vs over a period of time are two different complexities of action. * Recall that hypotheses are selected from an ambient pool of possibilities. Then we might imagine that some hypotheses dominate others, so that regardless of how much evidence is offered, we always insist that the evidence supports a simpler alternative. To wit: "Well, if I'm not a Matrix Lord, then how do you explain my amazing powers?" asks the Mugger. "Street magic," you say. "Very impressive sleight of hand. Perhaps some smoke, mirrors, lasers, assistants." * A Matrix Lord asking $5 of a person on the street in order to commit miracles is inherently irrational. If they just wanted $5, or wanted to deprive the person of $5, or wanted to humiliate and embarrass the person, or force them to accept certain philosophical truths, then those all could be achieved via Matrix Lordery. Therefore the Lord in this story is being a pointless dick, and it's silly to expect rational arguments to be part of the conversation. To wit: "Just give yourself $5. Give yourself any reward you like, for helping people; it's not my place to set or fulfill the price of such powerful entities, is it?" you ask. "But...but don't you want the feeling of doing good?" asks the Mugger. "Not really, no," you reply. "I have investments and equity already, and those dollars already have ripples that affect people far beyond my direct control. I don't feel much of anything about those investments. And it would be irrational for me to value a $5 investment more than $5. Really, if you can do all of this good, then you should turn yourself into an exchange-traded fund, and let people buy your time to do good in the world," you muse. "But...but this offer is for you, and you alone," the Mugger insists. "Okay, but why me? Let's talk about the Self-Sampling Assumption!" you say. The Mugger groans.
- deleted 7y ago[deleted]
- EvanWard97 7y agoExpected value != expected utility. Just taking this seriously pretty much resolves all these problems. It makes sense to take high cost/reward, low probability events seriously if the expected utility works out. Examples include reducing existential risk substantially, even at cost to short-term utility (say, in the form of well-being) to increase the probability that we can eventually figure out how to optimally arrange matter and energy and trigger a utilitronium shockwave.