17 ms·
Nassim Talebs case against Nate Silver is bad math
- ikeboy 8y ago>The converse to the statement above is also true: if prices form a set of probabilities, then they are consistent. Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free. I don't believe this is correct. You can lose money to arbitrage even with a calibrated model if you ignore data someone else has, or include random noise someone else isn't including, which is what Taleb's argument is.
- abecedarius 8y agoSomeone betting Taleb-style would have an edge over someone directly betting on Silver's forecast numbers, but technically that's not arbitrage -- arbitrage in the sense of the article means a strategy with guaranteed return, not just very probable. Is that right? So my impression is that this is an argument about words, where traders mean 'arbitrage' more broadly.
- sincerely 8y agoOn the other hand, he managed to get Nate Silver so mad that he called Taleb a cuck. So who can tell who is actually right
- api 8y agoThey both look kind of bad from this TBH...
- baddox 8y agoEach person's claims are either right or wrong (assuming they are objective claims), regardless of any attribute or action of the person making the claim.
- r00fus 8y agoThere are situations where they can be both correct or wrong based on how you judge them.
- viburnum 8y agoThese two idiots are used to winning arguments by saying they know the most math. Now they're cancelling each other out.
- jerf 8y ago"Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free." I'll do the author the courtesy of assuming they've got a more expanded version of their argument in their head and just failed to express that, but I don't think that follows at all. I don't think there's a constraint on "Nate Silver has a probability model" that mathematically guarantees "Nate Silver's probability model" is correct, in any sense of that term. One could certainly provide evidence and/or proofs that Nate Silver's model follows those rules, but I don't think it just goes without saying. If nothing else the model could contain an straight-up error; goodness knows that's easy enough. (If one wishes to add to the concept "probability model" that it must be correct by definition, than I can't accept the statement "Nate Silver has a probability model" with mathematical levels of confidence. He claims to have one, yes, but that's a long way from a proof.)
- baddox 8y ago> I don't think there's a constraint on "Nate Silver has a probability model" that mathematically guarantees "Nate Silver's probability model" is correct If I'm understanding the article correctly, it's not talking about models being "correct," it's just talking about the probabilities of all the outcomes adding up to 100%, which means you can't place a bet that guarantees you make money regardless of the outcome. In other words, offering a bet that was based on the claim that there is a 99% chance of rain today in the Atacama Desert, and a 1% chance of no rain there, is arbitrage-free. It's still clearly a poor probability model for one of the driest places on Earth, but it's arbitrage-free.
- ska 8y agoYes, that's exactly what they mean.
- ikeboy 8y agoIf over time, your forecasts predictably change, then you can be arbitraged. If e.g. I know that you'll have a probability below 40% at some point in the future, and I know you'll have a number above 41% at some point in the future, then it's trivial to create a strategy over time that is guaranteed to make money. Taleb's claim, as I understand it, is that Silver is incorporating random noise into his model and so will exhibit such predictably swings, which enables arbitrage.
- swanson 8y agoRelated meta-analysis from the last go-around: https://towardsdatascience.com/why-you-should-care-about-the-nate-silver-vs-nassim-taleb-twitter-war-a581dce1f5fc https://towardsdatascience.com/why-you-should-care-about-the... (Implicitly endorsed by Taleb via retweet)
- kevinventullo 8y agoWhat's the probability that these guys are friends in real life who both mutually benefit from the popularity of their public feud?
- phpnode 8y agoPretty low.
- kolbe 8y agoI have a hard time believing Taleb can keep any friends. He thinks everyone is an idiot, and doesn't have any reservation about telling them.
- spitfire 8y agoHe's good friends with Stephen Wolfram. Take that any way you like.
- mwerty 8y agoA new kind of friendship.
- whitepoplar 7y agounderrated comment
- dralley 8y agoI know that this is probably exactly what you mean, but Wolfram doesn't have the best reputation for humility himself.
- ben509 8y agoMaybe not a lot of friends among his professional peers, but that wouldn't narrow his circle much.
- ar0 8y agoI am not so sure about this. When you read his books or look at his Twitter feed you surely get the impression he is a self-obsessed, extremely arrogant weirdo. But when he gives e.g. a talk in person, he comes across very differently - self-reflective, engaging, emphatic and funny. As someone else has mentioned in this thread, there are really two Talebs out there...
- KorematsuFred 8y agoI have zero respect for election forecasters because I think they are not any better than astrologers. Nate silver has used the word probability in a way that is not consistent with mathematical definition of probability so often in this context. Taleb despite being brilliant mathematician is actually pretty mediocre when it comes to real world. A lot of his "discoveries" such as "skin in the game" are well known to economists in the form of various theories. I once exchanged few tweets with him about his criticism of Pinker's claim that we are probably living in one of the most peaceful times in human history. At no point Pinker makes a claim it was inevitable or it being the evidence that peace will continue to exist. Taleb wrote a length paper on the topic which I found to be totally incoherent. When I asked questions his retort was that I do not understand statistics.
- brian_cloutier 8y agoDo you have an example of a time where Nate Silver has shown a misunderstanding of probability? Everything I've read by him has seemed informed.
- baddox 8y agoThe best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today. This seems to be one of Taleb's (and others') criticisms of their predictions: that they don't consider the probability of major unexpected events occurring between now and the event (like an athlete getting injured in the future, or the FBI announcing an investigation of a political candidate). But as I understand it, 538 is explicitly not attempting to consider those things, and is simply publishing probabilities of outcomes if the event were to happen now.
- ska 8y agoI think overall 538 did a pretty good job of communicating a difficult subject to a lay audience, but agree this point could perhaps be communicated better. I don't know of a good way to try and capture these sort of events meaningfully, do you?
- mathattack 8y agoTaleb in person at a talk is very different than Taleb with a bottle of wine on Twitter. To both of their credit, they’ve expanded my knowledge of probability and statistics.
- iron0013 8y agoHave they though, or have they only made you feel as though your knowledge of probability and statistics has expanded? Taleb is the poster boy for "sound and fury, signifying nothing"
- dmix 8y agoI don't know about Taleb, but Nate Silver's book on Bayesian statistics and prediction in general got me really interested in statistics from a general-reader standpoint. Where I ended up digging harder into the more math/sciencey books afterwards. The part about predicting Earthquakes and his work on sports is really good. https://www.amazon.com/Signal-Noise-Many-Predictions-Fail/dp/B009HL6444/ https://www.amazon.com/Signal-Noise-Many-Predictions-Fail/dp...
- iron0013 8y agoI agree about Silver, and that was kind of my point! Silver knows his shit and is able to communicate it in such a way that the reader actually learns something. Taleb may or may not know anything, but all he is able to convey to the reader is the impression that they've learned something (however, he is very good at giving the reader an unearned sense of intellectual superiority).
- dmix 8y agoSure but he sounds like a decent scientist to me. While it’s true his organization is very much an entertainment vector rather than a more academic science output, which lowers the bar, that doesn’t necessarily mean the work he’s doing doesn’t have some utility and value. Probably a lot more real value than a typical news outlet these days which pump out garbage at high speeds.
- andrewla 8y agoAs I understand it, the feud in its current form started from a tweet from Dinesh D'Souza [1] in which he interpreted Nate Silver's statement that "Dems, GOP winning House are 'both extremely possible'" to mean that they had gone from an 80% chance of winning to a 50% chance. Taleb inexplicably chose this as the hill to die on [2] and wrote a lengthy and technical paper to refute, and replied (on the [2] thread) "When someone says are event and its opposite are extremely possible I infer either 1) 50/50 or 2) the predictor is shamelessly hedging, in other words, BS." I've often found Taleb's ideas to be profound and interesting, and unfortunately offset by his arrogance and occasionally cryptic pronouncements and colorful colloquial "Brooklyn-ese" intentional misspellings of words (like "gabish" for "capisce"). But in this case I'm having trouble following what his criticism even is -- forecasts, especially probability forecasts, are hard, and evaluating them is hard -- the dimensions of skill vs. calibration vs. precision are hard to evaluate, and Nate Silver does a fairly good job of describing the tension. [1] https://twitter.com/DineshDSouza/status/1059147114826162177 https://twitter.com/DineshDSouza/status/1059147114826162177 [2] https://twitter.com/nntaleb/status/1059202026184282113 https://twitter.com/nntaleb/status/1059202026184282113
- theseatoms 8y agoTaleb's forte is applying various insights from practitioners of financial options to other domains.
- Alex3917 8y agoThat's not specifically a Brooklyn thing, it supposedly has more to do with the fact that most Italian families in the tri-state area came to the U.S. before Italy was a country with a standardized language. That's why e.g. in Connecticut they pronounce pizza as abeetz. For whatever reason knowing how to pronounce and spell Italian words "incorrectly" has become a point of local pride, even for people who aren't at all ethnically Italian.
- mhermher 8y agoIs this related to the characters in the Sopranos dropping the final vowels in a bunch of Italian words? Madonn, gabagool, prozhut, etc.
- kiernanmcgowan 8y agoWhilst the fighting between the two is childish, I've always found this interesting in a "statistics vs probability" perspective. They may look like similar fields of art, but are radically different in their approaches and how they seek to understand the world. It also doesn't help that probability theory takes digs at statistics with rules such as the "Law of the unconscious statistician"
- maxxxxx 8y agoI think the world would be a better place if nobody did any election forecasting. I don't see what positive value these forecasts provide.
- darkpuma 8y agoWhat positive value does American Football provide? It gives us brain damage and entertainment, not much unlike election forecasting.
- neuralk 8y agoTaleb's response to this article: https://twitter.com/nntaleb/status/1115684446081040386 https://twitter.com/nntaleb/status/1115684446081040386
- ska 8y agoHe remains a real charmer I see.
- tomp 8y agoBut is he wrong?
- ska 8y agoOrthogonal question. Unless you mean "Is he wrong in his approach?". Then probably yes.
- throwawaymath 8y agoIn regards to the very specific claim that the volatility of an option's price decreases asymptotically as the uncertainty of its underlying increases? No, that's entirely correct. In regards to Nate Silver's forecasts more generally? I don't know - it's hard to get past the tone of his arguments to understand what his actual disagreement is.
- aubreyclayton 8y agoHe is wrong in that the sentence of his that I quoted contained two statements: "when the volatility of the underlying security increases, [1] arbitrage pressures push the corresponding binary option to trade closer to 50% and [2] become less variable over the remaining time to expiration." His calculation proves [1], but it's [2] that is the basis of his criticism of Silver. And it's just not true, as can be seen even in a simple random-walk model.
- mike_ivanov 8y agoProve it.
- hwestiii 8y agoNassem Taleb is an arrogant asshole who can't write. I tried reading "The Black Swan" and gave up in a flurry of swearing after less than 100 pages because he utterly failed to justify any of his breathless assertions with anything resembling an argument. Assertion after assertion with no reasoning linking them. Game to Silver on points. At least he can back up his arguments with explanations.
- vtail 8y agoSo Nassim Taleb's "Silent Risk"[1], a mathematical parallel version to Incerto, is not good enough? [1] http://www.fooledbyrandomness.com/SilentRisk.pdf http://www.fooledbyrandomness.com/SilentRisk.pdf
- SilasX 8y agoHah. IIRC, the Audible audiobook for Antifragile bleeped out all the swearing.
- LanguageGamer 8y agoFrom a business perspective, the volatility of Silver's models seem like a feature - they enhance the drama / sensationalism of election coverage. One week, he's telling me candidate X will likely win, the next week it's candidate Y, and I'm on the edge of my seat. If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted.
- throwawaymath 8y ago> If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted. Yes, the "skin in the game" argument. That is Nassim Taleb's calling card. Personally, I consider it a little unreasonable to expect people to literally put their money where their mouth is every time they make a forecast.
- deleted 8y ago[deleted]
- dictum 8y agoIt's a kind of ad hoc censitary suffrage. One can only put possessions on the line if they have them, and the greater one's possessions, the greater their perceived "skin".
- throwawaymath 8y agoYeah I get the idea, I just don't like that it encourages dismissal of opinions which aren't backed by some form of collateral.
- chrismanfrank 8y agoYes, this is the source of the disagreement. Silver, I think, probably gets the math. But his aim is to entertain. Taleb's aim is something like predictive rigor. God knows why he cares about Silver. I think everyone gets that Silver is doing entertainment.
- jolesf 8y agoRuss Roberts invited them both to discuss their differences on econtalk- Nassim promptly declined. Until that happens, I think the twitter feud is without value.
- dalore 8y agoIt might look bad for both for them but also a good way to drum up some twitter followers. I followed them.
- mrfredward 8y agoIf I understand Taleb's argument, it can be boiled down to the following: A year before an election, there's 365 days worth of news stories, interviews, scandals, and other stuff that will be released to the public before election day. The day before the election, 364 days of that election influencing stuff has already happened and the effects of it are now known. As you get closer to an election, you have more information, and there is less time for public opinion to change, so it follows that you can be more certain of the election's outcome. Taleb's assumption that voter psychology can be modeled with a random walk is pretty questionable, but if you give him that assumption, what's wrong with his math? The article doesn't actually address this.
- ham_sandwich 8y agoIsn’t it just one big misunderstanding between them? Taleb thinks 538’s probabilities represent a binary option price on the event, in which case, yes, the probabilities should stay very close to 50% because the vol is so high. Whereas Silver’s models are actually saying “Based on current polls, if the election were held tomorrow, then the probability candidate X wins is Y%” and are thus allowed to swing more wildly. Aren’t both just fundamentally different things or am I missing something in Taleb’s argument?
- iainmerrick 8y agoHuh, yeah, that sounds very plausible. A forecast that really did try to be an option pricing model like that would be interesting to see. It would have the advantage over a “now cast” that you could actually run the numbers and see how accurate it is. Whereas nobody can ever know what would actually happen if there were an election now rather than a year from now.
- dragonwriter 8y ago> A forecast that really did try to be an option pricing model like that would be interesting to see. Silver’s election forecast models are exactly that. The nowcast expressly is not, but it's also not the headline model. > It would have the advantage over a “now cast” that you could actually run the numbers and see how accurate it is. That's true, and not just in theory: Silver has recently run the numbers for all of 538s forecasts (together and separated by sports vs. elections) and they are relatively accurate but not perfect; the supporting data is available for download, too. https://projects.fivethirtyeight.com/checking-our-work/ https://projects.fivethirtyeight.com/checking-our-work/
- iainmerrick 7y agoAha, thanks for clarifying that. I actually read that article, but the earlier comment got me confused over whether it was merely validating the final forecasts, or all forecasts over time. Makes sense that it was written as an explicit rebuttal to Taleb -- but without mentioning him directly. :)
- trpc 8y agoI really want to be Nassim Taleb, I don't want to get a real job, I don't want to be a slave for any kind of rules or laws whether in corporates, government or academia, I am much bigger than that, I just want to make enough money trading and shorting stocks during an economic bubble and retire young and then set myself emperor of the world and make fun of whoever I don't like such those academics who can't make 6 figure until in their 50s, of course academics are much less smarter than a stock trader, otherwise why couldn't they become rich like Taleb? right?!
- coldtea 8y agoWell, you're being sarcastic, but he does have a point in all that. The proof of the pudding being in the eating, and all that. There's something to be said for respecting somebody who betted on something he called and won (especially if they did it repeatedly), especially when most others were all against the very possibility of them being right... And being free with fuck you money does give you some freedom to say truths (even if they're just your personal truths) employees can't.
- trpc 8y agoeven though I am kinda sarcastic I extremely admire this kind of thinking (assuming that I got him right), many stock traders have this kind of free thinking mentality and disdain for all kinds of rules and authority, George Soros once said that the only reason people read about his philosophy is that he made a few billions, he knew from the very beginning that having enough money would make his ideas worth of being considered for smart people and absolutely right for retarded people. It's like money shifts the truth or something!
- thatoneuser 8y agoI think nate silver is bad math because when he can capitalize on drama he does. He paints himself as a fortune teller via math but has clear demonstrable bias. When you do that as a predictor then you've thrown out your integrity.
- aubreyclayton 8y agoSo... I wrote this. Maybe I can clarify a little what I meant by the statement "Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free.", since this seems to be confusing some people: What I'm addressing here is Taleb's claim that Silver's probabilities would allow for arbitrage (i.e., riskless profit, not just profit on average) if turned into betting prices. This is in the sense of arbitrage-through-time, buying low and then selling high. As I discussed in the piece, an old argument due to de Finetti says that if prices are arbitrage-free they must satisfy the equations of probability, meaning you can in some sense think of the price as giving a probability of the outcome. For time-dynamic arbitrage the relevant equation is Bayes' Theorem. All I meant by my statement above is that the converse to de Finetti's argument is also true, trivially. If prices obey the equations of probability they are automatically arbitrage-free. And since Silver's probabilities begin life as probabilities, they satisfy all the relevant equations (one would expect). Technically, the way we'd express this in modern finance is through the Fundamental Theorem of Asset Pricing, which says a (complete) market is arbitrage-free if and only if there exists an equivalent measure under which asset prices are martingales. Silver's probabilities are necessarily martingales, just because of the way conditional probability math works, so unless Taleb can claim that his and Silver's probabilities aren't equivalent, meaning they disagree on what events have probability zero, then there is no chance of arbitrage. That's just the mathy way of saying the same thing I said in the post. There are many other possible errors Silver could be making, and many other possible criticisms Taleb could have made but did not. In this case he wrote a paper claiming a mathematical result that just isn't true. Hope that's helpful!
- songeater 8y agoAside (but not really): guessing this is you? https://www.youtube.com/watch?v=P6P1rjJuD_M https://www.youtube.com/watch?v=P6P1rjJuD_M A totally awesome set of lectures... highly recommended.
- aubreyclayton 7y agoYep! Thanks!
- hnuser355 8y agoTaleb is smart etc but he’s basically a rambling eccentric who I ignore at this point
- vertline3 8y agoI stopped paying attention when Silver said the Buckeyes only had a 2 percent chance to win the title, because I knew that was bogus, team was stacked. Then he did the same thing with the Cavs when they turned around and won, so it just seemed to fit what Taleb says about "predictors" in The Black Swan. Since then, I don't listen much to forecasters, their track record is dubious.
- rossdavidh 8y agoI really like reading Nassim Taleb's books, but he appears to have all of the social skills of, say, Linus Torvalds, when interacting with others in a public way.
- acqq 8y agoSome old answers on Math Stack Exchange (Asked Sep 1 '16): https://math.stackexchange.com/questions/1911110/what-is-nassim-talebs-criticism-of-538s-election-model https://math.stackexchange.com/questions/1911110/what-is-nas...
- cryptidselfie 7y agoDropping in to say that Nassim is literally a genius. Him being rude doesn't change that - it just makes you a postmodernist.
- iron0013 7y ago"Postmodernist"? Are you just lobbing out random words that you think will be perceived as insults? Go gather your thoughts--if you are capable of doing so--and try again.
- cryptidselfie 7y agoPostmodernists think that being upset about someone's personality changes the very fabric of spacetime. "If he's mean, he must be wrong." That's hardly the case.
- iron0013 7y agoThat's not what "postmodernism" means. Care to try again?
- cryptidselfie 7y agoWell, that's not how I see it so I'm going to deconstruct your statement and come to the conclusion that you are merely a collection of your own biases, your case is ultimately racist, and your snarky tone means that you can't possibly be correct. “Postmodernity is said to be a culture of fragmentary sensations, eclectic nostalgia, disposable simulacra, and promiscuous superficiality, in which the traditionally valued qualities of depth, coherence, meaning, originality, and authenticity are evacuated or dissolved amid the random swirl of empty signals.” ― Jean Baudrillard An encyclopedia link for your reading pleasure: https://www.britannica.com/topic/postmodernism-philosophy https://www.britannica.com/topic/postmodernism-philosophy "There is an objective natural reality, a reality whose existence and properties are logically independent of human beings—of their minds, their societies, their social practices, or their investigative techniques. Postmodernists dismiss this idea as a kind of naive realism. Such reality as there is, according to postmodernists, is a conceptual construct, an artifact of scientific practice and language."
- ohboyherewego 7y agoThere are several problems with both of them. (1) Most of Nassim Taleb’s math is more branding and showing off than actual science or statistics. It has been stated by people who are far smarter than Taleb that “if you can’t state it simply then you don’t understand it.” If Richard Feynman can explain physics in a relatable way, then taleb can talk about statistics in the same. He doesn’t because explaining is not his objective, personal branding is. Taleb uses obscure, nonsensical math to show off and brow beat opponents. In my experience, working with people who are actually smart, this is not what very educated and intelligent people do. The general public may fall for pictures of pages of obscure nonsensical calculations depicting god knows what, all I see is fraudulent personal marketing. I say this having worked for and known CEOs who use the same tricks to lie to investors. It’s bullshit. (2) Nate Silver runs a media opinion network. I followed his work and early on they did very rigorous analysis and had great and accurate visualizations which did seem to accurate predict results. For whatever reason, I saw five thirty eight evolve into a bunch of media talking points and opinion mongering. The coverage on five thirty eight for the 2016 election resembled strongly that of standard news networks. Why? Because if you want to make money on advertising you need to completely fill the airwaves with as much content as possible and opinion content is the cheapest and easiest to produce. My opinion is that Nate slipped. His data analysis got distracted by running a media business, which focuses on advertising revenue and being likable by target demographics. Frankly I don’t trust either of them. I also side more strongly with Taleb - predicting outcomes with random hidden variables is an exercise in chicanery. No one wants to hear that the universe is random and that predicting it is impossible for long stretches of time. Look at the comments here, 90% of people (probably programmers) think that if they can just find the right equations then predicting the future is possible. Nope.
- nicholast 7y agoAcademic math: deriving precise solutions to predictive equations (eg @NateSilver538) Risk taker math: evaluating exposure to potential distributions (eg @nntaleb) With whom would you invest? Also, a useful heuristic to detect charlatans. Our knowledge of future never precise. more at https://twitter.com/_NicT_/status/1115818015004688386 https://twitter.com/_NicT_/status/1115818015004688386