5 ms·
A) This is NOT Big Data B) This election was 101 in ML. The biases were obvious and easy to undo. C) I called it and won a lot of money https://news.ycombina
by huffpopo 10y ago
A) This is NOT Big Data
B) This election was 101 in ML. The biases were obvious and easy to undo.
C) I called it and won a lot of money
https://news.ycombinator.com/item?id=12863445 https://news.ycombinator.com/item?id=12863445
D) Best not to take advice from people who are constantly wrong
E) Big Data and ML are still awsome :)
See below from my linked post a week before the election:
The problem with Bayesian is that it's often over confident in the model. Error bars don't take this into account and people over rely on error bars - especially with complex domains.
Then there are the intentional and unintentional sampling biases in the polls that have to be taken into account. Then there are known human effects that add additional biases. E.g. the shy voter effect (e.g. Torres and Brexit), silent majorities (Nixon), momentum, dam breaking (Reagan), enthusiasm gap (Obama), the giant fk you to the establishment vote, pending indictments etc. In my view with the election being so emotional this year these effects are big enough to swing it to Trump making a poll based model wildly inaccurate.
I've taken an actual position in the election. $4k on Trump to win at 15% odds made just after the tapes. At the time I felt Trump had at least 50% chance so the bet was positive expected return for me. I figured the public would get over it. My Bayesian friends had his odds at 2%. Today I consider Trump to be at least 70% with the whole FBI inditement saga picking up steam.
In addition, the betting market behavior mirrors Brexit. A few very big bets on the status quo and many small bets against. It appears as if once again deep pocket punters are intentionality trying to manipulate the odds on the illiquid market in order to send a message that effects the vastly larger financial markets. So I think the odds are that there is some free money there.
- huffpopo 10y agoNOTE: The link only appears to work for my old account?? Was I shadow banned? Perhaps banning decenting opinions are a big reason why this election was such a supprise to everyone. https://news.ycombinator.com/threads?id=ttyltrful https://news.ycombinator.com/threads?id=ttyltrful
- matt4077 10y agoNot banned / save the conspiracies for another day / "dissenting" / "surprise"
- huffpopo 10y agoThen where is it? Why are my posts missing?
- grzm 10y agoIt does look like ttyltrful was banned. https://news.ycombinator.com/item?id=12684780 https://news.ycombinator.com/item?id=12684780 If you enable "showdead" in your account settings for huffpopo, you should be able to see them. As for banning dissenting opinions, watching the behavior of sctb and dang, it's pretty clear to me that they ban based on uncivil behavior, not content. Users on the other hand do sometimes flag or down vote to express disagreement.
- huffpopo 10y agoThanks. Makes sense, I was AB testing the community. I'll be quitting HN now. I won't bore anyone with details. All the best.
- nkurz 10y agoYes, your previous https://news.ycombinator.com/threads?id=ttyltrful https://news.ycombinator.com/threads?id=ttyltrful appears to be banned. Likely, it was after you ignored Scott's warning here: https://news.ycombinator.com/item?id=12684066 https://news.ycombinator.com/item?id=12684066. Statistically, it's probably a good decision to ban new accounts that start off with two snide comments. The particular comment you linked (https://news.ycombinator.com/item?id=12863445 https://news.ycombinator.com/item?id=12863445) seemed reasonable, so I 'vouched' for it. It doesn't look like that was enough to bring it back from the dead, though. You (or anyone else) can see it if you turn on the "Show Dead" option in your profile.
- huffpopo 10y ago
- matt4077 10y agoCongratulations on the money, but it doesn't really proof anything, especially not (B). There were hundreds of people working on this, considering all sorts of possible sources of errors. Any bias that is "obvious and easy to undo" was priced in. And the polls were actually not as bad – the difference was below the 3% from 2012. It just happened to swing the result.
- huffpopo 10y agoBy "obvious" I meant to ML practitioners (esp frequentists which is more typical for big data). Not everyone. So not priced in. It is not an efficient market. There are huge information asymmetries. I will give you that calling it ML 101 was over the top. If you dig into the polling breakdown and compare the individual polls (states as well as national) with the actuals it is clear that the polls were bad. Just because the aggregate error was less off than it otherwise could have been is more luck than anything. It could be argued that you can assume a gaussian error distribution so a more precise aggregate is to be expected but I would argue that that is not a safe assumption. TLDR; Just because it was close doesn't make it a good model. As an aside. All of my Bayesian friends lost money and all of my Frequents friends made money :)