5 ms·
Some previous predictions: In 2021 Paul Christiano wrote he would update from 30% to "50% chance of hard takeoff" if we saw an IMO gold by 2025. He thought th
by z7 1y ago
Some previous predictions:
In 2021 Paul Christiano wrote he would update from 30% to "50% chance of hard takeoff" if we saw an IMO gold by 2025.
He thought there was an 8% chance of this happening.
Eliezer Yudkowsky said "at least 16%".
Source:
https://www.lesswrong.com/posts/sWLLdG6DWJEy3CH7n/imo-challenge-bet-with-eliezer https://www.lesswrong.com/posts/sWLLdG6DWJEy3CH7n/imo-challe...
- exegeist 1y agoImpressive prediction, especially pre-ChatGPT. Compare to Gary Marcus 3 months ago: https://garymarcus.substack.com/p/reports-of-llms-mastering-math-have https://garymarcus.substack.com/p/reports-of-llms-mastering-... We may certainly hope Eliezer's other predictions don't prove so well-calibrated.
- rafaelero 1y agoGary Marcus is so systematically and overconfidently wrong that I wonder why we keep talking about this clown.
- qoez 1y agoPeople just give attention to people making surprising bold counter narrative predictions but don't give them any attention when they're wrong.
- keeda 1y agoPeople like him and Zitron do serve a useful purpose in balancing the hype from the other side, which, while justified to a great extent, is often a bit too overwhelming.
- Philpax 1y agoBeing wrong in the other direction doesn't mean you've found a great balance, it just means you've found a new way to be wrong.
- dcre 1y agoI do think Gary Marcus says a lot of wrong stuff about LLMs but I don’t see anything too egregious in that post. He’s just describing the results they got a few months ago.
- m3kw9 1y agoHe definitely cannot use the original arguments from then ChatGPT arrived, he's a perennial goal post shifter.
- causal 1y agoThese numbers feel kind of meaningless without any work showing how he got to 16%
- shuckles 1y agoMy understanding is that Eliezer more or less thinks it's over for humans.
- 0xDEAFBEAD 1y agoHe hasn't given up though: https://xcancel.com/ESYudkowsky/status/1922710969785917691#m https://xcancel.com/ESYudkowsky/status/1922710969785917691#m
- sigmoid10 1y agoWhile I usually enjoy seeing these discussions, I think they are really pushing the usefulness of bayesian statistics. If one dude says the chance for an outcome is 8% and another says it's 16% and the outcome does occur, they were both pretty wrong, even though it might seem like the one who guessed a few % higher might have had a better belief system. Now if one of them had said 90% while the other said 8% or 16%, then we should pay close attention to what they are saying.
- grillitoazul 1y agoFrom a mathematical point of view there are two factors: (1) Initial prior capability of prediction from the human agents and (2) Acceleration in the predicted event. Now we examine the result under such a model and conclude that: The more prior predictive power of human agents imply the more a posterior acceleration of progress in LLMs (math capability). Here we are supposing that the increase in training data is not the main explanatory factor. This example is the gem of a general framework for assessing acceleration in LLM progress, and I think its application to many data points could give us valuable information.
- grillitoazul 1y agoAnother take at a sound interpretation: (1) Bad prior prediction capability of humans imply that result does not provide any information (2) Good prior prediction capability of humans imply that there is acceleration in math capabilities of LLMs.
- zeroonetwothree 1y agoA 16% or even 8% event happening is quite common so really it tells us nothing and doesn’t mean either one was pretty wrong.
- davidclark 1y agoThe correctness of 8%, 16%, and 90% are all equally unknown since we only have one timeline, no?
- andrepd 1y agoContext? Who are these people and what are these numbers and why shouldn't I assume they're pulled from thin air?
- Maxious 1y agoask chatgpt
- sailingparrot 1y ago> why shouldn't I assume they're pulled from thin air? You definitely should assume they are. They are rationalists, the modus operandi is to pull stuff out of thin air and slap a single digit precision percentage prediction in front to make it seems grounded in science and well thought out.
- c1ccccc1 1y agoYou should basically assume they are pulled from thin air. (Or more precisely, from the brain and world model of the people making the prediction.) The point of giving such estimates is mostly an exercise in getting better at understanding the world, and a way to keep yourself honest by making predictions in advance. If someone else consistently gives higher probabilities to events that ended up happening than you did, then that's an indication that there's space for you to improve your prediction ability. (The quantitative way to compare these things is to see who has lower log loss [1].) [1] https://en.wikipedia.org/wiki/Cross-entropy https://en.wikipedia.org/wiki/Cross-entropy
- lucianbr 1y agoIs there some database where you can see predictions of different people and the results? Or are we supposed to rely on them keeping track and keeping themselves honest? Because that is not something humans do generally, and I have no reason to trust any of these 'rationalists'. This sounds like a circular argument. You started explaining why them giving percentage predictions should make them more trustworthy, but when looking into the details, I seem to come back to 'just trust them'.
- 1y ago
- sailingparrot 1y agoOff topic, but am I the only one getting triggered every time I see a rationalist quantify their prediction of the future with single digit accuracy? It's like their magic way of trying to get everyone to forget that they reached their conclusion in completely hand-wavy way, just like every other human being. But instead of saying "low confidence" or "high confidence" like the rest of us normies, they will tell you they think there is 16.27% chance because they really really want you to be aware that they know bayes theorem.
- drexlspivey 1y agoYes
- jdmoreira 1y agoObviously you know nothing about a brier score. https://en.wikipedia.org/wiki/Brier_score https://en.wikipedia.org/wiki/Brier_score also: https://en.m.wikipedia.org/wiki/Superforecaster https://en.m.wikipedia.org/wiki/Superforecaster
- deleted 1y ago[deleted]
- danlitt 1y agoNo, you are right, this hyper-numericalism is just astrology for nerds.
- OldfieldFund 1y agoThe whole community is very questionable, at best. (AI 2027, etc.)
- mewpmewp2 1y agoIn military they estimate distances this way if they don't have proper tools. Each says a min max range and then where there's most overlap, that will be taken. It's a reasonable way to make quick intuition based decisions when no other way is available.
- empiricus 1y ago16% is just a way of saying one in six chances
- Xenoamorphous 1y agoOr just “twice as likely as the guy who said 8%”.
- UltraSane 1y agoThose percentages are completely meaningless. No better than astrology.
- Workaccount2 1y agoOne of the most worrying trends in AI has been how wrong the experts have been with overestimating timelines. On the other hand, I think human hubris naturally makes us dramatically overestimate how special brains are.