3 ms·
> "[To Lex Fridman:] We don't want our next model to be shockingly good" (Really ?) Yes, really. They're strongly influenced by Yudkowsky constantly telling e
by ben_w 3y ago
> "[To Lex Fridman:] We don't want our next model to be shockingly good" (Really ?)
Yes, really.
They're strongly influenced by Yudkowsky constantly telling everyone who will listen that we only get one chance to make a friendly ASI, and that we don't have the faintest idea what friendly even means yet.
While you may disagree with, perhaps even mock, Yudkowsky — FWIW, I am significantly more optimistic than Yudkowsky on several different axies of AI safety, so while his P(doom) is close to 1, mine is around 0.05 — this is consistent with their initial attempt to not release the GPT-2 weights at all, to experiment with RLHF in the first place, to red-team GPT-4 before release, asking for models at or above GPT-4 level to be restricted by law, and their "superalignment" project: https://openai.com/blog/introducing-superalignment https://openai.com/blog/introducing-superalignment
If OpenAI produces a model which "shocks" people with how good it is, that drives the exact race dynamics which they (and many of the people they respect) have repeatedly said would be bad.
- thorncorona 3y agoinfluence from yudkowsky is surprising, considering if you've ever touched hpmor, you'd realize the dude is a moron. re p(doom): the latest slatestarcodex[0] has a great little blurb about the difficulties of applying bayes to hard problems because there's too many different underlying priors which perturb the final number, so you end up fudging it until it matches your intuition. [0] https://www.astralcodexten.com/p/practically-a-book-review-rootclaim https://www.astralcodexten.com/p/practically-a-book-review-r...
- ben_w 3y agoI find it curious how many people severely downrate the intelligence of others: to even write a coherent text the length of HPMOR — regardless of how you feel about the plot points or if you regard it as derivative because of, well, being self-insert Marty Stu/Harry Potter fanfic[0] — requires one to be significantly higher than "moron", or even "normal", level. Edit: I can't quite put this into a coherent form, but this vibes with Gell-Mann amnesia, with the way LLMs are dismissed, and with how G W Bush was seen (in the UK at least). Ironically, a similar (though not identical) point about Bayes was made by one of the characters in HPMOR… [0] Also the whole bit of HPMOR in Azkaban grated on me for some reason, and I also think Yudkowsky re-designed Dementors due to wildly failing to understand how depression works; however I'm now getting wildly side-tracked… Oh, and in case you were wondering, my long-term habit of footnotes is somewhat inspired by a different fantasy author, one who put the "anthro" into "anthropomorphic personification of death". I miss Pratchett.
- thorncorona 3y agoIf you think that volume is a replacement for quality, you really need to read more. If you think volume is correlated with intelligence, you really need to read and write more. In case you still don't believe me, you are welcome to hop onto your favorite fan fiction site, such as AO3 [0], and search for stories over [large k] words. [0] https://archiveofourown.org/works/search?work_search%5Bquery%5D=&work_search%5Btitle%5D=&work_search%5Bcreators%5D=&work_search%5Brevised_at%5D=&work_search%5Bcomplete%5D=&work_search%5Bcrossover%5D=&work_search%5Bsingle_chapter%5D=0&work_search%5Bword_count%5D=%3E+500000&work_search%5Blanguage_id%5D=&work_search%5Bfandom_names%5D=&work_search%5Brating_ids%5D=&work_search%5Bcharacter_names%5D=&work_search%5Brelationship_names%5D=&work_search%5Bfreeform_names%5D=&work_search%5Bhits%5D=&work_search%5Bkudos_count%5D=&work_search%5Bcomments_count%5D=&work_search%5Bbookmarks_count%5D=&work_search%5Bsort_column%5D=_score&work_search%5Bsort_direction%5D=desc&commit=Search https://archiveofourown.org/works/search?work_search%5Bquery...
- ben_w 3y agoThat volume coherently. Obviously mere volume can be done by a literal monkey on a literal typewriter. Also, I didn't say "correlated with intelligence", what I said was more of a cut-off threshold — asserting that one cannot be an actual moron given writing coherently on that scale is more of a boolean than a correlation. I do need to write more (my own attempt at a novel has been stalled at "why can't I ever be satisfied with how I lead into the dramatic showdown!" for some years now, as none of my attempts have pleased me once written); but as for reading? Well, if you think my taste must result from insufficient breadth and depth, I must wonder what you think of Arthur C. Clarke, Francis Bacon, Larry Niven, Alexandre Dumas, Adrian Tchaikovsky, Neal Stephenson, Robert Heinlein, Alastair Reynolds, Isaac Asimov, Carl Jung, … I'm going to stop there rather than enumerate my whole library, but I can critique each in a different way without resorting to calling them playground insults, even the ones I dislike. But I will say it was interesting to contrast Chris Hadfield's "An Astronaut's Guide to Life on Earth" with Richard Wiseman's "Shoot for the Moon" — or H. G. Wells with Jules Verne.
- thorncorona 3y ago
- reducesuffering 3y agoIt's interesting that you're slinging around moron accusations at Yudkowsky seemingly unaware that slatestarcodex thinks very highly of his intelligence and Scott's blog is downwind of Yudkowsky.