4 ms·
Context: the author is peripherally around the rationalist / lesswrong subculture (or at least read by them). A ~200-upvoted post in that community from yesterd
by sdrinf 4y ago
Context: the author is peripherally around the rationalist / lesswrong subculture (or at least read by them). A ~200-upvoted post in that community from yesterday basically starts with "we are in AGI endgame mode, with no solution to the alignment problem, everything is fucked, we are all going to die", reflecting on chatGPT, which is created by OpenAI, the organization the author is working for. That community has 1.6M visitors per month, and their members are about as much tech-savvy as HN.
That's the target audience for this piece, and I think he got a lot.
- panarky 4y agoDoes the should-not-exist camp say how they would eliminate rationality? Sufficiently powerful governments can sometimes eliminate physical objects in territory they control when the means to produce them are difficult to acquire, such as fighter jets. Simpler objects are harder to eliminate, such as rifles. And as objects merge with information, they're almost impossible to eliminate, as with 3D-printed rifles. Pure information is more difficult still, but powerful governments can and do censor history, criticism and ridicule they don't like, when it is distributed over a network they control, like telecom wires or government schools. But ML is more slippery than all of these because ML isn't just information, it's math. Math can be independently derived. Once enough people understand geometry, it's impossible to eliminate the knowledge that the sum of the angles of every triangle equals 180 degrees. If the government of one region purges the people and burns the books, the knowledge still exists in another region. If a world government burns all the books everywhere, people will derive it again, independently.