4 ms·

The premise is wrong. Humans don't listen to one another and nod unthinkingly. They criticize and validate relentlessly. Friends aren't mistaken for oracles. We

The premise is wrong. Humans don't listen to one another and nod unthinkingly. They criticize and validate relentlessly. Friends aren't mistaken for oracles. We have learned to not trust one another, except when one is speaking from deep experience and expertise in a given domain. AI is presented as an expert in every domain though, so we are lulled into a vulnerable state of unvigilance.

robotbikes 1y ago
I really found the story in chapter 14 (recursive self-improvement) about the guy who got so addicted to self-improvement that he ended up in his own meta-reality unable to understand even himself because he was getting so much better and hacking his learning. A completely fabricated story with no basis in reality that I'm aware of but man there are a lot of bullet points to make it seem factual. What are we going to do about the worrying trend of 10X hackers self-improving so much that they aren't able to exist in the real world. Here's an excerpt "The Addiction to Acceleration The fourth uncomfortable truth is how recursive improvement becomes compulsive. Kenji can’t stop because each day of not improving his improvement feels like stagnation. When you’re accelerating, constant velocity feels like moving backward. This addiction manifests as: • Inability to accept plateau phases • Anxiety when not optimizing optimization • Devaluing of steady-state excellence • Compulsion to add meta-levels • Fear of falling behind yourself Recursive improvement can become its own trap." I find that this criticism is far less applicable to say individuals but perhaps it could be levied against the way companies are currently treating AI. Which of course is where this comes from.
- robotbikes 1y ago
  Honestly there are some interesting concepts and broad overviews of them but this is hardly a "book" but just a verbose LLM document that briefly lists a lot of concepts without sufficiently or consistently fleshing them out into actual meaningful chapters. Not to say that this sort of thing isn't potentially useful but it seems more like the starting point of an outline of a book rather than anything resembling a finished published book.
tpmoney 1y ago
> Humans don't listen to one another and nod unthinkingly. They criticize and validate relentlessly. Friends aren't mistaken for oracles. We have learned to not trust one another, except when one is speaking from deep experience and expertise in a given domain. Do we though? I think a quick jaunt through any of the “-o-spheres” or “-tubes” of the internet would quickly disabuse someone of the idea that we default to not trusting other people. Even before the internet, “urban legends” are effectively “mass social hallucinations”.

robotbikes 1y ago

I really found the story in chapter 14 (recursive self-improvement) about the guy who got so addicted to self-improvement that he ended up in his own meta-reality unable to understand even himself because he was getting so much better and hacking his learning. A completely fabricated story with no basis in reality that I'm aware of but man there are a lot of bullet points to make it seem factual. What are we going to do about the worrying trend of 10X hackers self-improving so much that they aren't able to exist in the real world. Here's an excerpt "The Addiction to Acceleration The fourth uncomfortable truth is how recursive improvement becomes compulsive. Kenji can’t stop because each day of not improving his improvement feels like stagnation. When you’re accelerating, constant velocity feels like moving backward. This addiction manifests as: • Inability to accept plateau phases • Anxiety when not optimizing optimization • Devaluing of steady-state excellence • Compulsion to add meta-levels • Fear of falling behind yourself Recursive improvement can become its own trap." I find that this criticism is far less applicable to say individuals but perhaps it could be levied against the way companies are currently treating AI. Which of course is where this comes from.

robotbikes 1y ago
Honestly there are some interesting concepts and broad overviews of them but this is hardly a "book" but just a verbose LLM document that briefly lists a lot of concepts without sufficiently or consistently fleshing them out into actual meaningful chapters. Not to say that this sort of thing isn't potentially useful but it seems more like the starting point of an outline of a book rather than anything resembling a finished published book.

tpmoney 1y ago

> Humans don't listen to one another and nod unthinkingly. They criticize and validate relentlessly. Friends aren't mistaken for oracles. We have learned to not trust one another, except when one is speaking from deep experience and expertise in a given domain. Do we though? I think a quick jaunt through any of the “-o-spheres” or “-tubes” of the internet would quickly disabuse someone of the idea that we default to not trusting other people. Even before the internet, “urban legends” are effectively “mass social hallucinations”.