I agree that LLMs are not a threat to humanity, since they are trying to output text and not actually change the world, and even giving them agency via plugins

I agree that LLMs are not a threat to humanity, since they are trying to output text and not actually change the world, and even giving them agency via plugins is probably not going to lead to ruin because there's no real reason to believe that an LLM will try to "escape the box" in any meaningful sense. It just predicts text. However, it's possible that in a few years we'll have models that are directly trying to influence the world, and possess the sort of intelligence that GPT has proven is possible. We should be very careful about proceeding in this space.