I think the main problem is the compounding error rate. These LLM models are very impressive, however they still have an error rate albeit very low. When you ha

I think the main problem is the compounding error rate. These LLM models are very impressive, however they still have an error rate albeit very low. When you have an agent that is recursively using its own output, you are multiplying the error rate in each step. Which makes the LLM go off-topic or out of control. We humans have all kinds of mechanisms to incorporate different kinds of feedback into our thought processes. I don't want to say that this is impossible with LLMs, but for a LLM everything is text, and it must be difficult to distinguish. I think for self-driving the situation is different, because there are all kinds of different feedback loops controlling the car. But I do think it would be much better to focus on better cruise control experiences instead of self-driving cars, or maybe changing our environment to make self-driving cars more safe. There are already self-driving metro's after all.