4 ms·

There's something that feels tremendously off to me about these defenses which are to the effect of "What's so wrong with this? In principle, there's nothing sa

There's something that feels tremendously off to me about these defenses which are to the effect of "What's so wrong with this? In principle, there's nothing saying a human couldn't also make one of these errors." I think about the AI vending machine business simulation. Yes, LLMs were able to keep it running for a long time, but they were also prone to nosediving failure spirals due to simple errors a human would spot immediately. People point to an infinity in the distance where these LLMs cross some incredibly high threshold of accuracy that lets us thoughtlessly replace human workers and thinkers with them... I have not seen sufficient evidence to believe this.

disgruntledphd2 1y ago
> There's something that feels tremendously off to me about these defenses which are to the effect of "What's so wrong with this? In principle, there's nothing saying a human couldn't also make one of these errors." The big difference is that (most) humans will learn from this mistake. An LLM, however, is always a blank slate so will make the same mistakes over and over again. And more generally, because of how these models work, it would be very strange if they didn't make the same mistakes as humans.
SkyBelow 1y ago
Many humans not only could, but do and will keep doing so. The catch is that we have systems in place that keep these impacts from being harmful. Largely by preventing people without the ability to self reflect and improve their actions from being put in roles where they can cause disruption or harm. This is why we have some many different credentialing systems, from school diplomas to professional licenses to even something like good reviews of a business, to say nothing of our own personal filtering when we interact with someone before we do business with them. AIs are a horrible fit for the current system. They aren't really better or worse, but instead completely alien. From one angle they look like a good fit and end up passing initial checks, and even doing a somewhat decent job, but then a request comes in from another angle that results in them making mistakes that we would normally only associate with a complete beginner. The question becomes, can we modify the existing systems, or design new ones, where the alien AIs do fit in. I think so. Not enough to completely replace all humans, but 8 people with AI systems being able to do the job 10 people use to do still means 2 people were replaced, even if no one person's entire job was taken over entirely by the AI. What remains unknown is how far this can extend, between getting better AI and getting better systems where the AI can be slotted in. (And then to grandparent's point, what new roles in these systems become available for AI that weren't financially viable to turn into full jobs for humans.)

disgruntledphd2 1y ago

> There's something that feels tremendously off to me about these defenses which are to the effect of "What's so wrong with this? In principle, there's nothing saying a human couldn't also make one of these errors." The big difference is that (most) humans will learn from this mistake. An LLM, however, is always a blank slate so will make the same mistakes over and over again. And more generally, because of how these models work, it would be very strange if they didn't make the same mistakes as humans.

SkyBelow 1y ago

Many humans not only could, but do and will keep doing so. The catch is that we have systems in place that keep these impacts from being harmful. Largely by preventing people without the ability to self reflect and improve their actions from being put in roles where they can cause disruption or harm. This is why we have some many different credentialing systems, from school diplomas to professional licenses to even something like good reviews of a business, to say nothing of our own personal filtering when we interact with someone before we do business with them. AIs are a horrible fit for the current system. They aren't really better or worse, but instead completely alien. From one angle they look like a good fit and end up passing initial checks, and even doing a somewhat decent job, but then a request comes in from another angle that results in them making mistakes that we would normally only associate with a complete beginner. The question becomes, can we modify the existing systems, or design new ones, where the alien AIs do fit in. I think so. Not enough to completely replace all humans, but 8 people with AI systems being able to do the job 10 people use to do still means 2 people were replaced, even if no one person's entire job was taken over entirely by the AI. What remains unknown is how far this can extend, between getting better AI and getting better systems where the AI can be slotted in. (And then to grandparent's point, what new roles in these systems become available for AI that weren't financially viable to turn into full jobs for humans.)