3 ms·

Either we are able to work with LLMs with sufficient reliability to engineer products, in which case, we can fix bugs like this (where "fix" is understood as "r

Either we are able to work with LLMs with sufficient reliability to engineer products, in which case, we can fix bugs like this (where "fix" is understood as "reduce to an acceptable rate") - or we can't work with them, and we can't engineer products, and it doesn't really matter one way or the other because they're doomed to be curiosities. I think you're thinking, "a better aligned LLM would 'freak out' more often," which is a weird mental model because what you're talking about is an alignment issue. Wikipedia defines alignment as "[aiming] to steer the AI system towards humans intended goals, preferences, or ethical principles". Ethical principles consumes a lot of oxygen, but setting them aside, this is an example where the AI is not functioning in a way that is compatible with our goals and preferences. [1] You can't actually escape alignment of the LLM by just accepting whatever it gives you without reinforcement, an unaligned LLM is gunnuh give you garbage (more or less by definition). It's not like alignment is something people slap onto the LLM, it's a property of the LLM that needs to be adjusted for it to perform according to spec. It's kind of like saying, "software was fine before performance was introduced." You can absolutely introduce bugs when optimizing for performance, but the software has certain performance characteristics whether you do so or not, and if you ignore performance you will still run into performance issues. [1] An example that helped this click with me is this racing boat AI, which found an exploit for infinite points without ever completing the race: https://en.m.wikipedia.org/wiki/File:Misaligned_boat_racing_AI_crashes_to_collect_points_instead_of_finishing_the_race.ogg https://en.m.wikipedia.org/wiki/File:Misaligned_boat_racing_... This is a clarifying example because there's no ethics for us to debate, it's just a malfunctioning piece of software in an uncontroversial way.

maxbond 3y ago
A clearer way to make my point might be, the LLM is always aligned to something. If you don't bring that in line with what you want, then it's aligned to noise. Kind of like if you hired someone to do secretarial work in the same way you might be processing documents with an LLM, if you don't give them instruction about how the work needs to be performed, you shouldn't have a high confidence they'll perform it correctly.

maxbond 3y ago

A clearer way to make my point might be, the LLM is always aligned to something. If you don't bring that in line with what you want, then it's aligned to noise. Kind of like if you hired someone to do secretarial work in the same way you might be processing documents with an LLM, if you don't give them instruction about how the work needs to be performed, you shouldn't have a high confidence they'll perform it correctly.