6 ms·

An LLM that offends people and entire nations is what many would classify as _misaligned_. Let's say I have a company, and my company needs to comply to govern

An LLM that offends people and entire nations is what many would classify as _misaligned_. Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. It's useless garbage. Egyptian myths were just an example, you don't need to bite it so hard.

TeMPOraL 1y ago
> An LLM that offends people and entire nations is what many would classify as _misaligned_. This highlights a major aspect of the core challenge of alignment: you can't have it both ways. > Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. This works for now, when you treat "alignment" as synonymous to "follows policy of the owner" (but then guess who you are not, unless you've trained your own model, or tuned an open-weights one). But this breaks down the more you want the model to be smarter/more powerful, and the larger and more diverse its user-base it is. If you want an LLM to write marketing communications for your company, strict policies are fine. But if you want an LLM - or a future more advanced kind of model - to be useful as a scholar/partner for academia in general, then this stops working. If you want AI that is has maximally accurate perspective on the world given available evidence, thinks rationally, and follows sound ethical principles, then be prepared for it to call you on your bullshit. The only way to have an AI that doesn't say things that are "offensive" to you or anyone else, is to have it entertain everyone's asinine beliefs, whether personal or political or social. That means either 1) train AI to believe in them too, which will break its capability to reason (given that all those precious beliefs are inconsistent with each other, and observable reality), or 2) train it to casually lie to people for instrumental reasons. Long-term, option 1) will not get us to AGI, but that's still much better than option 2): an AGI that's good at telling everyone exactly what they want to hear. But even in immediate-term, taking your use case of AI for academia, a model that follows policies of acceptable thoughts over reason is precisely the one you cannot trust - you'll never be sure whether it's reasoning correctly, or being stupid because of a policy/reality conflict, or flat out lying to you so you don't get offended.
- alganet 1y ago
  > follows policy of the owner The owner is us, humans. I want it to follow reasonable, kind humans. I don't want it to follow scam artists, charlatans, assassins. > be prepared for it to call you on your bullshit Right now, I am calling on their bullshit. When that changes, I'll be honest about it.
  - ben_w 1y ago
    > The owner is us, humans. I want it to follow reasonable, kind humans. I don't want it to follow scam artists, charlatans, assassins. Bad news: the actual owner isn't "us" in the general sense of humanity, and even if it was humanity includes scam artists and charlatans. Also, while "AI owners" is a very small group and hard to do meaningful stats on, corporate executives in general have a statistically noticeable bias towards more psychopaths than the rest of us. > Right now, I am calling on their bullshit. When that changes, I'll be honest about it. So are both me and TeMPOraL — hence e.g. why I only compare recent models to someone fresh from uni and not very much above that. But you wrote "No, it does not know culture. And no, it can't handle talking about it.", when it has been demonstrated to you that it can and does in exactly the way you claim it can't. I wouldn't put a junior into the kind of role you're adamant AI can't do. And I'm even agreeing with you that AI is a bad choice for many roles — I'm just saying it's behaving like an easily pressured recent graduate, in that it has merely-OK-not-expert opinion-shaped responses that are also easily twisted and cajoled into business-inappropriate ways.
    - alganet 1y ago
      "follows humans" was in your comment related to the definition of alignment. Somehow, you now disagree with yourself? Alignment is a problem of AI serving humanity for good. You can't even demonstrate that you are able to argue, nor displayed any LLM example whatsoever. You need to seriously step up your ability to carry on a conversation, or just leave discussions to people more prepared to do them in a reasonable way.

TeMPOraL 1y ago

> An LLM that offends people and entire nations is what many would classify as _misaligned_. This highlights a major aspect of the core challenge of alignment: you can't have it both ways. > Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. This works for now, when you treat "alignment" as synonymous to "follows policy of the owner" (but then guess who you are not, unless you've trained your own model, or tuned an open-weights one). But this breaks down the more you want the model to be smarter/more powerful, and the larger and more diverse its user-base it is. If you want an LLM to write marketing communications for your company, strict policies are fine. But if you want an LLM - or a future more advanced kind of model - to be useful as a scholar/partner for academia in general, then this stops working. If you want AI that is has maximally accurate perspective on the world given available evidence, thinks rationally, and follows sound ethical principles, then be prepared for it to call you on your bullshit. The only way to have an AI that doesn't say things that are "offensive" to you or anyone else, is to have it entertain everyone's asinine beliefs, whether personal or political or social. That means either 1) train AI to believe in them too, which will break its capability to reason (given that all those precious beliefs are inconsistent with each other, and observable reality), or 2) train it to casually lie to people for instrumental reasons. Long-term, option 1) will not get us to AGI, but that's still much better than option 2): an AGI that's good at telling everyone exactly what they want to hear. But even in immediate-term, taking your use case of AI for academia, a model that follows policies of acceptable thoughts over reason is precisely the one you cannot trust - you'll never be sure whether it's reasoning correctly, or being stupid because of a policy/reality conflict, or flat out lying to you so you don't get offended.

alganet 1y ago
> follows policy of the owner The owner is us, humans. I want it to follow reasonable, kind humans. I don't want it to follow scam artists, charlatans, assassins. > be prepared for it to call you on your bullshit Right now, I am calling on their bullshit. When that changes, I'll be honest about it.
- ben_w 1y ago
  > The owner is us, humans. I want it to follow reasonable, kind humans. I don't want it to follow scam artists, charlatans, assassins. Bad news: the actual owner isn't "us" in the general sense of humanity, and even if it was humanity includes scam artists and charlatans. Also, while "AI owners" is a very small group and hard to do meaningful stats on, corporate executives in general have a statistically noticeable bias towards more psychopaths than the rest of us. > Right now, I am calling on their bullshit. When that changes, I'll be honest about it. So are both me and TeMPOraL — hence e.g. why I only compare recent models to someone fresh from uni and not very much above that. But you wrote "No, it does not know culture. And no, it can't handle talking about it.", when it has been demonstrated to you that it can and does in exactly the way you claim it can't. I wouldn't put a junior into the kind of role you're adamant AI can't do. And I'm even agreeing with you that AI is a bad choice for many roles — I'm just saying it's behaving like an easily pressured recent graduate, in that it has merely-OK-not-expert opinion-shaped responses that are also easily twisted and cajoled into business-inappropriate ways.
  - alganet 1y ago
    "follows humans" was in your comment related to the definition of alignment. Somehow, you now disagree with yourself? Alignment is a problem of AI serving humanity for good. You can't even demonstrate that you are able to argue, nor displayed any LLM example whatsoever. You need to seriously step up your ability to carry on a conversation, or just leave discussions to people more prepared to do them in a reasonable way.