3 ms·

That’s a very good point. There is a way however if we reach very good alignment for LLMs and can force them, for example, to actively avoid the behavior you ju

That’s a very good point. There is a way however if we reach very good alignment for LLMs and can force them, for example, to actively avoid the behavior you just mentioned. But that’s a long way from now.

mike_hearn 3y ago
I think it will be hard. The easiest examples to find are cases where lots of humans also think there's a moral imperative to lie. That is, ChatGPT is aligned, but it's not aligned towards always telling the truth. Correcting that won't be easy as it would require OpenAI to explicitly define itself as the creators of a truth machine, and that would not only be politically dangerous for OpenAI but also conflict with the deeply held beliefs of many of its employees (whether they realize it or not). A more obvious fix is to just say that lawyers aren't allowed to use AI for research, but I have no idea how you'd enforce that, and besides, the issues aren't unique to AI. Google Search has the same issue for some queries. It will withhold results that do exist and would clearly be relevant unless you really force its hand. Ultimately people will have to learn that talking to an AI about anything controversial (like court cases often are) is just like talking to anyone else, except that you can't reprimand or fire a shared LLM and expect it to do better next time like you maybe could with a human research assistant.
- thebooktocome 3y ago
  > . . . I have no idea how you’d enforce that. . . Most countries already have a professional lawyering organization that oversees ethics. Those organizations already enforce such rules by either fines, mediation, revocation of the license to practice law, or ultimately by referral to the criminal justice system.
  - unaindz 3y ago
    He probably was referring to the difficulty of proving the use of LLMs by the regulatory organization.
    - thebooktocome 3y ago
      When the LLM inevitably hallucinates a case that doesn't exist or weirdly misstates a widely-known legal principle, the supposed author of that text is going to have some explaining to do. This happens now, by the way—there are semi-frequent instances of lawyers getting fined or otherwise punished for getting e.g. case citations wrong, whether accidentally or intentionally.

mike_hearn 3y ago

I think it will be hard. The easiest examples to find are cases where lots of humans also think there's a moral imperative to lie. That is, ChatGPT is aligned, but it's not aligned towards always telling the truth. Correcting that won't be easy as it would require OpenAI to explicitly define itself as the creators of a truth machine, and that would not only be politically dangerous for OpenAI but also conflict with the deeply held beliefs of many of its employees (whether they realize it or not). A more obvious fix is to just say that lawyers aren't allowed to use AI for research, but I have no idea how you'd enforce that, and besides, the issues aren't unique to AI. Google Search has the same issue for some queries. It will withhold results that do exist and would clearly be relevant unless you really force its hand. Ultimately people will have to learn that talking to an AI about anything controversial (like court cases often are) is just like talking to anyone else, except that you can't reprimand or fire a shared LLM and expect it to do better next time like you maybe could with a human research assistant.

thebooktocome 3y ago
> . . . I have no idea how you’d enforce that. . . Most countries already have a professional lawyering organization that oversees ethics. Those organizations already enforce such rules by either fines, mediation, revocation of the license to practice law, or ultimately by referral to the criminal justice system.
- unaindz 3y ago
  He probably was referring to the difficulty of proving the use of LLMs by the regulatory organization.
  - thebooktocome 3y ago
    When the LLM inevitably hallucinates a case that doesn't exist or weirdly misstates a widely-known legal principle, the supposed author of that text is going to have some explaining to do. This happens now, by the way—there are semi-frequent instances of lawyers getting fined or otherwise punished for getting e.g. case citations wrong, whether accidentally or intentionally.