5 ms·

it’s similar asking the gemini-1.5 models about coding questions that involve auth one of my questions about a login form also tripped a harassment flag

michaelt 2y ago
I suspect the refusal to answer questions about auth aren't a matter of hacking or offensive material. I suspect instead the people training these models have identified areas of questioning where their model is 99% right, but because the 1% wrong is incredibly costly they dodge the entire question. Would you want your LLM to give out any legal advice, or medical advice, or can-I-eat-this-mushroom advice, if you knew due to imperfections in your training process, it sometimes recommended people put glue in their pizza sauce?
- TeMPOraL 2y ago
  "If you can't take a little bloody nose, maybe you ought to go back home and crawl under your bed. It's not safe out here. It's wondrous, with treasures to satiate desires both subtle and gross... but it's not for the timid." So sure, the LLM occasionally pranks someone, in a way similar to how random Internet posts do; it is confidently wrong, in a way similar to how most text on the Internet is confidently wrong because content marketers don't give a damn about correctness, that's not what the text is there for. As much as this state of things pains me, general population has mostly adapted. Meanwhile, people who would appreciate a model that's 99% right on things where the 1% is costly, rightfully continue to ignore Gemini and other models by companies too afraid to play in the field for real.
  - rockskon 2y ago
    AI is not like some random person posting on the Internet. A random person on the Internet often has surrounding context to help discern trustworthiness. A researcher can also query multiple sources to determine how much there is concensus about. You can't do that with LLMs. I cannot stress strongly enough that direct comparisons between LLMs and experts on the Internet are inappropriate.
    - TeMPOraL 2y ago
      > I cannot stress strongly enough that direct comparisons between LLMs and experts on the Internet are inappropriate. In this context, I very much agree. But I'd like to stress that "experts on the Internet" is not what 99% of the users read 99% of the time, because that's not what search engines surface by default. When you make e.g. food or law or health-related queries, what you get back isn't written by experts - it's written by content marketers. Never confuse the two. > A researcher can also query multiple sources to determine how much there is concensus about. > You can't do that with LLMs. A person like that will know LLMs hallucinate, and query multiple sources and/or their own knowledge, and/or even re-query the LLM several times. Such people are not in danger - but very much annoyed when perfectly reasonable queries get rejected on the grounds of "safety".
    - Y_Y 2y ago
      Why can't you estimate the trustworthiness of an LLM? I happen to think that you can, and that the above analogy was fine. You don't need to read someone's forum history to know you shouldn't to trust them on something high-stakes. Maybe instead of strongly stressing you should present a convincing argument.
      - rockskon 2y ago
        Because if I already knew the answer then I wouldn't be asking the LLM?
  - pjc50 2y ago
    The only underlying question here is "who is liable for the output of the LLM?" I just don't think the "nobody is" current solution is going to last in the current litigious environment.
    - TeMPOraL 2y ago
      Good point. Since LLM isn't a person, this leaves only the vendor and the user as liable parties. That's one less legal person than in regular search, where you have the user, the search engine vendor, and the author/publisher of the content involved in a harm scenario. What is the consensus on liability in case of regular web search? Your comment made me realize that I never thought much about it in 20+ years of using the Internet; I kind of always assumed it's all on the user.
      - pjc50 2y ago
        > What is the consensus on liability in case of regular web search? Your comment made me realize that I never thought much about it in 20+ years of using the Internet Have you never noticed those "google has removed some results to comply with the DMCA" notices?
        voxic11 2y ago
        But the reason we "needed" the DMCA is because they wouldn't have been liable under existing law, and the DMCA only covers copyright violations.
        realusername 2y ago
        The DMCA is the copyright industry's response to "nobody is liable for results" which was the statu quo before.
    - raxxorraxor 2y ago
      The person who prompts would be responsible. Everything else doesn't really make sense. This is usually the trivial solution for any form of tool we use.

michaelt 2y ago

I suspect the refusal to answer questions about auth aren't a matter of hacking or offensive material. I suspect instead the people training these models have identified areas of questioning where their model is 99% right, but because the 1% wrong is incredibly costly they dodge the entire question. Would you want your LLM to give out any legal advice, or medical advice, or can-I-eat-this-mushroom advice, if you knew due to imperfections in your training process, it sometimes recommended people put glue in their pizza sauce?

TeMPOraL 2y ago
"If you can't take a little bloody nose, maybe you ought to go back home and crawl under your bed. It's not safe out here. It's wondrous, with treasures to satiate desires both subtle and gross... but it's not for the timid." So sure, the LLM occasionally pranks someone, in a way similar to how random Internet posts do; it is confidently wrong, in a way similar to how most text on the Internet is confidently wrong because content marketers don't give a damn about correctness, that's not what the text is there for. As much as this state of things pains me, general population has mostly adapted. Meanwhile, people who would appreciate a model that's 99% right on things where the 1% is costly, rightfully continue to ignore Gemini and other models by companies too afraid to play in the field for real.
- rockskon 2y ago
  AI is not like some random person posting on the Internet. A random person on the Internet often has surrounding context to help discern trustworthiness. A researcher can also query multiple sources to determine how much there is concensus about. You can't do that with LLMs. I cannot stress strongly enough that direct comparisons between LLMs and experts on the Internet are inappropriate.
  - TeMPOraL 2y ago
    > I cannot stress strongly enough that direct comparisons between LLMs and experts on the Internet are inappropriate. In this context, I very much agree. But I'd like to stress that "experts on the Internet" is not what 99% of the users read 99% of the time, because that's not what search engines surface by default. When you make e.g. food or law or health-related queries, what you get back isn't written by experts - it's written by content marketers. Never confuse the two. > A researcher can also query multiple sources to determine how much there is concensus about. > You can't do that with LLMs. A person like that will know LLMs hallucinate, and query multiple sources and/or their own knowledge, and/or even re-query the LLM several times. Such people are not in danger - but very much annoyed when perfectly reasonable queries get rejected on the grounds of "safety".
  - Y_Y 2y ago
    Why can't you estimate the trustworthiness of an LLM? I happen to think that you can, and that the above analogy was fine. You don't need to read someone's forum history to know you shouldn't to trust them on something high-stakes. Maybe instead of strongly stressing you should present a convincing argument.
    - rockskon 2y ago
      Because if I already knew the answer then I wouldn't be asking the LLM?
- pjc50 2y ago
  The only underlying question here is "who is liable for the output of the LLM?" I just don't think the "nobody is" current solution is going to last in the current litigious environment.
  - TeMPOraL 2y ago
    Good point. Since LLM isn't a person, this leaves only the vendor and the user as liable parties. That's one less legal person than in regular search, where you have the user, the search engine vendor, and the author/publisher of the content involved in a harm scenario. What is the consensus on liability in case of regular web search? Your comment made me realize that I never thought much about it in 20+ years of using the Internet; I kind of always assumed it's all on the user.
    - pjc50 2y ago
      > What is the consensus on liability in case of regular web search? Your comment made me realize that I never thought much about it in 20+ years of using the Internet Have you never noticed those "google has removed some results to comply with the DMCA" notices?
      - voxic11 2y ago
        But the reason we "needed" the DMCA is because they wouldn't have been liable under existing law, and the DMCA only covers copyright violations.
      - realusername 2y ago
        The DMCA is the copyright industry's response to "nobody is liable for results" which was the statu quo before.
  - raxxorraxor 2y ago
    The person who prompts would be responsible. Everything else doesn't really make sense. This is usually the trivial solution for any form of tool we use.