4 ms·

I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral c

I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral concerns. I think there's tremendous value in end user facing LLMs being trained against moral policies, but for internal or private usage, if these models are trained on essentially raw WWW sourced data, I would personally want raw output. I'm also finding it particularly interesting to see what ethical strategies OpenAI comes up with considering that if you train a model on the raw prejudices of humanity, you're getting at least one category of "garbage in" that requires a lot of processing to avoid getting "garbage out."

cwkoss 4y ago
isn't llama in the wild now?
- andrewmcwatters 4y ago
  I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken. It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training: > There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models. We also provide in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.
  - jfowief 4y ago
    "Moral training" Just as dystopian it sounds. Fixing current subjective moral norms into the machine.
    - Dylan16807 4y ago
      Do you think public schools are inherently dystopian? I don't think you're using the right critique here. Picking a common system of moral norms is a lot better than no moral norms.
      - Veen 4y ago
        I’m not confident the “moral norms” prevalent in SV and/or US academia are common, if by that you mean norms that are prevalent in the general populace.
        Dylan16807 4y ago
        I mean primary school, and I don't think that counts as academia.
      - himinlomax 4y ago
        Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something. With that kind of moral compass, I’m not sure I'd be missing its absence.
        vkou 4y ago
        > Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something. > With that kind of moral compass, I’m not sure I'd be missing its absence. Please note that most forms of media and social media have no problem with politicians making credible threats of violence against entire groups of people. Politicians are subject to a different set of rules, and enjoy a lot more protection than you and I.
        filoleg 4y ago
        The issue here is not with online platform services allowing politicians more leeway in terms of what they can get away with on their platform. The actual issue is Midjourney not allowing regular users generate certain type of material solely because it makes fun of a political figure. What you are talking about is entirely tangential to the issue the grandparent comment is talking about.
  - sebzim4500 4y ago
    If they tried to make LLaMA woke, they did a terrible job. If you prompt it right you can basically get it to write Mein Kampf.
    - inawarminister 4y ago
      Yes that's because LLaMA is unfiltered, which is very good for general usage.
      - sebzim4500 4y ago
        Yeah, I wasn't trying to criticise. Just point out that it doesn't appear to be filtered.
  - schroeding 4y ago
    It doesn't appear to be filtered in a significant way. While toying with the 30B model, it suddenly started to steer a chat about a math problem into quite a sexual direction, with very explicit language. It also happily hallucinated, when prompted, that climate change is a hoax, as the earth is actually cooling down rapidly, multiple degrees per year, with a new ice age approaching in the next years. :D
    - andrewmcwatters 4y ago
      Ah there it is. Raw uncooked humanity.
    - nickthegreek 4y ago
      The 7B model brought up rape on my 3rd try out on an innocuous prompt.
    - hrn_34 4y ago
      To me it appears to lack certain knowledge. For instance, common genres of erotic manga. Somehow, the only genres I successfully made it to say are "shoujo" and "yaoi".
ars 4y ago
> who wants unfettered ability to interact with a LLM without moral concerns. Is that bad? It's just a language model - it says things. Humans have been saying all sorts of terrible things for ages, and we are still here. I mean it makes for nice headline "model said something racist", but does it actually change anything? These aren't decision making AI's (which would need to be much more careful), they are language models.
- aaomidi 4y ago
  Humans don’t scale infinitely. Humans have agency.
  - ars 4y ago
    And if the LLM scales infinitely it still does nothing unless a human reads and acts on it. And as you said: Humans don't scale, and have agency. Just writing something bad doesn't actually mean something bad happened. These days it seems like people are oversensitive to how things are said to them, and what things are said to them.
    - aaomidi 4y ago
      Wow. You just found the solution to propaganda! People just shouldn’t be sensitive!
      - chii 4y ago
        You jest, but the idea of critical thinking and rationalism is what you'd use to combat propaganda.
- rzmmm 4y ago
  The content it produces is quite hard to distinct from text written by real person. Not long ago, Facebook was accused of "playing a critical role" in Rohingya genocide. I dont really know but I believe worst case LLM risks are in that same category.
nico 4y ago
> into the hands of a hacker Forget “hackers”, think government agencies. Which is probably already happening right now. Food for thought: What’s the intersection of people closely related to OpenAI and Palantir? Edit: related thread on another front page post - https://news.ycombinator.com/item?id=35201992 https://news.ycombinator.com/item?id=35201992
- deleted 4y ago
  [deleted]
version_five 4y ago
[flagged]
tarruda 4y ago
> if these models are trained on essentially raw WWW sourced data, I would personally want raw output. Llama is a very high-quality foundation LLM, you can already run it very easily using llama.cpp and will get the raw output you need. https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp There's already instructions on how anyone can fine-tune it to behave similarly to ChatGPT for as little as $100: https://crfm.stanford.edu/2023/03/13/alpaca.html https://crfm.stanford.edu/2023/03/13/alpaca.html
- b33j0r 4y ago
  “Easily” was a minor canard, or at least… it took me a couple of efforts over a couple of days to get the dependencies to play as nicely as “someone with a brand new M2 arm laptop.” If nothing else, I continue to be amazed and how uninteroperable certain technologies are. I had to remove glibc and gcc to get llama to compile on my intel macbook. Masking/hiding them from my environment didn’t work, as it went out and found them and their header files instead of clang. Which eventually worked fine.
  - b33j0r 4y ago
    Well. I guess I’ll never understand why someone would dislike a comment about an experience different from theirs In a forum like this. i’m confused why someone would hate my report of how I had to solve a problem in my circustances. I hope to learn someday.
    - tarruda 4y ago
      I did not dislike your comment. The reason I considered easy was because I have very little knowledge in this area, in fact this is the first time I ever ran a machine learning model on my computer. I could not do it with the unmodified pytorch model (my GPU is not powerful enough to run even the 7B model), but I was surprised on how easy it was running with llama.cpp. I literally just followed the steps in the github page. But I was biased in saying it was easy, since I do have other knowledge (such as C development on Linux) which helped me.
SpaceManNabs 4y ago
Oh wow, I finally see a HN comment, with a positive opinion on more AI ethics, that is not grayed out.
- LoganDark 4y ago
  It's very easy to give into FUD when it comes to AI because of how much of a wildcard it is. I see no evidence of that in GP which is indeed pretty rare.

cwkoss 4y ago

isn't llama in the wild now?

andrewmcwatters 4y ago
I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken. It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training: > There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models. We also provide in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.
- jfowief 4y ago
  "Moral training" Just as dystopian it sounds. Fixing current subjective moral norms into the machine.
  - Dylan16807 4y ago
    Do you think public schools are inherently dystopian? I don't think you're using the right critique here. Picking a common system of moral norms is a lot better than no moral norms.
    - Veen 4y ago
      I’m not confident the “moral norms” prevalent in SV and/or US academia are common, if by that you mean norms that are prevalent in the general populace.
      - Dylan16807 4y ago
        I mean primary school, and I don't think that counts as academia.
    - himinlomax 4y ago
      Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something. With that kind of moral compass, I’m not sure I'd be missing its absence.
      - vkou 4y ago
        > Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something. > With that kind of moral compass, I’m not sure I'd be missing its absence. Please note that most forms of media and social media have no problem with politicians making credible threats of violence against entire groups of people. Politicians are subject to a different set of rules, and enjoy a lot more protection than you and I.
        filoleg 4y ago
        The issue here is not with online platform services allowing politicians more leeway in terms of what they can get away with on their platform. The actual issue is Midjourney not allowing regular users generate certain type of material solely because it makes fun of a political figure. What you are talking about is entirely tangential to the issue the grandparent comment is talking about.
- sebzim4500 4y ago
  If they tried to make LLaMA woke, they did a terrible job. If you prompt it right you can basically get it to write Mein Kampf.
  - inawarminister 4y ago
    Yes that's because LLaMA is unfiltered, which is very good for general usage.
    - sebzim4500 4y ago
      Yeah, I wasn't trying to criticise. Just point out that it doesn't appear to be filtered.
- schroeding 4y ago
  It doesn't appear to be filtered in a significant way. While toying with the 30B model, it suddenly started to steer a chat about a math problem into quite a sexual direction, with very explicit language. It also happily hallucinated, when prompted, that climate change is a hoax, as the earth is actually cooling down rapidly, multiple degrees per year, with a new ice age approaching in the next years. :D
  - andrewmcwatters 4y ago
    Ah there it is. Raw uncooked humanity.
  - nickthegreek 4y ago
    The 7B model brought up rape on my 3rd try out on an innocuous prompt.
  - hrn_34 4y ago
    To me it appears to lack certain knowledge. For instance, common genres of erotic manga. Somehow, the only genres I successfully made it to say are "shoujo" and "yaoi".

ars 4y ago

> who wants unfettered ability to interact with a LLM without moral concerns. Is that bad? It's just a language model - it says things. Humans have been saying all sorts of terrible things for ages, and we are still here. I mean it makes for nice headline "model said something racist", but does it actually change anything? These aren't decision making AI's (which would need to be much more careful), they are language models.

aaomidi 4y ago
Humans don’t scale infinitely. Humans have agency.
- ars 4y ago
  And if the LLM scales infinitely it still does nothing unless a human reads and acts on it. And as you said: Humans don't scale, and have agency. Just writing something bad doesn't actually mean something bad happened. These days it seems like people are oversensitive to how things are said to them, and what things are said to them.
  - aaomidi 4y ago
    Wow. You just found the solution to propaganda! People just shouldn’t be sensitive!
    - chii 4y ago
      You jest, but the idea of critical thinking and rationalism is what you'd use to combat propaganda.
rzmmm 4y ago
The content it produces is quite hard to distinct from text written by real person. Not long ago, Facebook was accused of "playing a critical role" in Rohingya genocide. I dont really know but I believe worst case LLM risks are in that same category.

nico 4y ago

> into the hands of a hacker Forget “hackers”, think government agencies. Which is probably already happening right now. Food for thought: What’s the intersection of people closely related to OpenAI and Palantir? Edit: related thread on another front page post - https://news.ycombinator.com/item?id=35201992 https://news.ycombinator.com/item?id=35201992

deleted 4y ago
[deleted]

version_five 4y ago

[flagged]

tarruda 4y ago

> if these models are trained on essentially raw WWW sourced data, I would personally want raw output. Llama is a very high-quality foundation LLM, you can already run it very easily using llama.cpp and will get the raw output you need. https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp There's already instructions on how anyone can fine-tune it to behave similarly to ChatGPT for as little as $100: https://crfm.stanford.edu/2023/03/13/alpaca.html https://crfm.stanford.edu/2023/03/13/alpaca.html

b33j0r 4y ago
“Easily” was a minor canard, or at least… it took me a couple of efforts over a couple of days to get the dependencies to play as nicely as “someone with a brand new M2 arm laptop.” If nothing else, I continue to be amazed and how uninteroperable certain technologies are. I had to remove glibc and gcc to get llama to compile on my intel macbook. Masking/hiding them from my environment didn’t work, as it went out and found them and their header files instead of clang. Which eventually worked fine.
- b33j0r 4y ago
  Well. I guess I’ll never understand why someone would dislike a comment about an experience different from theirs In a forum like this. i’m confused why someone would hate my report of how I had to solve a problem in my circustances. I hope to learn someday.
  - tarruda 4y ago
    I did not dislike your comment. The reason I considered easy was because I have very little knowledge in this area, in fact this is the first time I ever ran a machine learning model on my computer. I could not do it with the unmodified pytorch model (my GPU is not powerful enough to run even the 7B model), but I was surprised on how easy it was running with llama.cpp. I literally just followed the steps in the github page. But I was biased in saying it was easy, since I do have other knowledge (such as C development on Linux) which helped me.

SpaceManNabs 4y ago

Oh wow, I finally see a HN comment, with a positive opinion on more AI ethics, that is not grayed out.

LoganDark 4y ago
It's very easy to give into FUD when it comes to AI because of how much of a wildcard it is. I see no evidence of that in GP which is indeed pretty rare.