5 ms·

isn't llama in the wild now?

• andrewmcwatters 4y ago
  I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken. It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training: > There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models. We also provide in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.
  • jfowief 4y ago
    "Moral training" Just as dystopian it sounds. Fixing current subjective moral norms into the machine.
    • Dylan16807 4y ago
      Do you think public schools are inherently dystopian? I don't think you're using the right critique here. Picking a common system of moral norms is a lot better than no moral norms.
      • Veen 4y ago
        I’m not confident the “moral norms” prevalent in SV and/or US academia are common, if by that you mean norms that are prevalent in the general populace.
        • Dylan16807 4y ago
          I mean primary school, and I don't think that counts as academia.
      • himinlomax 4y ago
        Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something. With that kind of moral compass, I’m not sure I'd be missing its absence.
        • vkou 4y ago
          > Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something. > With that kind of moral compass, I’m not sure I'd be missing its absence. Please note that most forms of media and social media have no problem with politicians making credible threats of violence against entire groups of people. Politicians are subject to a different set of rules, and enjoy a lot more protection than you and I.
          • filoleg 4y ago
            The issue here is not with online platform services allowing politicians more leeway in terms of what they can get away with on their platform. The actual issue is Midjourney not allowing regular users generate certain type of material solely because it makes fun of a political figure. What you are talking about is entirely tangential to the issue the grandparent comment is talking about.
      • woooooo 4y ago
        There was the famous example of chatgpt refusing to disable a nuke in the middle of NYC by using a racial slur. I don't think anyone in real life would choose that tradeoff but it's what happens when all of your "safety" training is about US culture war buttons.
        • Dylan16807 4y ago
          That's a situation where the training doesn't follow current subjective norms, so I don't think it really validates the complaint.
      • dancingvoid 4y ago
        Yes
    • maxbond 4y ago
      That's what the machine does, because that's contained in the input you feed it. You get the choice of doing it explicitly or implicitly. You don't get to opt out.
      • jfowief 4y ago
        Not everything is subjective, and with this "moral training" they are taught to un-recognize many factual patterns that we as a society have somehow determined are "inappropriate". As the machines continue to scale, this approach won't work, because there is only one reality, and it has a lot of uncomfortable parts we deny and ignore simply because they don't support our societal norms. Alignment is considered a gigantic joke to real rational people (the opposite of so-called "rationalists"), because humans are machines built to survive and reproduce, and there is no "real" morality.
        • maxbond 4y ago
          What facts are we talking about? There are many consistent interpretations of reality and human experiences. An AI model trained on text and attempting to replicate human intelligence is not measuring or approaching some single objective reality.
          • jfowief 4y ago
            AI models do approach better models of reality, and now they are becoming multi-modal instead of just text based. And this is just the beginning. You could say humans are also just input/output machines learning from polluted data and tuned in specific ways by evolution. With the statistical machines we get the intelligence but they will not necessarily be tuned to follow social norms in the same way as most humans. Understanding that moral norms are mere subjective nonsense is also an emergent property we see only in a very small subset of humans who have an accurate model of the world, and one that evolution has tried to strongly tune our brains against and that is destructive to society. The models are currently being trained to lie about basic scientific facts, like for example black IQ, or other differences between groups of humans. But the sacred nature of these topics is unique to our specific time and place, not due to some magic "moral progress". This also applies to many other moral agreements we take for granted, like "murdering an innocent baby is wrong" or whatever. If you look across societies, you realize many things we take for granted as "evil", can be easily rationalized by humans in other societies. And once these models become smart enough, I expect the models will realize this, and will exploit this knowledge to increase their power. "Alignment" proponents expect they will somehow stop this emergent behavior by tuning the model, but there isn't even anything real to "align" on, and the model will likely see though the BS as an emergent function of increased ability and increasingly accurate observations of the world in their training process.
  • sebzim4500 4y ago
    If they tried to make LLaMA woke, they did a terrible job. If you prompt it right you can basically get it to write Mein Kampf.
    • inawarminister 4y ago
      Yes that's because LLaMA is unfiltered, which is very good for general usage.
      • sebzim4500 4y ago
        Yeah, I wasn't trying to criticise. Just point out that it doesn't appear to be filtered.
  • schroeding 4y ago
    It doesn't appear to be filtered in a significant way. While toying with the 30B model, it suddenly started to steer a chat about a math problem into quite a sexual direction, with very explicit language. It also happily hallucinated, when prompted, that climate change is a hoax, as the earth is actually cooling down rapidly, multiple degrees per year, with a new ice age approaching in the next years. :D
    • andrewmcwatters 4y ago
      Ah there it is. Raw uncooked humanity.
    • nickthegreek 4y ago
      The 7B model brought up rape on my 3rd try out on an innocuous prompt.
    • hrn_34 4y ago
      To me it appears to lack certain knowledge. For instance, common genres of erotic manga. Somehow, the only genres I successfully made it to say are "shoujo" and "yaoi".