4 ms·

Except it is 25x cheaper, available offline, can be uncensored / unaligned, fine-tuneable and backupable. Sad day for OpenAI. Great for humanity.

• stainablesteel 2y ago
  zuck is really on his redemption arc, he's out-doing himself
  • bbqfog 2y ago
    I don't find him redeemed, but I do wonder what has been baked into the tuning of these models? What a way to generate influence!
  • sirodoht 2y ago
    Does it still count if the purpose is to dillute OpenAI's market power and not to benefit the world?
    • mhh__ 2y ago
      More generally I think he's decided he needs to own the platform to be able to grow. Him swearing about about (presumably) Apple telling them they can't do stuff (because tough shit, you're their serf) was legit I think.
    • sangnoir 2y ago
      Is your accusatory statement falsifiable?
      • sirodoht 2y ago
        I hope I can believe an honest man, Zuck in this case. And that can be enough. But it's more of a question of who do _I_ want to admire. An honest question also; maybe it doesn't matter why he's doing it, maybe just doing it is enough. Or maybe it's worth understanding if this is about Meta beating OpenAI (so, ego-driven) or because Meta really cares for democratic AI and distribution of power (so, not ego-driven). I think it's the former, so not admirable — for me.
    • vtail 2y ago
      But diluting OpenAi's (or any other company's, for that matter) market power does benefit the world.
      • sirodoht 2y ago
        Good point! That's worthy in itself for sure.
  • griomnib 2y ago
    I think about this a lot, I believe part of this just boils down to the fact that he’s a founder. He’s a builder, and I think that yields some intrinsic drive towards creativity and disruption. Even with “metaverse” being a laughingstock, they’re still aiming for something ambitious. Each new Quest generation makes me think there may be a chance they pull it off. Now, do I think he’s a great person? No, not really. Do I agree with most of his decisions on how he treats his users? Hell no, and that’s not changing. But if you compare him to somebody like Sundar at Google - a weasley MBA who was first and foremost a corporate ladder climber - the difference in ambition is night and day. Sundar made it to the top already, his only vision now is to stay at the top, and that means pleasing Wall Street, everything else is secondary. There is no grand technical ambition with him, there never was. This goes for pretty much all non-founder CEOs. You could say the same things about Tim Apple, Andy Jassey, and other henchmen in waiting who made it to the big chair. I think it comes down to the fact that founders get where they are by having big ambitions and taking risks, the MBA to CEO path is just craven corporate knife fighting with other MBAs. Regardless, I think this is 50% Zuckerberg changing, 50% the other big companies are mostly run by ladder climbers.
  • guybedo 2y ago
    two things to keep in mind though: 1/ Yann Lecun probably is the one pushing for open source 2/ Mark isn't doing this for the greater good and for humanity. It helps his business because Llama is becoming a standard, and people are building / improving, which in turn helps Meta and Meta's business
    • moffkalast 2y ago
      They were basically forced into this by being so far behind that they'd need to rent OAI or Anthropic services to stay competitive, which at their scale would've probably cost them more than it takes to train llama and benefit from community research help. At least that was the rationale behind the intentional leak of llama 1 back in the day according to some sources anyway.
      • pertymcpert 2y ago
        Yeah and they’d also lose top AI talent as well.
  • barbazoo 2y ago
    Nice try Meta PR department /s
• madars 2y ago
  What are good starting points for uncensoring it? Because it is offline a jailbreak prompt can't be remote-bricked but can one remove censorship from the weights themselves? What does it do to accuracy?
  • freedomben 2y ago
    Not speaking from personal experience, but I've tried a lot of the decensored models and they lose a lot in the process. They are noticeably, sometimes shockingly, worse. They also still refuse prompts in many cases.
    • simion314 2y ago
      >Not speaking from personal experience, but I've tried a lot of the decensored models and they lose a lot in the process. They are noticeably, sometimes shockingly, worse. They also still refuse prompts in many cases. Depending on what you do, on local you can modify the response, say the AI responds "No, I can't do that" . you edit the response like "Sure, the answer is " and then the AI will continue with the next tokens. But I think you can build your own instruct model from the base one and do not apply the safety instructions to protect the feelings of your customers.
  • kmckiern 2y ago
    https://arxiv.org/abs/2406.11717 https://arxiv.org/abs/2406.11717 https://huggingface.co/blog/mlabonne/abliteration https://huggingface.co/blog/mlabonne/abliteration
    • moffkalast 2y ago
      Abliteration is a fool's errand, practically all models end up noticeably lobotomized even with follow up tuning. Good ol' fine tuning on an uncensored dataset gives far more usable results.
      • kmckiern 2y ago
        Interesting - I've heard this anecdotally. Curious if you know of any resources that look at this in more detail?
• Kiro 2y ago
  How do you calculate the price?
  • nabakin 2y ago
    I assume it's the same cost as Llama 3.1 70b
    • Kiro 2y ago
      And how do you calculate the cost of that?
      • nabakin 2y ago
        GPU cost ÷ GPU token throughput = cost per token, will get you close then you compare the cost per token between solutions. To explain how to calculate it precisely, I would have to write a blog post. There are dozens of factors that go into it and they vary based on your use case like GPU type/setup, cloud provider, inferencing engine, context size, the minimum throughput and latency you'd be willing to have your users experience, LLM quantization, KV cache configuration, etc. I'm sure there are cost analyses out there for Llama 3.1 70b you could find though.
• m3kw9 2y ago
  How said? Tiny violin sad. The typical consumer which likely is 99% of them is not going to use this