4 ms·

Wouldn't opensourcing the weights and trying to commoditize LLMs be a smarter move? That seems to be the only chance to catch up with OpenAI.

• nordsieck 4y ago
  > Wouldn't opensourcing the weights and trying to commoditize LLMs be a smarter move? That seems to be the only chance to catch up with OpenAI. Depends on what they're afraid of. If they want to commoditize the space, then I think you're right. If they think they can catch OpenAI, and they want to charge for AI services, then what they're doing makes sense.
  • capableweb 4y ago
    > If they think they can catch OpenAI, and they want to charge for AI services, then what they're doing makes sense. Then they wouldn't release anything to the researchers. Reason LLaMA weights are spreading in the first place is because they let a big group of people get access, someone is bound to upload it as a torrent. Seems they gave people with .edu emails access relatively quickly too, researcher or not.
    • nordsieck 4y ago
      > Then they wouldn't release anything to the researchers. I'm sure FB has shared enormous amounts of data with researchers in the past. And they've probably had very little trouble with said researchers uploading data to the internet. I think it's a pretty bad take that FB sharing data with researchers means that they wanted it spread freely. Even if that data is LLM weights.
      • flangola7 4y ago
        They were sharing it with everyone. I applied through their google form and put "zero" for previous research papers and NA for my organization. They still sent me a link
      • deleted 4y ago
        [deleted]
      • capableweb 4y ago
        My point obviously wasn't that every researcher leaks data all the time, nor that because Facebook shared data with researcher, they want it to spread freely. But I'm sure that Facebook when sharing data that is more precarious, like social connections, have been more selective with who they share the data with, compared to the LLaMA dataset which seems to be shared 100% with everyone with a .edu domain, and almost without review for others.
    • dTal 4y ago
      This halfway solution might even be the best of both worlds for Facebook - AI researchers are distracted by the shiny-shiny of gray-market weights, instead of working on truly open models. This might also explain why LLaMA wasn't trained to convergence - they're keeping the converged model for themselves.
      • flangola7 4y ago
        What do you mean converged?
        • terafo 4y ago
          It looks like LLaMa is undertrained, especially 65B(that's why it looses on a couple of important benchmarks to 30B).
• bottled_poe 4y ago
  Hmm.. explain the financial model behind this approach?
  • spdy 4y ago
    Fragmentation at first and you hinder the first mover in this case OpenAI in sizing the market completely and later on you can acquire the necessary pieces to get back into the market or buy time to close the gap. Current Microsoft strategy seems so push into this direction. Open Source certain technologies and acquire important pieces e.g. Github / Stake in OpenAI etc. to build a bigger picture that they can monetize later. OpenAI is a real threat to Google & Facebook
    • refulgentis 4y ago
      Facebook? Google, I understand. Facebook??
• chatmasta 4y ago
  That's what I thought they were doing. This DMCA approach seems like a mistake.