4 ms·

I agree; perhaps they are afraid of negative impacts of releasing their work to the public. Either way, I’m not sure I understand the name choice at this point.

• sva_ 4y ago
  Honestly, I think the main problem is, that training large models like this, or GPT-any, or anything like that costs a lot of compute power - which costs a lot of money. So if you're just investing a lot of compute but give everything away for free, that isn't a strategy which can be maintained. So what we really need, are ways to train models in a distributed fashion. Their initial goal was to 'democratize AI'. But democratizing these models doesn't just mean giving everyone access to your trained parameters, it also means that everyone should be able to spend an effort on improving it. I think a good solution for this would be huge.
  • emilsedgh 4y ago
    So when they formed OpenAI and thought "we are gonna hire the most brilliant scientists in the field" they forgot to account for hardware costs? At this point it's more like doublespeak.
    • moffkalast 4y ago
      They just announced they'll be rebranding as OpenYourWallet
      • texasbigdata 4y ago
        Do they have a revenue model? I thought it was a non-profit / public benefit company.
  • leereeves 4y ago
    > So what we really need, are ways to train models in a distributed fashion. How is the replication effort for this project handling that? They have over 600 people on their Discord right now, do they have some way for them to cooperate on training a single model?
    • sva_ 4y ago
      As far as I can see, No.
      • astrange 4y ago
        GPT-J and GPT-Neo are previous open source reproductions of GPT, but distributed training is overrated. There’s way too much communications overhead. I’m not sure any of the various @home projects have ever accomplished much for that matter. Instead they use donated time on a TPU or a GPU cloud provider. An exception is Leela Zero since you can realistically have it play chess at home.
        • airstrike 4y ago
          > Instead they use donated time on a TPU or a GPU cloud provider. So instead distribute the funding? And distributing running models too? Something something crowdfunding blockchain cryptocurrency Or alternatively https://www.microsoft.com/en-us/research/publication/decentralized-collaborative-ai-on-blockchain/ https://www.microsoft.com/en-us/research/publication/decentr...
        • kache_ 4y ago
          >donated time on a TPU or a GPU cloud provider hmmm... what about people who mine crypto on their hardware, and then that hardware goes into a pool of shared $ used to rent out a TPU
  • joshcryer 4y ago
    The training set is a one time computation though, running the model still costs cycles and people have to pay for it (see sites like Hugging Face or GooseAI). What people are asking about is the code itself and the one time computed training set. They're not asking to run the trained model for free. Anyway, getting cycles isn't too hard, expect CoreWeave or another compute-centric company to hop on board in due course.
    • moffkalast 4y ago
      > They're not asking to run the trained model for free. I am :D
  • gisely 4y ago
    The could have chosen to work on smaller models and used their already substantial amount of funding to be truly open. Instead they chose a path that demonstrates the Open part in OpenAI was just marketing.
• pshc 4y ago
  It's the same phenomena as when a protocol or data interchange format brands itself as "simple"
• joshcryer 4y ago
  That was their fear with GPT-3 because it would enable whole "fake news" articles to be written without much effort. At this point why should they release trained models? AI developers have a problem now, either show their work (locked down and limited to protect their image), or don't show it. If they do show it, it'll be reproduced. What's interesting to me is how we are approaching the free open source use of these systems. An open source DALL-E 2 could create some very abhorrent imagery.