7 ms·

Not sure why the AI community has a weird obsession with being non-profit (or not). It's not like non-profits can't be evil, and it's not like OpenAI has done a

by dvt 4y ago

Not sure why the AI community has a weird obsession with being non-profit (or not). It's not like non-profits can't be evil, and it's not like OpenAI has done anything particularly egregious, either. What we do need is more weights being released in the public domain (hard to find even on Huggingface), easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts, and so on. My big gripe, and for obvious reasons, is that we need to step away from cloud-based inference, and it doesn't seem like anyone's working on that.

vikp 4y ago
If you want more open research and weights, you should be happy about this announcement. Incorporating as a nonprofit doesn't guarantee that an organization will act ethically, but it does make it more likely. Nonprofits have more restrictions around how they can spend their money. They also pay less, so the teams is more likely to be mission-aligned. The profit motive pushes organizations to keep their work secret (which is what has happened with OpenAI).
fl0ps 4y ago
I'm with you there on hoping for more concentration on local training and inference, especially on something like the new Orin (open to donations, NVidia!). I think there's still something of a financial disincentive to promote selfhost over cloud capabilities for just about every party involved except the selfhoster. NVidia loses out if they're selling a few Nano or Orin platforms relative to the much pricier datacenter cards sold in huge lots. The cloud hosters are making up for the cost of hosting in terms of end-user pricing, snarfing all that lovely customer data and likely selling analytics, and probably other measures I can't understand yet. And the large companies that fund research and initial model development want to know what's being done with their models so as to gain any possible competitive advantage. They can't necesarily guarentee that intel from a self-hoster. Almost nobody is willing to spend the time explicitly to make it easier for the individual dev at a small lab or at home to do this because that's essentially a donation and not a business expense that might yeild obvious returns.
warent 4y ago
It seems like a rift between "Enlightenment" and "Authoritarian High Modernism." People want this progress to benefit reason and humanities, not contribute to our backsliding. Being nonprofit is at least symbolic of enlightenment.
bootsmann 4y ago
> easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts I think there are steps being taken in this direction (check out [1] and [2] for interesting lightweight transpile / ad-hoc training projects) but there is a lack of centralized community for these constrained problems. [1] https://github.com/emlearn/emlearn https://github.com/emlearn/emlearn [2] https://github.com/BayesWitnesses/m2cgen https://github.com/BayesWitnesses/m2cgen
josephjacks 4y ago
Sam doing a 180 on non-profit to for-profit was egregious, IMHO. Having said that, plenty of extremely profitable organizations masquerade (literally and figuratively) as non-profits. Also: https://twitter.com/elonmusk/status/1626516035863212034 https://twitter.com/elonmusk/status/1626516035863212034
- zitterbewegung 4y ago
  Sort of meta that Sam did something that Elon Musk would do…
- leesec 4y ago
  Capped profit*
  - nwoli 4y ago
    Capped at 100x investment which would be the most profitable company in history lol
- versteegen 4y ago
  It's disruptive technology and an enormous amount of value is likely to be created. I would much rather the profits go to a nonprofit which can distribute them charitably rather than them giving them away freely to the megacorps which actually have the resources to run these massive models (edit^2: at scale). I do think the 100x cap on return on Microsoft's $1B investment was way too high (we don't know what the cap is on later investments but it's meant to be reducing over time). Edit: Plus it's still the case that OpenAI aren't putting profit above all else. E.g. they just released the ChatGPT API and it's 10x cheaper per token than GPT3! And their Charter does count for something, it's binding on them.
  - ketzo 4y ago
    A nonprofit wouldn't have to distribute profits to charity. It just can't distribute profits to private or public shareholders (because it can't have them). A nonprofit is obligated to reinvest any earnings towards its mission. Not remotely an expert, this is just my basic understanding. "Nonprofit" is a pretty abused term.
braingenious 4y ago
> Not sure why the AI community has a weird obsession… Can you elaborate on this “obsession”? Aren’t a lot of AI things made/implemented by for-profit companies? Google, Meta, Midjourney, etc?
- __loam 4y ago
  Yeah lol. The obsession is understanding this is going to be pretty impactful technology and thinking it might be nice to not have that be closed source for once
chologrande 4y ago
I can't help but wonder if non-profit makes it easier to get access to or train on certain datasets/models.
Bloating 4y ago
> It's not like non-profits can't be evil There's no shortage of non-profits that exist to pay saleries. As an example, growing up in the bible-belt, and I can confirm that religion can be good business.
stellaathena 4y ago
You're in luck! EleutherAI has trained and released open source weights of several LLMs, including GPT-Neo (2.7B parameters), GPT-J (6B parameters), and GPT-NeoX (20B parameters). This last model is currently tied for second on the list of the largest open source LLMs in the world. We also developed VQGAN-CLIP and CLIP-Guided Diffusion, techniques for doing text-to-image synthesis that don't require training and can easily be run locally for inference.
- ImprobableTruth 4y ago
  I don't want to be too cynical, but OpenAI used to be more open too until they decided releasing weights was too dangerous (/not profitable enough?), what guarantee is there that Eleuther doesn't also close their doors at some point?
  - jerpint 4y ago
    Yes but in their wake they still left great contributions (CLIP, whisper, etc). So far they still seem net positive, even if they’re now for profit
    - bayindirh 4y ago
      I fail to see creating and releasing models as black boxes and strictly monetizing them while harvesting public data under the name of academic research as net positive. They suck in public data, create models, let them loose, let hyped people improve them and profit off of data and people's free work. IOW, a mechanical Turk which doesn't pay.
  - deleted 4y ago
    [deleted]
  - jamilton 4y ago
    There is no guarantee, but it sounds like if they didn't incorporate they wouldn't be doing much of anything, public or not.
  - 4y ago
xwdv 4y ago
> Not sure why the AI community has a weird obsession with being non-profit (or not) A lot of times, I find the obsession isn’t a desire for collaboration or anything noble, it’s as simple as this: people want to use the end products for free or very cheaply.
underlines 4y ago
I was looking for open alternatives to self hosted, or crowd hosted finetuned LLMs like ChatGPT and found LAION Open Assistant. Then found resources to further optimize inference as well as training: - Open source fine tuned assistants like LAION Open-Assistant [1] - inference optimizations like VoltaML, FlexGenm Distributed Inference [2] - training optimizations like Hivemind [2] 1 https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant 2 https://github.com/underlines/awesome-marketing-datascience/blob/master/awesome-ai.md#optimization https://github.com/underlines/awesome-marketing-datascience/...
dustingetz 4y ago
taxes!

by dvt 4y ago

vikp 4y ago

If you want more open research and weights, you should be happy about this announcement. Incorporating as a nonprofit doesn't guarantee that an organization will act ethically, but it does make it more likely. Nonprofits have more restrictions around how they can spend their money. They also pay less, so the teams is more likely to be mission-aligned. The profit motive pushes organizations to keep their work secret (which is what has happened with OpenAI).

fl0ps 4y ago

I'm with you there on hoping for more concentration on local training and inference, especially on something like the new Orin (open to donations, NVidia!). I think there's still something of a financial disincentive to promote selfhost over cloud capabilities for just about every party involved except the selfhoster. NVidia loses out if they're selling a few Nano or Orin platforms relative to the much pricier datacenter cards sold in huge lots. The cloud hosters are making up for the cost of hosting in terms of end-user pricing, snarfing all that lovely customer data and likely selling analytics, and probably other measures I can't understand yet. And the large companies that fund research and initial model development want to know what's being done with their models so as to gain any possible competitive advantage. They can't necesarily guarentee that intel from a self-hoster. Almost nobody is willing to spend the time explicitly to make it easier for the individual dev at a small lab or at home to do this because that's essentially a donation and not a business expense that might yeild obvious returns.

warent 4y ago

It seems like a rift between "Enlightenment" and "Authoritarian High Modernism." People want this progress to benefit reason and humanities, not contribute to our backsliding. Being nonprofit is at least symbolic of enlightenment.

bootsmann 4y ago

> easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts I think there are steps being taken in this direction (check out [1] and [2] for interesting lightweight transpile / ad-hoc training projects) but there is a lack of centralized community for these constrained problems. [1] https://github.com/emlearn/emlearn https://github.com/emlearn/emlearn [2] https://github.com/BayesWitnesses/m2cgen https://github.com/BayesWitnesses/m2cgen

josephjacks 4y ago

Sam doing a 180 on non-profit to for-profit was egregious, IMHO. Having said that, plenty of extremely profitable organizations masquerade (literally and figuratively) as non-profits. Also: https://twitter.com/elonmusk/status/1626516035863212034 https://twitter.com/elonmusk/status/1626516035863212034

zitterbewegung 4y ago
Sort of meta that Sam did something that Elon Musk would do…
leesec 4y ago
Capped profit*
- nwoli 4y ago
  Capped at 100x investment which would be the most profitable company in history lol
versteegen 4y ago
It's disruptive technology and an enormous amount of value is likely to be created. I would much rather the profits go to a nonprofit which can distribute them charitably rather than them giving them away freely to the megacorps which actually have the resources to run these massive models (edit^2: at scale). I do think the 100x cap on return on Microsoft's $1B investment was way too high (we don't know what the cap is on later investments but it's meant to be reducing over time). Edit: Plus it's still the case that OpenAI aren't putting profit above all else. E.g. they just released the ChatGPT API and it's 10x cheaper per token than GPT3! And their Charter does count for something, it's binding on them.
- ketzo 4y ago
  A nonprofit wouldn't have to distribute profits to charity. It just can't distribute profits to private or public shareholders (because it can't have them). A nonprofit is obligated to reinvest any earnings towards its mission. Not remotely an expert, this is just my basic understanding. "Nonprofit" is a pretty abused term.

braingenious 4y ago

> Not sure why the AI community has a weird obsession… Can you elaborate on this “obsession”? Aren’t a lot of AI things made/implemented by for-profit companies? Google, Meta, Midjourney, etc?

__loam 4y ago
Yeah lol. The obsession is understanding this is going to be pretty impactful technology and thinking it might be nice to not have that be closed source for once

chologrande 4y ago

I can't help but wonder if non-profit makes it easier to get access to or train on certain datasets/models.

Bloating 4y ago

> It's not like non-profits can't be evil There's no shortage of non-profits that exist to pay saleries. As an example, growing up in the bible-belt, and I can confirm that religion can be good business.

stellaathena 4y ago

You're in luck! EleutherAI has trained and released open source weights of several LLMs, including GPT-Neo (2.7B parameters), GPT-J (6B parameters), and GPT-NeoX (20B parameters). This last model is currently tied for second on the list of the largest open source LLMs in the world. We also developed VQGAN-CLIP and CLIP-Guided Diffusion, techniques for doing text-to-image synthesis that don't require training and can easily be run locally for inference.

ImprobableTruth 4y ago
I don't want to be too cynical, but OpenAI used to be more open too until they decided releasing weights was too dangerous (/not profitable enough?), what guarantee is there that Eleuther doesn't also close their doors at some point?
- jerpint 4y ago
  Yes but in their wake they still left great contributions (CLIP, whisper, etc). So far they still seem net positive, even if they’re now for profit
  - bayindirh 4y ago
    I fail to see creating and releasing models as black boxes and strictly monetizing them while harvesting public data under the name of academic research as net positive. They suck in public data, create models, let them loose, let hyped people improve them and profit off of data and people's free work. IOW, a mechanical Turk which doesn't pay.
- deleted 4y ago
  [deleted]
- jamilton 4y ago
  There is no guarantee, but it sounds like if they didn't incorporate they wouldn't be doing much of anything, public or not.
- 4y ago

xwdv 4y ago

> Not sure why the AI community has a weird obsession with being non-profit (or not) A lot of times, I find the obsession isn’t a desire for collaboration or anything noble, it’s as simple as this: people want to use the end products for free or very cheaply.

underlines 4y ago

I was looking for open alternatives to self hosted, or crowd hosted finetuned LLMs like ChatGPT and found LAION Open Assistant. Then found resources to further optimize inference as well as training: - Open source fine tuned assistants like LAION Open-Assistant [1] - inference optimizations like VoltaML, FlexGenm Distributed Inference [2] - training optimizations like Hivemind [2] 1 https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant 2 https://github.com/underlines/awesome-marketing-datascience/blob/master/awesome-ai.md#optimization https://github.com/underlines/awesome-marketing-datascience/...

dustingetz 4y ago

taxes!