4 ms·

If "AI" tools are eventually going to have to be "free" (as in beer) to compete, I shudder to think of what companies like OpenAI will have to extract from user

If "AI" tools are eventually going to have to be "free" (as in beer) to compete, I shudder to think of what companies like OpenAI will have to extract from users to please investors...

Zambyte 2y ago
To combat those incentives, they should consider being non-profit and releasing their models as open.
- exe34 2y ago
  they could found a charity, with the aim of creating open AI models. they could call it OpenAI to draw attention that it's not a proprietary commercial technology.
  - mritchie712 2y ago
    yeah, they'd probably need a board to see over it tho, sounds tricky, but doable.
    - BoiledCabbage 2y ago
      Then they could have their largest benefactor first pledge $100 Million to fund them. Then after suddenly insist they are woefully falling behind google on developing the tech and insist the organization be handed over to his sole control to help them catch up. After being told he's wrong, he could then get upset and renege on his pledge after giving only 10% of his commitment - they'd be left scrambling running out of funds and realize that being dependent on donations at their scale isn't working and they need a way to earn a profit or partnership to self-fund. Oh and after pulling the pledged money, that other party could go buy a social media site and use it to whine about how they aren't a non-profit anymore... while knowing they're the cause of it.
      - fragmede 2y ago
        that sounds ridiculous. there's no way that could ever happen. what are you on‽
      - exe34 2y ago
        this seems like an unrealistic scenario, nobody is that dumb?
- Dig1t 2y ago
  That’s a great idea, they could even change the name of their company to reflect their new open source philosophy.
dehrmann 2y ago
I wonder how feasible it would be to do massively distributed training, a la Folding@home or SETI@home.
- techjamie 2y ago
  I would think that you'd need a model format that could be updated in different places at the same time and easily merged at various stages. Because right now, adding all that network latency only slows things down. I'm sure there's a method to permit it, but it doesn't seem like anyone's worked it out yet.
- fragmede 2y ago
  the problem is internode bandwidth and latency. supercomputers are super because they have insanely fast bandwidth between the nodes so the GPUs can talk to each other as if they were local.
- __sy__ 2y ago
  Sounds like a really interesting technical challenge. Couple of issues I can see: (1) most devices out there would probably be mobile, so no NVIDIA/CUDA for you; (2) even with binding to, say, Apple Silicon, you might still be memory limited, i.e. can you fit the entire net on a single mobile GPU; (3) network latency?
- Sohcahtoa82 2y ago
  My somewhat limited understanding of LLMs (and ML in general) is that you would end up using a LOT of bandwidth, not to mention that training would be insanely slow, since each layer of the neural network requires the results of the previous layer, and then needs to send its results back. The greatest limiter for training isn't raw computing power, but storage. If your model is 400B parameters, you need 800GB to store it, assuming fp16. Then you need another 800 GB for calculating gradients. Sharding all this out means transferring a lot of data. If your device only stores 1B parameters out of the 400B, that means having to download 2 GB of data, doing your share of the work, then uploading 2 GB of results. Even with gigabit internet, you'll spend an order of magnitude more time transferring data than actually processing it. At that point, it'd be faster to train on standard-specced PC that had to constantly page out most of the model.
minimaxir 2y ago
There are many ways to extract value using a commodity: in OpenAI's case (and a few other LLM providers), the current strategy appears to be lock-in with additional services.
tempodox 2y ago
I think enough people have already demonstrated that they essentially don't care. À la “facebook knows you're pregnant before you do” – and now so will “Open”“AI”.
- vineyardmike 2y ago
  Just an FYI that this story was target (the department store) not facebook.
vineyardmike 2y ago
For a while the answer will probably be that they have better economies around GPU utilization rates.

If "AI" tools are eventually going to have to be "free" (as in beer) to compete, I shudder to think of what companies like OpenAI will have to extract from users to please investors...

Zambyte 2y ago

To combat those incentives, they should consider being non-profit and releasing their models as open.

exe34 2y ago
they could found a charity, with the aim of creating open AI models. they could call it OpenAI to draw attention that it's not a proprietary commercial technology.
- mritchie712 2y ago
  yeah, they'd probably need a board to see over it tho, sounds tricky, but doable.
  - BoiledCabbage 2y ago
    Then they could have their largest benefactor first pledge $100 Million to fund them. Then after suddenly insist they are woefully falling behind google on developing the tech and insist the organization be handed over to his sole control to help them catch up. After being told he's wrong, he could then get upset and renege on his pledge after giving only 10% of his commitment - they'd be left scrambling running out of funds and realize that being dependent on donations at their scale isn't working and they need a way to earn a profit or partnership to self-fund. Oh and after pulling the pledged money, that other party could go buy a social media site and use it to whine about how they aren't a non-profit anymore... while knowing they're the cause of it.
    - fragmede 2y ago
      that sounds ridiculous. there's no way that could ever happen. what are you on‽
    - exe34 2y ago
      this seems like an unrealistic scenario, nobody is that dumb?
Dig1t 2y ago
That’s a great idea, they could even change the name of their company to reflect their new open source philosophy.

dehrmann 2y ago

I wonder how feasible it would be to do massively distributed training, a la Folding@home or SETI@home.

techjamie 2y ago
I would think that you'd need a model format that could be updated in different places at the same time and easily merged at various stages. Because right now, adding all that network latency only slows things down. I'm sure there's a method to permit it, but it doesn't seem like anyone's worked it out yet.
fragmede 2y ago
the problem is internode bandwidth and latency. supercomputers are super because they have insanely fast bandwidth between the nodes so the GPUs can talk to each other as if they were local.
__sy__ 2y ago
Sounds like a really interesting technical challenge. Couple of issues I can see: (1) most devices out there would probably be mobile, so no NVIDIA/CUDA for you; (2) even with binding to, say, Apple Silicon, you might still be memory limited, i.e. can you fit the entire net on a single mobile GPU; (3) network latency?
Sohcahtoa82 2y ago
My somewhat limited understanding of LLMs (and ML in general) is that you would end up using a LOT of bandwidth, not to mention that training would be insanely slow, since each layer of the neural network requires the results of the previous layer, and then needs to send its results back. The greatest limiter for training isn't raw computing power, but storage. If your model is 400B parameters, you need 800GB to store it, assuming fp16. Then you need another 800 GB for calculating gradients. Sharding all this out means transferring a lot of data. If your device only stores 1B parameters out of the 400B, that means having to download 2 GB of data, doing your share of the work, then uploading 2 GB of results. Even with gigabit internet, you'll spend an order of magnitude more time transferring data than actually processing it. At that point, it'd be faster to train on standard-specced PC that had to constantly page out most of the model.

minimaxir 2y ago

There are many ways to extract value using a commodity: in OpenAI's case (and a few other LLM providers), the current strategy appears to be lock-in with additional services.

tempodox 2y ago

I think enough people have already demonstrated that they essentially don't care. À la “facebook knows you're pregnant before you do” – and now so will “Open”“AI”.

vineyardmike 2y ago
Just an FYI that this story was target (the department store) not facebook.

vineyardmike 2y ago

For a while the answer will probably be that they have better economies around GPU utilization rates.