2 ms·

> We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 Not sure about that since it seems to being baked into a lot

> We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 Not sure about that since it seems to being baked into a lot of products at places like Microsoft. However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain.

skissane 4y ago
> However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain. There are a lot of people who'd love to have their own on-premise instance of ChatGPT (or equivalent), that they fully control, and could use for whatever purpose they want–even purposes that OpenAI might consider "harmful". They'd be happy to pay for that product if it were on offer. Not just private individuals, even businesses – sending customer data to OpenAI involves lots of regulatory/legal/contractual hurdles, an on-premise offering avoids all those. Also, once you get to a certain scale, owning your own hardware works out cheaper than cloud. If someone was to offer a ChatGPT-like service as an on-premise offering, I don't think they'd have any trouble finding people willing to pay for it. Even if they have to spend $X million to train a new model from scratch, I'm sure some VC would view it as a worthwhile investment. Of course, a free open source model would be even better, but a paid/commercial/proprietary on-premise model would remove many of the disadvantages of OpenAI. I'm 100% sure it is coming soon, I bet there are multiple teams working on it even as I type this.
- UncleOxidant 4y ago
  Sure, but that's still going to be a commercial product you'll have to pay for. Right now you can run LLaMA (and it's rapidly multiplying fine-tuned descendants) for free. The risk for these startups you describe as working on this as you type is the same thing happening to them that happened to Meta when they released their LLaMA params: they started getting copied all over the place. And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable.
  - skissane 4y ago
    > And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable. There is a legal argument that they aren’t in the US, but I don’t think that argument has been tested in court yet. Even if the courts uphold that argument-it is likely to fail in other countries, many of which have lower standards for copyrightability than the US does; and it is always possible Congress will respond by creating a new form of IP protection for them. That’s happened before - courts ruled that semiconductor masks weren’t copyrightable, so Congress invented a new “semiconductor mask right” to give copyright-equivalent protection to them. Given the amount of media focus on AI, if courts rule params can’t be copyrighted, very likely Congress invents “AI parameter rights”

skissane 4y ago

> However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain. There are a lot of people who'd love to have their own on-premise instance of ChatGPT (or equivalent), that they fully control, and could use for whatever purpose they want–even purposes that OpenAI might consider "harmful". They'd be happy to pay for that product if it were on offer. Not just private individuals, even businesses – sending customer data to OpenAI involves lots of regulatory/legal/contractual hurdles, an on-premise offering avoids all those. Also, once you get to a certain scale, owning your own hardware works out cheaper than cloud. If someone was to offer a ChatGPT-like service as an on-premise offering, I don't think they'd have any trouble finding people willing to pay for it. Even if they have to spend $X million to train a new model from scratch, I'm sure some VC would view it as a worthwhile investment. Of course, a free open source model would be even better, but a paid/commercial/proprietary on-premise model would remove many of the disadvantages of OpenAI. I'm 100% sure it is coming soon, I bet there are multiple teams working on it even as I type this.

UncleOxidant 4y ago
Sure, but that's still going to be a commercial product you'll have to pay for. Right now you can run LLaMA (and it's rapidly multiplying fine-tuned descendants) for free. The risk for these startups you describe as working on this as you type is the same thing happening to them that happened to Meta when they released their LLaMA params: they started getting copied all over the place. And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable.
- skissane 4y ago
  > And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable. There is a legal argument that they aren’t in the US, but I don’t think that argument has been tested in court yet. Even if the courts uphold that argument-it is likely to fail in other countries, many of which have lower standards for copyrightability than the US does; and it is always possible Congress will respond by creating a new form of IP protection for them. That’s happened before - courts ruled that semiconductor masks weren’t copyrightable, so Congress invented a new “semiconductor mask right” to give copyright-equivalent protection to them. Given the amount of media focus on AI, if courts rule params can’t be copyrighted, very likely Congress invents “AI parameter rights”