8 ms·

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was firewalled and available only to large corporations and those with personal relations to big tech execs. Extrapolating from OpenAI's change of philosophy and business practices from their early days to now, it seems to be the way things are going. I only hope it doesn't go the way of that one paper which wanted to ban GPUs for sale to the public.

favaq 4y ago
>What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. Look at how Facebook closed down their APIs when Cambridge-Analytica occurred.
throwaway1851 4y ago
A concern I have about OpenAI is that, if you're using their APIs to develop an application, they can mine your data to compete with you, or even beat you to market. They can do this indirectly, by sharing information with preferred business partners. The conflict of interest, combined with the lack of robust data privacy guarantees, makes me queasy. If serving up generic LLM APIs becomes commoditized -- and I think it will -- they will want to monetize in other ways.
- asdff 4y ago
  Do you consent to that when you sign up for them? Its a microsoft product now and competitors to microsoft probably host their code on microsoft owned github without worry right now. Why start worrying now?
  - jonathankoren 4y ago
    Competitors to Microsoft buy the self hosting github option.
    - moi2388 4y ago
      Please name the competitors to Microsoft that use self hosted GitHub.
      - nicornk 4y ago
        SAP
      - yipbub 4y ago
        OS: linux,fedora, bsd Cloud: They're all closed-source. Doesn't seem black and white since they have their hand in so many pies, but name a real competitor to Microsoft that uses github.com?
        Jensson 4y ago
        Linux doesn't use github. They use git, not github. The question was if a competitor buys the self hosted github, not whether they use some other git solution.
      - deleted 4y ago
        [deleted]
- deleted 4y ago
  [deleted]
- gnicholas 4y ago
  Can you jump back and forth between competing AIs to prevent any of them from seeing the complete picture?
- precompute 4y ago
  This is the allure of AI, and this is also why OpenAI chose Micro$oft, the flame extinguisher par excellence. They have struck gold, they can now monopolize the very act of writing software, nevermind if it was based on a bait-and-switch and trained on code that wasn't legally open for usage in this manner. Pretty soon, this will lead to microsoft using their black box defense to make copycats of every service possible for their own windows platform, and then put it all around a paywall.
  - ChatGTP 4y ago
    This is 100% my concern too, no wonder it's good at coding when it it's spitting everything you make straight back at you. I'm not sure how to mitigate this yet? I'd say step one would be to get off GitHub, keep your innovative solutions evolving so they start to lose track of your work (if possible) and wait until open source alternatives are good enough to use.
- illiarian 4y ago
  I'd say: find a niche. Milk it for all it's worth. Be ready for access to be removed at any moment.
UncleOxidant 4y ago
> We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 Not sure about that since it seems to being baked into a lot of products at places like Microsoft. However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain.
- skissane 4y ago
  > However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain. There are a lot of people who'd love to have their own on-premise instance of ChatGPT (or equivalent), that they fully control, and could use for whatever purpose they want–even purposes that OpenAI might consider "harmful". They'd be happy to pay for that product if it were on offer. Not just private individuals, even businesses – sending customer data to OpenAI involves lots of regulatory/legal/contractual hurdles, an on-premise offering avoids all those. Also, once you get to a certain scale, owning your own hardware works out cheaper than cloud. If someone was to offer a ChatGPT-like service as an on-premise offering, I don't think they'd have any trouble finding people willing to pay for it. Even if they have to spend $X million to train a new model from scratch, I'm sure some VC would view it as a worthwhile investment. Of course, a free open source model would be even better, but a paid/commercial/proprietary on-premise model would remove many of the disadvantages of OpenAI. I'm 100% sure it is coming soon, I bet there are multiple teams working on it even as I type this.
  - UncleOxidant 4y ago
    Sure, but that's still going to be a commercial product you'll have to pay for. Right now you can run LLaMA (and it's rapidly multiplying fine-tuned descendants) for free. The risk for these startups you describe as working on this as you type is the same thing happening to them that happened to Meta when they released their LLaMA params: they started getting copied all over the place. And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable.
nutanc 4y ago
You guard your API, I guard my data.
snickerbockers 4y ago
>a lot of the talk about guarding models, public safety and misuse of models The stuff about models potentially being misused is just their public justification to look like the good guys. They're not going to withhold their technology because they don't want it to be misused, they're withholding it because they want control over who misuses it. Of course, it won't be called "misuse" when the right parties are doing it.
- pixl97 4y ago
  Eh, this is rather reductive to the point that the statement is meaningless. If you release a product in the wild with no safeties at all and then advertize "This product has no safeties at all", you'll likely find yourself in civil court on the losing side of the case. Now, if you put "some safeties" in the product, the person suing you is going to have a much more difficult and expensive time arguing that in front of the jury.
beambot 4y ago
> it was firewalled and available only to large corporations and those with personal relations to big tech execs. The previous call-out to IBM seems relevant: before PCs, this exact statement would've been true for (mini)computers and mainframes.
- skissane 4y ago
  > > it was firewalled and available only to large corporations and those with personal relations to big tech execs. > The previous call-out to IBM seems relevant: before PCs, this exact statement would've been true for (mini)computers and mainframes. Pre-PCs, IBM mainframes actually were quite open – up until the mid-1970s, IBM released its mainframe operating systems into the public domain. On the software side, the IBM S/360 was actually a lot more open than the IBM PC was – OS/360 was public domain with publicly available source code and even design documents (logic manuals), PC-DOS was copyrighted proprietary software whose source code and design documents were only publicly released decades after it had ceased to be commercially relevant. As we move through the 1970s, IBM became less and less open. The core OS remained in the public domain, but new features were increasingly only available in copyrighted add-ons – but IBM still shipped its customers source code, design documents, etc, for those add-ons. Finally, in 1983, IBM announced that the public domain core was being replaced by a new copyrighted version, for which it would withhold source code access from customers ("object code only", or "OCO" for short). The main way in which IBM mainframes in the 1950s-1970s were "firewalled" was simply by being fiendishly expensive – most people's houses cost significantly less. It is true that IBM did engage in anti-competitive business practices, but those were primarily non-technological in nature – contractual terms, pricing, etc – the kind of techniques which Thomas J. Watson Sr had mastered as an NCR sales executive in the lead-up to World War I. In fact, a big contributor to IBM becoming "less open" was the US Justice Department's 1969 anti-trust lawsuit, which led to IBM unbundling software and services from hardware–and its software culture became progressively more closed as software came to be seen as a product in its own right.
  - beambot 4y ago
    > The main way in which IBM mainframes in the 1950s-1970s were "firewalled" was simply by being fiendishly expensive – most people's houses cost significantly less. This was the primary aspect I was referring to, in the same way that training a ChatGPT-like NN can be (or could become) prohibitively expensive. But your comments about openness are relevant on an entirely different axis.
danbmil99 4y ago
Time to get serious about competitive open source models. Can't we do a seti at home sort of thing to distribute the training?
- renonce 4y ago
  Training is a bandwidth-intensive operation and requires huge (20Gbps+ stable and uninterrupted) bandwidth between all peers.
- avereveard 4y ago
  "GPT-3 175B model required 3.14E23 flops" according to their marketing material. Seti at home was about 1PetaFlops iirc so about 3 years training, possibly less if you can generate enough attention to the project that the people with the beefy devices will partecipate. The problem is that you need to train the full model you can't train aspect of it and even with each node doing independent tiny batches the network bandwidth for sinchronization would be massive. Seti was massively parallelizable because of the nature of the job oddnt require sinchronization between every peer.
  - dylan604 4y ago
    is there something to be said that seti@home was CPU only? would the GPU give a performance benefit that seti did not have? are people still using the GPUs to mine coins, or is that GPU compute at home available now?
    - lyu07282 4y ago
      The problem is not compute power the problem is weight and data synchronization. Each iteration or epoch builds on the previous, you either need to run the full model on each node with part of the data and you synchronize every epoch or you run part of the model but then you need to synchronize weights after each iteration. In proof of work mining you don't need to synchronize between each iteration, that's why in mining rigs the GPUs are connected to the CPU with only a few pcie lanes (2x instead of 16x or something) and they are unsuitable for machine learning. The clusters that big orgs are using to parallelize training use extremely expensive infiniband interconnections with tens of gigabytes per second and low latency (400 Gbps in the latest oai cluster) for that reason. Unfortunately not something you can democratize anytime soon.
alden5 4y ago
Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's getting close to the versatility of chatgpt where I could actually use it for everyday tasks. I don't think open source is that far away, especially considering how quickly Alpaca came out and how much better it is vs Lamma, which frequently would hallucinate and often didn't make sense. Example of prompt to Lamma-7b: > write a poem about open ai not being open source Open AI is not really “open” As this project isn’t open sore It can be seen as closed fortress, Inside which secrets are hidden. Not mind blowing but still really interesting, I will note that its much better at things like answering trivia where's there's already lots of examples in its model
- blueorange8 4y ago
  Were you able to integrate any of your data into it yet ?
  - alden5 4y ago
    I wouldn't be able to retrain the model as my computer isn't capable enough, but I can change the prompt to change how the model acts. The prompt i'm currently using is: "Below is an instruction that describes a task. Write a response that appropritely completes the request." That base prompt can be customized to complete specific tasks like classifying text or acting like an assistant.
    - omega3 4y ago
      You have a typo in the prompt: appropriately. I wonder if it makes any difference to the output.
    - b800h 4y ago
      Out of interest - how capable a computer is required to retrain that model?
      - alden5 4y ago
dclowd9901 4y ago
Conversely, I also had a brief moment of panic considering a bunch people somehow bumbling their way into making actual factual general AI and causing the end of civilization. I realize the cat’s out of the bag, but I feel like anything we can do to keep weaponized AI out of peoples hands as long as possible might not be the worst thing.
- atleastoptimal 4y ago
  That's true. The real question is whether or not the people in this new era calling the shots have the necessary capability and intentions to maximize the benefit these models will provide humanity and minimize the risk.
matheusmoreira 4y ago
We need this technology running locally on our computers as soon as possible.
- pixl97 4y ago
  At current computing growth rates that is still decades away. These things require exabytes of compute to train.
ChatGTP 4y ago
I only hope it doesn't go the way of that one paper which wanted to ban GPUs for sale to the public. I'd say this is a real possibility though? Not necessarily for or against it, but you can't see this happening, or at least serious discussion of it?
gwd 4y ago
I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people they work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. From that perspective, the opening of ChatGPT has actually been very effective at raising awareness. All the way back in GPT-1 they were trying to raise warnings, but those warnings didn't get much popular traction. Now that so many people have used ChatGPT (or Bing), I'm now having conversations about what computers "know" and "want" with my aunt on Facebook. Furthermore, if OpenAI has the best tools and sells them to everyone at a reasonable price, then there's a reduced incentive for other people to make their own tools. Whereas, if they were to close off access to the API, and only offer it to large corporations, there would be much more incentive for people to experiment with AI on their own -- and in doing so, possibly create an "un-aligned" super-intelligent AI which would destroy the human race. So my prediction is that given their motivations, they will 1) stop releasing details of their models to anyone other than research organizations they consider careful enough 2) continue to sell reasonably-priced access to the APIs, to reduce the risk that other people will step up to fill the demand who are less careful.
- anonymousDan 4y ago
  What a bunch of BS. The only reason they are keeping it private is for commercial gain .
  - UncleEntity 4y ago
    Just look at how much flak the stable diffusion folks got for the deep fake porn (and whatever else people were pearl grasping over) and tell me how a corporation will ever release a model. Meta was a fluke but they also did due diligence and made it look like they tried to do a responsible release — right up until someone put it on BitTorrent.
jillesvangurp 4y ago
You are assuming openai is going to end up with a monopoly on all this. IMHO the opposite is going to happen. There are going to be a multitude of companies and researchers competing on outdoing what they are doing in terms of quality, cost, and use cases. If big companies put a straight jacket in place to limit access, constrain usage, etc., that just creates the opportunity for others to step up and grab some market share. There are going to be use cases that are uncomfortable for big companies for ethical, political or other reasons. That's fine. That's their reality. But of course others will step into the void that creates with solutions of their own. And there is also the notion that big companies don't like being dependent on other big companies. OpenAI despite the name is very much not so open and really a Microsoft subsidiary in all but name. So, the likes of Amazon, Facebook, Google, and others are not going to be waiting for them to deliver new features and be creating their own strategies for competing. And that's just the big companies. The rest of the industry will do the same as soon as cost allows them to do that.
- DoingIsLearning 4y ago
  Indeed, I just saw a demo of Adobe Firefly, and the surprising thing to me is the whole thing was developed internally from data they control. Looking at Nvidia's rental solutions for Nvidia’s A100, it really feels like Future products will be driven by who is sitting on the biggest closed source training datasets more so than this specific success from OpenAI's research.
- malwrar 4y ago
  I personally expect that regulation or collusion among big tech players (e.g. the suppression of parlor) will prevent the average person or company from having the legal or practical ability to amass the compute power and dataset necessary to train a competing LLM (or future arch). No one really seems to know if OpenAI’s use of copyrighted materials like published works and open source code for training its LLM is legal. I can easily see a future where use of copyrighted works like this simply can’t be repeated legally, and the compute power necessary to do it as an individual is made inaccessible. This especially if the resulting model is made open to everyone, it’s such a wild cultural shift to me seeing tech nerds advocating against democratization of this tech due to personal doomsday fantasies. The sheer number of people who exist in the community and are obsessed with alignment and ethics will provide plenty of ideas on practical constraints the non-technical powerful could impose to make this real.
paganel 4y ago
> big company to pull public access of their APIs. This has been in effect since at least 10 years, I'd say. Twitter was the exception until relatively recently, but trying to build a product using the APIs of companies like Meta or Google became practically useless long ago.

favaq 4y ago

>What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. Look at how Facebook closed down their APIs when Cambridge-Analytica occurred.

throwaway1851 4y ago

A concern I have about OpenAI is that, if you're using their APIs to develop an application, they can mine your data to compete with you, or even beat you to market. They can do this indirectly, by sharing information with preferred business partners. The conflict of interest, combined with the lack of robust data privacy guarantees, makes me queasy. If serving up generic LLM APIs becomes commoditized -- and I think it will -- they will want to monetize in other ways.

asdff 4y ago
Do you consent to that when you sign up for them? Its a microsoft product now and competitors to microsoft probably host their code on microsoft owned github without worry right now. Why start worrying now?
- jonathankoren 4y ago
  Competitors to Microsoft buy the self hosting github option.
  - moi2388 4y ago
    Please name the competitors to Microsoft that use self hosted GitHub.
    - nicornk 4y ago
      SAP
    - yipbub 4y ago
      OS: linux,fedora, bsd Cloud: They're all closed-source. Doesn't seem black and white since they have their hand in so many pies, but name a real competitor to Microsoft that uses github.com?
      - Jensson 4y ago
        Linux doesn't use github. They use git, not github. The question was if a competitor buys the self hosted github, not whether they use some other git solution.
    - deleted 4y ago
      [deleted]
deleted 4y ago
[deleted]
gnicholas 4y ago
Can you jump back and forth between competing AIs to prevent any of them from seeing the complete picture?
precompute 4y ago
This is the allure of AI, and this is also why OpenAI chose Micro$oft, the flame extinguisher par excellence. They have struck gold, they can now monopolize the very act of writing software, nevermind if it was based on a bait-and-switch and trained on code that wasn't legally open for usage in this manner. Pretty soon, this will lead to microsoft using their black box defense to make copycats of every service possible for their own windows platform, and then put it all around a paywall.
- ChatGTP 4y ago
  This is 100% my concern too, no wonder it's good at coding when it it's spitting everything you make straight back at you. I'm not sure how to mitigate this yet? I'd say step one would be to get off GitHub, keep your innovative solutions evolving so they start to lose track of your work (if possible) and wait until open source alternatives are good enough to use.
illiarian 4y ago
I'd say: find a niche. Milk it for all it's worth. Be ready for access to be removed at any moment.

UncleOxidant 4y ago

> We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 Not sure about that since it seems to being baked into a lot of products at places like Microsoft. However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain.

skissane 4y ago
> However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain. There are a lot of people who'd love to have their own on-premise instance of ChatGPT (or equivalent), that they fully control, and could use for whatever purpose they want–even purposes that OpenAI might consider "harmful". They'd be happy to pay for that product if it were on offer. Not just private individuals, even businesses – sending customer data to OpenAI involves lots of regulatory/legal/contractual hurdles, an on-premise offering avoids all those. Also, once you get to a certain scale, owning your own hardware works out cheaper than cloud. If someone was to offer a ChatGPT-like service as an on-premise offering, I don't think they'd have any trouble finding people willing to pay for it. Even if they have to spend $X million to train a new model from scratch, I'm sure some VC would view it as a worthwhile investment. Of course, a free open source model would be even better, but a paid/commercial/proprietary on-premise model would remove many of the disadvantages of OpenAI. I'm 100% sure it is coming soon, I bet there are multiple teams working on it even as I type this.
- UncleOxidant 4y ago
  Sure, but that's still going to be a commercial product you'll have to pay for. Right now you can run LLaMA (and it's rapidly multiplying fine-tuned descendants) for free. The risk for these startups you describe as working on this as you type is the same thing happening to them that happened to Meta when they released their LLaMA params: they started getting copied all over the place. And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable.

nutanc 4y ago

You guard your API, I guard my data.

snickerbockers 4y ago

>a lot of the talk about guarding models, public safety and misuse of models The stuff about models potentially being misused is just their public justification to look like the good guys. They're not going to withhold their technology because they don't want it to be misused, they're withholding it because they want control over who misuses it. Of course, it won't be called "misuse" when the right parties are doing it.

pixl97 4y ago
Eh, this is rather reductive to the point that the statement is meaningless. If you release a product in the wild with no safeties at all and then advertize "This product has no safeties at all", you'll likely find yourself in civil court on the losing side of the case. Now, if you put "some safeties" in the product, the person suing you is going to have a much more difficult and expensive time arguing that in front of the jury.

beambot 4y ago

> it was firewalled and available only to large corporations and those with personal relations to big tech execs. The previous call-out to IBM seems relevant: before PCs, this exact statement would've been true for (mini)computers and mainframes.

skissane 4y ago
> > it was firewalled and available only to large corporations and those with personal relations to big tech execs. > The previous call-out to IBM seems relevant: before PCs, this exact statement would've been true for (mini)computers and mainframes. Pre-PCs, IBM mainframes actually were quite open – up until the mid-1970s, IBM released its mainframe operating systems into the public domain. On the software side, the IBM S/360 was actually a lot more open than the IBM PC was – OS/360 was public domain with publicly available source code and even design documents (logic manuals), PC-DOS was copyrighted proprietary software whose source code and design documents were only publicly released decades after it had ceased to be commercially relevant. As we move through the 1970s, IBM became less and less open. The core OS remained in the public domain, but new features were increasingly only available in copyrighted add-ons – but IBM still shipped its customers source code, design documents, etc, for those add-ons. Finally, in 1983, IBM announced that the public domain core was being replaced by a new copyrighted version, for which it would withhold source code access from customers ("object code only", or "OCO" for short). The main way in which IBM mainframes in the 1950s-1970s were "firewalled" was simply by being fiendishly expensive – most people's houses cost significantly less. It is true that IBM did engage in anti-competitive business practices, but those were primarily non-technological in nature – contractual terms, pricing, etc – the kind of techniques which Thomas J. Watson Sr had mastered as an NCR sales executive in the lead-up to World War I. In fact, a big contributor to IBM becoming "less open" was the US Justice Department's 1969 anti-trust lawsuit, which led to IBM unbundling software and services from hardware–and its software culture became progressively more closed as software came to be seen as a product in its own right.
- beambot 4y ago
  > The main way in which IBM mainframes in the 1950s-1970s were "firewalled" was simply by being fiendishly expensive – most people's houses cost significantly less. This was the primary aspect I was referring to, in the same way that training a ChatGPT-like NN can be (or could become) prohibitively expensive. But your comments about openness are relevant on an entirely different axis.

danbmil99 4y ago

Time to get serious about competitive open source models. Can't we do a seti at home sort of thing to distribute the training?

renonce 4y ago
Training is a bandwidth-intensive operation and requires huge (20Gbps+ stable and uninterrupted) bandwidth between all peers.
avereveard 4y ago
"GPT-3 175B model required 3.14E23 flops" according to their marketing material. Seti at home was about 1PetaFlops iirc so about 3 years training, possibly less if you can generate enough attention to the project that the people with the beefy devices will partecipate. The problem is that you need to train the full model you can't train aspect of it and even with each node doing independent tiny batches the network bandwidth for sinchronization would be massive. Seti was massively parallelizable because of the nature of the job oddnt require sinchronization between every peer.
- dylan604 4y ago
  is there something to be said that seti@home was CPU only? would the GPU give a performance benefit that seti did not have? are people still using the GPUs to mine coins, or is that GPU compute at home available now?
  - lyu07282 4y ago
    The problem is not compute power the problem is weight and data synchronization. Each iteration or epoch builds on the previous, you either need to run the full model on each node with part of the data and you synchronize every epoch or you run part of the model but then you need to synchronize weights after each iteration. In proof of work mining you don't need to synchronize between each iteration, that's why in mining rigs the GPUs are connected to the CPU with only a few pcie lanes (2x instead of 16x or something) and they are unsuitable for machine learning. The clusters that big orgs are using to parallelize training use extremely expensive infiniband interconnections with tens of gigabytes per second and low latency (400 Gbps in the latest oai cluster) for that reason. Unfortunately not something you can democratize anytime soon.

alden5 4y ago

Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's getting close to the versatility of chatgpt where I could actually use it for everyday tasks. I don't think open source is that far away, especially considering how quickly Alpaca came out and how much better it is vs Lamma, which frequently would hallucinate and often didn't make sense. Example of prompt to Lamma-7b: > write a poem about open ai not being open source Open AI is not really “open” As this project isn’t open sore It can be seen as closed fortress, Inside which secrets are hidden. Not mind blowing but still really interesting, I will note that its much better at things like answering trivia where's there's already lots of examples in its model

blueorange8 4y ago
Were you able to integrate any of your data into it yet ?
- alden5 4y ago
  I wouldn't be able to retrain the model as my computer isn't capable enough, but I can change the prompt to change how the model acts. The prompt i'm currently using is: "Below is an instruction that describes a task. Write a response that appropritely completes the request." That base prompt can be customized to complete specific tasks like classifying text or acting like an assistant.
  - omega3 4y ago
    You have a typo in the prompt: appropriately. I wonder if it makes any difference to the output.
  - b800h 4y ago
    Out of interest - how capable a computer is required to retrain that model?
    - alden5 4y ago

dclowd9901 4y ago

Conversely, I also had a brief moment of panic considering a bunch people somehow bumbling their way into making actual factual general AI and causing the end of civilization. I realize the cat’s out of the bag, but I feel like anything we can do to keep weaponized AI out of peoples hands as long as possible might not be the worst thing.

atleastoptimal 4y ago
That's true. The real question is whether or not the people in this new era calling the shots have the necessary capability and intentions to maximize the benefit these models will provide humanity and minimize the risk.

matheusmoreira 4y ago

We need this technology running locally on our computers as soon as possible.

pixl97 4y ago
At current computing growth rates that is still decades away. These things require exabytes of compute to train.

ChatGTP 4y ago

I only hope it doesn't go the way of that one paper which wanted to ban GPUs for sale to the public. I'd say this is a real possibility though? Not necessarily for or against it, but you can't see this happening, or at least serious discussion of it?

gwd 4y ago

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people they work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. From that perspective, the opening of ChatGPT has actually been very effective at raising awareness. All the way back in GPT-1 they were trying to raise warnings, but those warnings didn't get much popular traction. Now that so many people have used ChatGPT (or Bing), I'm now having conversations about what computers "know" and "want" with my aunt on Facebook. Furthermore, if OpenAI has the best tools and sells them to everyone at a reasonable price, then there's a reduced incentive for other people to make their own tools. Whereas, if they were to close off access to the API, and only offer it to large corporations, there would be much more incentive for people to experiment with AI on their own -- and in doing so, possibly create an "un-aligned" super-intelligent AI which would destroy the human race. So my prediction is that given their motivations, they will 1) stop releasing details of their models to anyone other than research organizations they consider careful enough 2) continue to sell reasonably-priced access to the APIs, to reduce the risk that other people will step up to fill the demand who are less careful.

anonymousDan 4y ago
What a bunch of BS. The only reason they are keeping it private is for commercial gain .
- UncleEntity 4y ago
  Just look at how much flak the stable diffusion folks got for the deep fake porn (and whatever else people were pearl grasping over) and tell me how a corporation will ever release a model. Meta was a fluke but they also did due diligence and made it look like they tried to do a responsible release — right up until someone put it on BitTorrent.

jillesvangurp 4y ago

You are assuming openai is going to end up with a monopoly on all this. IMHO the opposite is going to happen. There are going to be a multitude of companies and researchers competing on outdoing what they are doing in terms of quality, cost, and use cases. If big companies put a straight jacket in place to limit access, constrain usage, etc., that just creates the opportunity for others to step up and grab some market share. There are going to be use cases that are uncomfortable for big companies for ethical, political or other reasons. That's fine. That's their reality. But of course others will step into the void that creates with solutions of their own. And there is also the notion that big companies don't like being dependent on other big companies. OpenAI despite the name is very much not so open and really a Microsoft subsidiary in all but name. So, the likes of Amazon, Facebook, Google, and others are not going to be waiting for them to deliver new features and be creating their own strategies for competing. And that's just the big companies. The rest of the industry will do the same as soon as cost allows them to do that.

DoingIsLearning 4y ago
Indeed, I just saw a demo of Adobe Firefly, and the surprising thing to me is the whole thing was developed internally from data they control. Looking at Nvidia's rental solutions for Nvidia’s A100, it really feels like Future products will be driven by who is sitting on the biggest closed source training datasets more so than this specific success from OpenAI's research.
malwrar 4y ago
I personally expect that regulation or collusion among big tech players (e.g. the suppression of parlor) will prevent the average person or company from having the legal or practical ability to amass the compute power and dataset necessary to train a competing LLM (or future arch). No one really seems to know if OpenAI’s use of copyrighted materials like published works and open source code for training its LLM is legal. I can easily see a future where use of copyrighted works like this simply can’t be repeated legally, and the compute power necessary to do it as an individual is made inaccessible. This especially if the resulting model is made open to everyone, it’s such a wild cultural shift to me seeing tech nerds advocating against democratization of this tech due to personal doomsday fantasies. The sheer number of people who exist in the community and are obsessed with alignment and ethics will provide plenty of ideas on practical constraints the non-technical powerful could impose to make this real.

paganel 4y ago

> big company to pull public access of their APIs. This has been in effect since at least 10 years, I'd say. Twitter was the exception until relatively recently, but trying to build a product using the APIs of companies like Meta or Google became practically useless long ago.