8 ms·

I believe you can't, at the same time, accuse OpenAI of benefitting of ideas mostly invented elsewhere, and also claim that they are in a predominant position.

I believe you can't, at the same time, accuse OpenAI of benefitting of ideas mostly invented elsewhere, and also claim that they are in a predominant position. It does not make any sense, and they are not naturally positioned for a monopoly. Just other companies have to move their asses.

tomlue 4y ago
I'm a fan of OpenAI, but this is nonsense. All of human existence is mostly other people's ideas. Among a ridiculously huge list of other things, OpenAI benefits from the mountains of labor that made scalable neural networks possible. Have they had their own good ideas? Definitely. Are they benefitting of ideas mostly invented elsewhere? Also Definitely, just like everybody else.
- antirez 4y ago
  I don't think you understood me. I'm with you, but given that often OpenAI is accused of using public ideas for profit, this, in turn, means they are obviously not in a dominant position. So far they are just better than the others.
- version_five 4y ago
  I'm not really a fan of openAI, but I think we're seeing the classic mistake of confusing product with technology. Steve Jobs / Apple didn't anything you'd call new ideas either (obvious cliche but so is the criticism). It's execution and design once the tech reaches a certain level
  - tomlue 4y ago
    mostly agree. Though I wouldn't underestimate the tech and engineering work behind OpenAI. That microsoft partnership is no joke.
  - poopypoopington 4y ago
    I've heard Altman (on the Lex Friedman podcast) and Sundar Pichai (on the Hard Fork podcast) say things to this degree. The thing that OpenAI really managed to crack was building a great product in ChatGPT and finding a good product market fit for LLMs.
    - Robotbeat 4y ago
      Well sure, but there still aren’t any other LLMs at the level of GPT3/3.5 let alone GPT-4. GPT3.5 just using the API returns fantastic results even without the ChatGPT interface (which isn’t terribly hard to replicate, and others have using the API). There are dozens if not hundreds of companies that could’ve done something profound like ChatGPT if they had full access to GPT3/3.5. And honestly, OpenAI stumbled a lot with ChatGPT losing history access, showing other users’ history… but that doesn’t matter much as the underlying technology is so profound. I think this really is a case of the under-the-hood capability (GPT3/3.5/4) mattering more than productization and execution. (Now I think there are not a ton of companies that could do what Microsoft is trying to do by expanding GPT4 to power office productivity… that is a separate thing and probably only about 3 companies could do that, at best: Microsoft, Apple, and Google… and theoretically Meta but their lack of follow through with making Metaverse useful makes me doubt it.)
      - mustacheemperor 4y ago
        Hm, I wonder how much of the API’s performance is related to training/finetuning done by OpenAI planned towards the ChatGPT product. I think the RLHF is partly product design and partly engineering.
    - anon84873628 4y ago
      What's the product market fit for LLMs, and how does OpenAI fill it?
      - fooster 4y ago
        How much money is openai making from chatgpt premium now? How much revenue are they they making from the api?
  - brookst 4y ago
    It’s a good comparison. And once again tech enthusiasts are confused and outraged that the product people are getting credit for tech they didn’t invent. Once again missing the forest that people buy products, not tech.
    - simonh 4y ago
      There’s an awful lot of judgement, engineering and technique that goes into a really well thought out product. It’s often deeply underestimated, and culture makes a huge difference in execution. Bing/Sydney came out after ChatGPT, based on exactly the same tech, but was hot garbage.
      - FormerBandmate 4y ago
        Bing/Sydney is better than ChatGPT. It had serious bugs in beta testing
        simonh 4y ago
        It’s a dramatically worse chat bot, but being able to search the internet does give it an additional useful capability, while limiting it to five interactions papers over its psychotic tendencies.
        HarHarVeryFunny 4y ago
        Try it again - seems very good now (and allows longer conversations too).
      - mustacheemperor 4y ago
        It’s interesting to see the ad implementation, I recall some predictions that Microsoft would be particularly apt at finding a way to integrate advertising organically. Instead it just seems to have made the bot more stupid because sponsored products are forced into its recommendations. I’ve gone back to just using the GPT API, unless I absolutely need to search the internet or information after 2021 for some reason.
  - vitehozonage 4y ago
    Rather than execution or design perhaps this time it was mainly about being unethical enough to sell out to investors and unflinchingly gather enough data in one place by carelessly ignoring copyright, authors' desires, privacy regulations, etc.
- wslh 4y ago
  > OpenAI benefits from the mountains of labor ... Can't you say the same about Google? Google lives from the labor of others. Not entering into the debate if this is ethically or not just saying that OpenAI is not much different. When photo services such as Google Photos recognize the Eiffer Tower in Paris they are using images from others.
ftxbro 4y ago
> I believe you can't, at the same time, accuse OpenAI of benefitting of ideas mostly invented elsewhere, and also claim that they are in a predominant position. There used to be a team at Google called Google Brain and they all left to go to OpenAI after the employee protests against taking military AI contracts in 2018. Now Microsoft has those contracts and funneled $10B to OpenAI from the CIA. OK that's a little bit of exaggeration but not so much; I guess not all employees left, and Google Brain still technically exists. Also some of the brain employees went to other startups not only OpenAI.
- beebmam 4y ago
  Many of those employees have changed their minds about military contracts in this new cold war era.
  - InCityDreams 4y ago
    America! What is in your military now, eventually ends up in the hands of the civilian police. *and other countries are following. Fucking, sadly.
    - sebzim4500 4y ago
      What the hell are they going to do with an M1 Abrams? Having said that, I dread to see what the NYPD manage to achieve with an F-35B.
      - krainboltgreene 4y ago
        Trump tried to convince officials to use tanks to combat the Portland protests while he was president.
  - cowl 4y ago
    If I'm not mistaken their opposition was on principle not whether it was needed or not. So the fact that we are in a new cold war era does not change that Equation. The principle is still the same. The only way for this "change of mind" is if their opposition was due to "it's not needed because US has no rivals". Or what most probably happened, they realised that if they want to keep workign on this field there is no escape from those kind of implications and They don't have the big bad wolf Google to blame anymore.
ren_engineer 4y ago
OpenAI acts like they are some underdog startup for PR purposes while actually having access to effectively unlimited resources due to their relationship with Microsoft, which rubs people the wrong way Plenty of other companies had released GPT powered chat bots like ChatGPT, they just couldn't offer it for free because they didn't have a sweetheart deal with Microsoft for unlimited GPUs. Google did drop the ball though, they were afraid of reputation risk. Google should have used DeepMind or another spinoff to release their internal chatbot months ago
- nomel 4y ago
  > to release their internal chatbot months ago I’m not sure that would have been wise. Bard clearly isn’t ready.
  - sebzim4500 4y ago
    Yeah but if they released earlier that fact wouldn't have been so embarassing. As it was, they intially just claimed that releasing a competitor was irresponsible, then they eventually did it anyway (badly).
- simonh 4y ago
  I think you’re missing how much of a profound difference the intensive RLHF training OpenAI did makes in ChatGPT. Microsoft’s Sydney seems to also be GPT 3.5 based, came out after ChatGPT, and it was an utter dumpster fire on launch in comparison. Meanwhile nobody has even caught up to ChatGPT yet, not even Microsoft whose resources are the secret sauce you think is the game changer, and now 4.0 is out and even more massively moved the ball forward.
  - jamaliki 4y ago
    No, actually. Microsoft's Sydney is GPT 4 [1]. 1 - https://blogs.bing.com/search/march_2023/Confirmed-the-new-Bing-runs-on-OpenAI%E2%80%99s-GPT-4 https://blogs.bing.com/search/march_2023/Confirmed-the-new-B...
rhaway84773 4y ago
I’m not sure why that doesn’t make sense. In a winner takes all market, where the product is highly complex, which is likely the case for any product developed since the advent of computers, if not before, the predominant position will almost certainly be taken by somebody, and since it’s a highly complex product, it’s likely no one entity thought of and/or implemented even close to a majority of the ideas needed to make it work. In fact, it’s likely to be a random winner amongst 10-20 entities who implemented some of the ideas, and another potentially larger number of entities who implemented equally good, or even better, ideas, which happened to fail for reasons that couldn’t have been known in advance.
- curiousllama 4y ago
  > In a winner takes all market Just because Search was winner take all, doesn't mean AI will be. What network effects or economies of scale are unachievable by competitors? Besides, Alpaca showed you can replicate ChatGPT on the cheap once its built - what's stopping others from succeeding?
  - tyfon 4y ago
    Yeah, I don't think this will be a corporate thing but a private decentralized thing. It's probably the worst fear of the likes of google etc, a 100% local "search engine" / knowledge center that does not even require the internet. I've been running the 65B model for a bit. With the correct prompt engineering you get very good results even without any fine tuning. I can run stable diffusion etc fine too locally. If anything will let us break free from the corporate, it is this.
  - moritzwarhier 4y ago
    Alpaca is surprisingly good, but mostly because it supports the dialogue format out of the box. IME, it does not nearly replicate ChatGPT.
matthewdgreen 4y ago
There have been many examples of companies inventing new technology, then failing to take it to market until a competitor copied the ideas. The classic example is Apple and Xerox PARC. The criticism in this case is that while icons and GUIs are obviously harmless (so it's good we got them out of the lab!), maybe AI is the kind of tech we should have let researchers play with for a while longer, before we started an arms race that puts it in everyone's house.

tomlue 4y ago

I'm a fan of OpenAI, but this is nonsense. All of human existence is mostly other people's ideas. Among a ridiculously huge list of other things, OpenAI benefits from the mountains of labor that made scalable neural networks possible. Have they had their own good ideas? Definitely. Are they benefitting of ideas mostly invented elsewhere? Also Definitely, just like everybody else.

antirez 4y ago
I don't think you understood me. I'm with you, but given that often OpenAI is accused of using public ideas for profit, this, in turn, means they are obviously not in a dominant position. So far they are just better than the others.
version_five 4y ago
I'm not really a fan of openAI, but I think we're seeing the classic mistake of confusing product with technology. Steve Jobs / Apple didn't anything you'd call new ideas either (obvious cliche but so is the criticism). It's execution and design once the tech reaches a certain level
- tomlue 4y ago
  mostly agree. Though I wouldn't underestimate the tech and engineering work behind OpenAI. That microsoft partnership is no joke.
- poopypoopington 4y ago
  I've heard Altman (on the Lex Friedman podcast) and Sundar Pichai (on the Hard Fork podcast) say things to this degree. The thing that OpenAI really managed to crack was building a great product in ChatGPT and finding a good product market fit for LLMs.
  - Robotbeat 4y ago
    Well sure, but there still aren’t any other LLMs at the level of GPT3/3.5 let alone GPT-4. GPT3.5 just using the API returns fantastic results even without the ChatGPT interface (which isn’t terribly hard to replicate, and others have using the API). There are dozens if not hundreds of companies that could’ve done something profound like ChatGPT if they had full access to GPT3/3.5. And honestly, OpenAI stumbled a lot with ChatGPT losing history access, showing other users’ history… but that doesn’t matter much as the underlying technology is so profound. I think this really is a case of the under-the-hood capability (GPT3/3.5/4) mattering more than productization and execution. (Now I think there are not a ton of companies that could do what Microsoft is trying to do by expanding GPT4 to power office productivity… that is a separate thing and probably only about 3 companies could do that, at best: Microsoft, Apple, and Google… and theoretically Meta but their lack of follow through with making Metaverse useful makes me doubt it.)
    - mustacheemperor 4y ago
      Hm, I wonder how much of the API’s performance is related to training/finetuning done by OpenAI planned towards the ChatGPT product. I think the RLHF is partly product design and partly engineering.
  - anon84873628 4y ago
    What's the product market fit for LLMs, and how does OpenAI fill it?
    - fooster 4y ago
      How much money is openai making from chatgpt premium now? How much revenue are they they making from the api?
- brookst 4y ago
  It’s a good comparison. And once again tech enthusiasts are confused and outraged that the product people are getting credit for tech they didn’t invent. Once again missing the forest that people buy products, not tech.
  - simonh 4y ago
    There’s an awful lot of judgement, engineering and technique that goes into a really well thought out product. It’s often deeply underestimated, and culture makes a huge difference in execution. Bing/Sydney came out after ChatGPT, based on exactly the same tech, but was hot garbage.
    - FormerBandmate 4y ago
      Bing/Sydney is better than ChatGPT. It had serious bugs in beta testing
      - simonh 4y ago
        It’s a dramatically worse chat bot, but being able to search the internet does give it an additional useful capability, while limiting it to five interactions papers over its psychotic tendencies.
        HarHarVeryFunny 4y ago
        Try it again - seems very good now (and allows longer conversations too).
    - mustacheemperor 4y ago
      It’s interesting to see the ad implementation, I recall some predictions that Microsoft would be particularly apt at finding a way to integrate advertising organically. Instead it just seems to have made the bot more stupid because sponsored products are forced into its recommendations. I’ve gone back to just using the GPT API, unless I absolutely need to search the internet or information after 2021 for some reason.
- vitehozonage 4y ago
  Rather than execution or design perhaps this time it was mainly about being unethical enough to sell out to investors and unflinchingly gather enough data in one place by carelessly ignoring copyright, authors' desires, privacy regulations, etc.
wslh 4y ago
> OpenAI benefits from the mountains of labor ... Can't you say the same about Google? Google lives from the labor of others. Not entering into the debate if this is ethically or not just saying that OpenAI is not much different. When photo services such as Google Photos recognize the Eiffer Tower in Paris they are using images from others.

ftxbro 4y ago

> I believe you can't, at the same time, accuse OpenAI of benefitting of ideas mostly invented elsewhere, and also claim that they are in a predominant position. There used to be a team at Google called Google Brain and they all left to go to OpenAI after the employee protests against taking military AI contracts in 2018. Now Microsoft has those contracts and funneled $10B to OpenAI from the CIA. OK that's a little bit of exaggeration but not so much; I guess not all employees left, and Google Brain still technically exists. Also some of the brain employees went to other startups not only OpenAI.

beebmam 4y ago
Many of those employees have changed their minds about military contracts in this new cold war era.
- InCityDreams 4y ago
  America! What is in your military now, eventually ends up in the hands of the civilian police. *and other countries are following. Fucking, sadly.
  - sebzim4500 4y ago
    What the hell are they going to do with an M1 Abrams? Having said that, I dread to see what the NYPD manage to achieve with an F-35B.
    - krainboltgreene 4y ago
      Trump tried to convince officials to use tanks to combat the Portland protests while he was president.
- cowl 4y ago
  If I'm not mistaken their opposition was on principle not whether it was needed or not. So the fact that we are in a new cold war era does not change that Equation. The principle is still the same. The only way for this "change of mind" is if their opposition was due to "it's not needed because US has no rivals". Or what most probably happened, they realised that if they want to keep workign on this field there is no escape from those kind of implications and They don't have the big bad wolf Google to blame anymore.

ren_engineer 4y ago

OpenAI acts like they are some underdog startup for PR purposes while actually having access to effectively unlimited resources due to their relationship with Microsoft, which rubs people the wrong way Plenty of other companies had released GPT powered chat bots like ChatGPT, they just couldn't offer it for free because they didn't have a sweetheart deal with Microsoft for unlimited GPUs. Google did drop the ball though, they were afraid of reputation risk. Google should have used DeepMind or another spinoff to release their internal chatbot months ago

nomel 4y ago
> to release their internal chatbot months ago I’m not sure that would have been wise. Bard clearly isn’t ready.
- sebzim4500 4y ago
  Yeah but if they released earlier that fact wouldn't have been so embarassing. As it was, they intially just claimed that releasing a competitor was irresponsible, then they eventually did it anyway (badly).
simonh 4y ago
I think you’re missing how much of a profound difference the intensive RLHF training OpenAI did makes in ChatGPT. Microsoft’s Sydney seems to also be GPT 3.5 based, came out after ChatGPT, and it was an utter dumpster fire on launch in comparison. Meanwhile nobody has even caught up to ChatGPT yet, not even Microsoft whose resources are the secret sauce you think is the game changer, and now 4.0 is out and even more massively moved the ball forward.
- jamaliki 4y ago
  No, actually. Microsoft's Sydney is GPT 4 [1]. 1 - https://blogs.bing.com/search/march_2023/Confirmed-the-new-Bing-runs-on-OpenAI%E2%80%99s-GPT-4 https://blogs.bing.com/search/march_2023/Confirmed-the-new-B...

rhaway84773 4y ago

I’m not sure why that doesn’t make sense. In a winner takes all market, where the product is highly complex, which is likely the case for any product developed since the advent of computers, if not before, the predominant position will almost certainly be taken by somebody, and since it’s a highly complex product, it’s likely no one entity thought of and/or implemented even close to a majority of the ideas needed to make it work. In fact, it’s likely to be a random winner amongst 10-20 entities who implemented some of the ideas, and another potentially larger number of entities who implemented equally good, or even better, ideas, which happened to fail for reasons that couldn’t have been known in advance.

curiousllama 4y ago
> In a winner takes all market Just because Search was winner take all, doesn't mean AI will be. What network effects or economies of scale are unachievable by competitors? Besides, Alpaca showed you can replicate ChatGPT on the cheap once its built - what's stopping others from succeeding?
- tyfon 4y ago
  Yeah, I don't think this will be a corporate thing but a private decentralized thing. It's probably the worst fear of the likes of google etc, a 100% local "search engine" / knowledge center that does not even require the internet. I've been running the 65B model for a bit. With the correct prompt engineering you get very good results even without any fine tuning. I can run stable diffusion etc fine too locally. If anything will let us break free from the corporate, it is this.
- moritzwarhier 4y ago
  Alpaca is surprisingly good, but mostly because it supports the dialogue format out of the box. IME, it does not nearly replicate ChatGPT.

matthewdgreen 4y ago

There have been many examples of companies inventing new technology, then failing to take it to market until a competitor copied the ideas. The classic example is Apple and Xerox PARC. The criticism in this case is that while icons and GUIs are obviously harmless (so it's good we got them out of the lab!), maybe AI is the kind of tech we should have let researchers play with for a while longer, before we started an arms race that puts it in everyone's house.