4 ms·

"Follow the money" generally works as a proxy [0], and I can't get over the fact that Altman sold 49%, and effectively ceded control to MSFT for $29 B. If Chat

"Follow the money" generally works as a proxy [0], and I can't get over the fact that Altman sold 49%, and effectively ceded control to MSFT for $29 B. If ChatGPT was all that, you'd imagine at least 10x with OpenAIs founders keeping control being the baseline [0] https://garymarcus.substack.com/p/is-microsoft-about-to-get-the-deal https://garymarcus.substack.com/p/is-microsoft-about-to-get-...

api 3y ago
It is all that. The problem is that there isn’t much of a moat. You can get GPT-3 level performance on a high end laptop and GPT-4 equivalent on high end hardware seems like it’s coming. A lot of advancement is happening in quantization and pruning and training can be done by anyone with access to enough compute. This is going to just become a ubiquitous tool and part of the computing toolbox. I am increasingly thinking the winner take all dynamics of social media and search may not apply. There’s going to be many cloud hosted AIs and lots you can run yourself if you feel like spending a few thousand dollars on hardware. That cost will fall as more special purpose NPU hardware enters the market and acceleration even becomes a standard part of CPUs.
- cced 3y ago
  Is a GPT3-like model currently available for discussion type use and works on a laptop? How might I go about getting started with locally running a model of this quality?
  - api 3y ago
    The large Alpaca models can run on a M1 Pro MacBook or similar performance Linux PC with llama.cpp. You will need 32-64G of RAM and fast SSD for the large models. They show performance that reminds me of GPT-3 (not quite 3.5). There are some newer models out there I have not tried yet like the open llama, GPT4all, etc. so I’m not sure how good they are. I get the sense they are still GPT-3 level but are achieving that with less RAM. There’s a race on both for raw capability and optimization via pruning and quantization. The latter is important to make these things runnable locally without gigantic hardware. Lots of people have stuff with GPUs, fast CPUs, and 32-64G RAM. Few have huge workstations with hundreds of gigs of RAM. Unless progress stagnates I can see something approaching GPT-4 that can run on under $5k worth of hardware in a year or so. Open model progress seems to be lagging only 1-2 years behind big cloud hosted models.
  - panzi 3y ago
    I heard people claiming this is at GPT 3 level: https://open-assistant.io/ https://open-assistant.io/ Don't know if true. It's open source.
  - infinityio 3y ago
    The quality does not yet measure up exactly to ChatGPT (even 3.5), but yes it is possible Probably the fastest way to get started is to look into [0] - this only requires a beta chromium browser with WebGPU. For a more integrated setup, I am under the impression [1] is the main tool used. If you want to take a look at the quality possible before getting started, [2] is an online service by Hugging Face that hosts one of the best of the current generation of open models (OpenAssistant w/ 30B LLaMa) [0]: https://mlc.ai/web-llm/ https://mlc.ai/web-llm/ [1]: https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui [2]: https://huggingface.co/chat https://huggingface.co/chat
    - api 3y ago
      I downloaded a version of that openassistant model for llama.cpp and it’s at least on par with GPT-3 or a little beyond. It’s to the level of being generally useful.
  - diarized 3y ago
    Google “We have no moat, and neither does OpenAI” https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322
- nico 3y ago
  That’s the tech aspect, AI is essentially the new Linux On top of that, given Metcalfe’s law, communication and coordination cost of a team grows exponentially with the size of the team, which means small teams have a huge advantage over big teams
  - sitkack 3y ago
    Maybe LLMs upend Metcalfe's Law? But I agree, the amount of velocity of a 2-5 person team using LLMs is astrounding.
- ramraj07 3y ago
  Given that neither Google or Meta or any other company or open source solution is even coming close to the performance of merely the publicly released handicapped version of gpt-4 (no image interpretation, plug-ins is not fully released, etc), OpenAI might not have a moat (I’d argue there is a moat: the moat seems to be only a handful of people in the planet seem to be smart enough to innovate at this level), they have a year or more headstart and are clearly exploiting it and not slowing down. So yeah if I could buy openai shares I’ll buy it at a $500B valuation at this point.
  - api 3y ago
    It’s a lead but not a moat. If they slow down they are not likely to maintain that lead. Similar things could be said for example about SpaceX with reusable rockets, but with a capital intensive industry like that it might take a decade or more for others to catch up. Software iteration time is very fast, so if OpenAI slows down others could catch up in a year or two.
- osigurdson 3y ago
  OpenAI offers several services: completions (i.e. chat GPT), embeddings, moderation, whisper, etc. Of these, I am convinced that embeddings are the true vendor lock in of our time. Once you base everything on one company's embeddings engine, it will be very difficult to switch to another without significant cost / effort. As a concrete example, assume that embedding vectors are just two dimensional (in reality OpenAI's are 1536D). "cat" might map to [0.1, 0.9] in OpenAI while the same term maps to [0.7,0.3] in another company's engine. The mapping is completely non linear so matrix multiplication or other basic tools cannot be use to find a mapping. The "Rosetta stone" in this case is another massive LLM which would be exceedingly expensive to create. I'd posit that this will create a moat and OpenAI/MSFT are in a good position in this regard. Completions have such a simple API I can see this becoming almost as ubiquitous as s3. It will be trivial for companies to switch this aspect.
- yyyk 3y ago
  There are open source AI models which can approach GPT-3 on an academic comparison done by the people who created these OSS models. There's however zero progress towards OSS products using these AIs, while Microsoft and various startups are already working on products using OpenAI's tools. OpenAI/Microsoft don't need superior performance to have a moat (though they do have it, at least for now). They could just use the existing moats and make them unassailable. Or add new moats via API or hardware.
  - api 3y ago
    You are correct that typically OSS is unable or unwilling to do “product,” largely because OSS in its raw form is in fact enough of a product for its main developer-user demographic. That being said I still see a much shallower moat here than there was in say Internet search. I can download a model in a few minutes. One could not download an entire web crawl index in any reasonable amount of time, nor could one update the index in real time continuously without enormous amounts of bandwidth and compute. DIY self hosted Google was to my knowledge barely even attempted due to the intrinsic difficulties. This stuff is also not about communication or sharing, areas where there are very strong network effect moats. It’s not chat or social media or collaborative office software. Generative AI is more like a stand alone application software. It can be hosted in the cloud but it’s easy to stand up competitors and it can be self hosted if you are willing to spring for the hardware. I’m not saying you won’t have big dominant players, just that I see more opportunity for competition and diversity here than for lots of other things.
    - yyyk 3y ago
      >This stuff is also not about communication or sharing, areas where there are very strong network effect moats. It’s not chat or social media or collaborative office software. An example business need: Take all my documents and create an LLM acting as an internal knowledgebase. Likely eventual Microsoft/Google solution: Press this button to take all your documents from your Office 365/GSuite account into our LLM. The LLM provides answers and links to the original document. We have automatic retraining, but you can remove/add data and retrain with a few more clicks. We have set up authentication and filtering so that unauthorized users can't get at your data. Likely eventual OSS 'solution': Find your API key, download all your documents, and train them manually on the NVidia GPU cards you bought. Setting up a virtual python environment with CUDA is easy-peasy! Now, host the documents on your curlftpfs host so that our LLM could link to them.... (I could continue but this is too depressing). I’m not saying there won't be small players, but the Google Research spin of 'no point in investing anything, Open Source will eat all!' was silly in the extreme. There'll be other products, and a smart enough OpenAI has good chances to create its own moats.
ilarum 3y ago
Your link indicates that the founders get back full control once OpenAI makes a ton of money.
- HarHarVeryFunny 3y ago
  I don't think that's quite it. As I understand it their corporate structure basically has two entities a non-profit and OpenAI LP which has been accepting capped-profit investments. Different investors (early vs late) have received different capped-profit deals from 100x to 7x. Once all investors are paid off then all further profits are meant to flow to the non-profit (not sure what they are meant to do with it!). Not sure about how this related to the founders, other than Sam Altamn apparently having zero financial stake. I'm guessing the other founders may have put money in and have capped-profit deals ?
MichaelMoser123 3y ago
judging by the amount of money involved: is it correct to say that we now have a space race towards AGI going on between a few big corporations? Microsoft is reported to have invested billions into OpenAI

api 3y ago

It is all that. The problem is that there isn’t much of a moat. You can get GPT-3 level performance on a high end laptop and GPT-4 equivalent on high end hardware seems like it’s coming. A lot of advancement is happening in quantization and pruning and training can be done by anyone with access to enough compute. This is going to just become a ubiquitous tool and part of the computing toolbox. I am increasingly thinking the winner take all dynamics of social media and search may not apply. There’s going to be many cloud hosted AIs and lots you can run yourself if you feel like spending a few thousand dollars on hardware. That cost will fall as more special purpose NPU hardware enters the market and acceleration even becomes a standard part of CPUs.

cced 3y ago
Is a GPT3-like model currently available for discussion type use and works on a laptop? How might I go about getting started with locally running a model of this quality?
- api 3y ago
  The large Alpaca models can run on a M1 Pro MacBook or similar performance Linux PC with llama.cpp. You will need 32-64G of RAM and fast SSD for the large models. They show performance that reminds me of GPT-3 (not quite 3.5). There are some newer models out there I have not tried yet like the open llama, GPT4all, etc. so I’m not sure how good they are. I get the sense they are still GPT-3 level but are achieving that with less RAM. There’s a race on both for raw capability and optimization via pruning and quantization. The latter is important to make these things runnable locally without gigantic hardware. Lots of people have stuff with GPUs, fast CPUs, and 32-64G RAM. Few have huge workstations with hundreds of gigs of RAM. Unless progress stagnates I can see something approaching GPT-4 that can run on under $5k worth of hardware in a year or so. Open model progress seems to be lagging only 1-2 years behind big cloud hosted models.
- panzi 3y ago
  I heard people claiming this is at GPT 3 level: https://open-assistant.io/ https://open-assistant.io/ Don't know if true. It's open source.
- infinityio 3y ago
  The quality does not yet measure up exactly to ChatGPT (even 3.5), but yes it is possible Probably the fastest way to get started is to look into [0] - this only requires a beta chromium browser with WebGPU. For a more integrated setup, I am under the impression [1] is the main tool used. If you want to take a look at the quality possible before getting started, [2] is an online service by Hugging Face that hosts one of the best of the current generation of open models (OpenAssistant w/ 30B LLaMa) [0]: https://mlc.ai/web-llm/ https://mlc.ai/web-llm/ [1]: https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui [2]: https://huggingface.co/chat https://huggingface.co/chat
  - api 3y ago
    I downloaded a version of that openassistant model for llama.cpp and it’s at least on par with GPT-3 or a little beyond. It’s to the level of being generally useful.
- diarized 3y ago
  Google “We have no moat, and neither does OpenAI” https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322
nico 3y ago
That’s the tech aspect, AI is essentially the new Linux On top of that, given Metcalfe’s law, communication and coordination cost of a team grows exponentially with the size of the team, which means small teams have a huge advantage over big teams
- sitkack 3y ago
  Maybe LLMs upend Metcalfe's Law? But I agree, the amount of velocity of a 2-5 person team using LLMs is astrounding.
ramraj07 3y ago
Given that neither Google or Meta or any other company or open source solution is even coming close to the performance of merely the publicly released handicapped version of gpt-4 (no image interpretation, plug-ins is not fully released, etc), OpenAI might not have a moat (I’d argue there is a moat: the moat seems to be only a handful of people in the planet seem to be smart enough to innovate at this level), they have a year or more headstart and are clearly exploiting it and not slowing down. So yeah if I could buy openai shares I’ll buy it at a $500B valuation at this point.
- api 3y ago
  It’s a lead but not a moat. If they slow down they are not likely to maintain that lead. Similar things could be said for example about SpaceX with reusable rockets, but with a capital intensive industry like that it might take a decade or more for others to catch up. Software iteration time is very fast, so if OpenAI slows down others could catch up in a year or two.
osigurdson 3y ago
OpenAI offers several services: completions (i.e. chat GPT), embeddings, moderation, whisper, etc. Of these, I am convinced that embeddings are the true vendor lock in of our time. Once you base everything on one company's embeddings engine, it will be very difficult to switch to another without significant cost / effort. As a concrete example, assume that embedding vectors are just two dimensional (in reality OpenAI's are 1536D). "cat" might map to [0.1, 0.9] in OpenAI while the same term maps to [0.7,0.3] in another company's engine. The mapping is completely non linear so matrix multiplication or other basic tools cannot be use to find a mapping. The "Rosetta stone" in this case is another massive LLM which would be exceedingly expensive to create. I'd posit that this will create a moat and OpenAI/MSFT are in a good position in this regard. Completions have such a simple API I can see this becoming almost as ubiquitous as s3. It will be trivial for companies to switch this aspect.
yyyk 3y ago
There are open source AI models which can approach GPT-3 on an academic comparison done by the people who created these OSS models. There's however zero progress towards OSS products using these AIs, while Microsoft and various startups are already working on products using OpenAI's tools. OpenAI/Microsoft don't need superior performance to have a moat (though they do have it, at least for now). They could just use the existing moats and make them unassailable. Or add new moats via API or hardware.
- api 3y ago
  You are correct that typically OSS is unable or unwilling to do “product,” largely because OSS in its raw form is in fact enough of a product for its main developer-user demographic. That being said I still see a much shallower moat here than there was in say Internet search. I can download a model in a few minutes. One could not download an entire web crawl index in any reasonable amount of time, nor could one update the index in real time continuously without enormous amounts of bandwidth and compute. DIY self hosted Google was to my knowledge barely even attempted due to the intrinsic difficulties. This stuff is also not about communication or sharing, areas where there are very strong network effect moats. It’s not chat or social media or collaborative office software. Generative AI is more like a stand alone application software. It can be hosted in the cloud but it’s easy to stand up competitors and it can be self hosted if you are willing to spring for the hardware. I’m not saying you won’t have big dominant players, just that I see more opportunity for competition and diversity here than for lots of other things.
  - yyyk 3y ago
    >This stuff is also not about communication or sharing, areas where there are very strong network effect moats. It’s not chat or social media or collaborative office software. An example business need: Take all my documents and create an LLM acting as an internal knowledgebase. Likely eventual Microsoft/Google solution: Press this button to take all your documents from your Office 365/GSuite account into our LLM. The LLM provides answers and links to the original document. We have automatic retraining, but you can remove/add data and retrain with a few more clicks. We have set up authentication and filtering so that unauthorized users can't get at your data. Likely eventual OSS 'solution': Find your API key, download all your documents, and train them manually on the NVidia GPU cards you bought. Setting up a virtual python environment with CUDA is easy-peasy! Now, host the documents on your curlftpfs host so that our LLM could link to them.... (I could continue but this is too depressing). I’m not saying there won't be small players, but the Google Research spin of 'no point in investing anything, Open Source will eat all!' was silly in the extreme. There'll be other products, and a smart enough OpenAI has good chances to create its own moats.

ilarum 3y ago

Your link indicates that the founders get back full control once OpenAI makes a ton of money.

HarHarVeryFunny 3y ago
I don't think that's quite it. As I understand it their corporate structure basically has two entities a non-profit and OpenAI LP which has been accepting capped-profit investments. Different investors (early vs late) have received different capped-profit deals from 100x to 7x. Once all investors are paid off then all further profits are meant to flow to the non-profit (not sure what they are meant to do with it!). Not sure about how this related to the founders, other than Sam Altamn apparently having zero financial stake. I'm guessing the other founders may have put money in and have capped-profit deals ?

MichaelMoser123 3y ago

judging by the amount of money involved: is it correct to say that we now have a space race towards AGI going on between a few big corporations? Microsoft is reported to have invested billions into OpenAI