4 ms·

OpenAI has a big problem, the only moat on an LLM is the quality and inference speed. Which is quickly getting commoditized.

arthurcolle 3y ago
Yep, Groq is one example
- BoorishBears 3y ago
  Great example: can't scale enough to remove their 30 requests per minute (!) rate limit and enable billing, barely meets 3.5 levels of intelligence, etc; People don't seem to understand that scaling out LLMs efficiently is it's own art that OpenAI is probably learning lessons on faster than anyone else
  - kaibee 3y ago
    You're thinking of Grok. Not Groq. Blame Elon for this naming catastrophe.
    - BoorishBears 3y ago
      Lol what? No, I'm thinking of Groq. I love Groq though, every time someone hypes up their project using it you instantly know they have no real usage whatsoever. Even the most "toyish" toy project doesn't fit in their rate limits for anything more than personal use.
snapcaster 3y ago
Is it? I thought this would be the case by now but GPT-4 is still in the lead (and released a while ago). How many generations in a row does OpenAI have to dominate before we revisit the "there is no moat" idea?
- ldjkfkdsjnv 3y ago
  Anthropic Opus is better than GPT 4 on many (most?) tasks.
  - HarHarVeryFunny 3y ago
    Not only better as measured by benchmarks, but also better preferred by humans. https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
    - ThunderBee 3y ago
      The most surprising thing to me is that Opus is only slightly in the lead. I was feeding multiple python and c# coding challenges / questions to both and Opus blew GPT4 out of the water on every single task. Didn’t matter if I was giving them 50 lines or 5,000 Opus would consistently give working/correct solutions while GPT4 preferred to answer with pseudo code, half complete code with ‘do the thing here’ comments, or would just tell me that it’s too complicated.
      - happypumpkin 3y ago
        Another data point, I definitely find Opus better for coding, but not by much. The problems I give them are generally short (<= 100 lines) and well-defined so any advantage Opus has in larger contexts won't be apparent to me. They're also generally novel problems but NOT particularly challenging (anyone with a BS CS should be able to solve them in < 1hr). I have them working with mostly C++ and Clojure, a bit of Python, and Vimscript every once in a while. Both models are much better at Python and fairly bad at Vimscript. Clojure failure cases are mostly from invented functions and being bad at modifying existing code. I can't pick out a strong pattern in how they fail with C++, but there have been a few times where GPT4 ends up looping between the same couple unworkable solutions (maybe this indicates a poor understanding of prior context?).
        3y ago
ben_w 3y ago
We don't talk about Google "not having a moat" since their patents on PageRank expired. All the talk of OpenAI's moats (or lack thereof) since the memo, seems like humans being stochastic parrots.
- ldjkfkdsjnv 3y ago
  Google has a data moat on traditional search. Its really hard to train a traditional search system without google scale data. LLMs effectively side stepped that whole issue
  - ben_w 3y ago
    Google's special sauce, is the data of all user's responses to the things Google's algorithm shows them. This description also works for ChatGPT, what it has which others do not (at anything close to the same scale). Google's search engine is a well understood bit of matrix multiplication on a scale many others can easily afford to replicate. This description also works for GPT-3.
    - ldjkfkdsjnv 3y ago
      No becuause ChatGPT was trained on the whole internet using a new algorithm. You needed perfect tagged data for the old algorithm (page rank). LLMs are a fundamental leap forward
      - ben_w 3y ago
        The GPT algorithm isn't a secret. The RLFH data — users giving a thumbs up or down, or regenerating a response, or even getting detectably angry in their subsequent messages — is. PageRank isn't a secret. Google's version of RLFH — which link does a user click on, do they ignore the results without clicking any, do they write several related queries before clicking a link, do they return to the search results soon after clicking a link — are also secret. That the Transformer model is a breakthrough doesn't make it a moat; that the Transformer model is public doesn't mean people using it don't have a moat. Hence why I'm criticising the use of "moat" as mere parroting.
mcbuilder 3y ago
Turboderp released a custom quantized version of dbrx-base and instruct a few days after the official model landed. https://huggingface.co/turboderp/dbrx-base-exl2 https://huggingface.co/turboderp/dbrx-base-exl2 You are right, LLMs weights are fast getting commoditized. As fast as compute is scaling now, I don't think it can continue forever, so the huge advantages big players have is going to get wiped. I'm sure they will always be able to deliver at scale, but a super GPT4 performing model available on your local (or small player type) hardware in the next few years seems more likely than a few big mega-players and mega-models.
xpe 3y ago
The comment above seems to be interpreting OpenAI’s goals in a way that is not consistent with its charter. Structurally, for OpenAI, capped profit is a means not the end. If the capped profit part of OpenAI is not acting in alignment with its mission, it is the OpenAI board’s responsibility to rein it in. — https://openai.com/charter https://openai.com/charter OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. To that end, we commit to the following principles: Broadly distributed benefits … Long-term safety … Technical leadership … Cooperative orientation

arthurcolle 3y ago

Yep, Groq is one example

BoorishBears 3y ago
Great example: can't scale enough to remove their 30 requests per minute (!) rate limit and enable billing, barely meets 3.5 levels of intelligence, etc; People don't seem to understand that scaling out LLMs efficiently is it's own art that OpenAI is probably learning lessons on faster than anyone else
- kaibee 3y ago
  You're thinking of Grok. Not Groq. Blame Elon for this naming catastrophe.
  - BoorishBears 3y ago
    Lol what? No, I'm thinking of Groq. I love Groq though, every time someone hypes up their project using it you instantly know they have no real usage whatsoever. Even the most "toyish" toy project doesn't fit in their rate limits for anything more than personal use.

snapcaster 3y ago

Is it? I thought this would be the case by now but GPT-4 is still in the lead (and released a while ago). How many generations in a row does OpenAI have to dominate before we revisit the "there is no moat" idea?

ldjkfkdsjnv 3y ago
Anthropic Opus is better than GPT 4 on many (most?) tasks.
- HarHarVeryFunny 3y ago
  Not only better as measured by benchmarks, but also better preferred by humans. https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
  - ThunderBee 3y ago
    The most surprising thing to me is that Opus is only slightly in the lead. I was feeding multiple python and c# coding challenges / questions to both and Opus blew GPT4 out of the water on every single task. Didn’t matter if I was giving them 50 lines or 5,000 Opus would consistently give working/correct solutions while GPT4 preferred to answer with pseudo code, half complete code with ‘do the thing here’ comments, or would just tell me that it’s too complicated.
    - happypumpkin 3y ago
      Another data point, I definitely find Opus better for coding, but not by much. The problems I give them are generally short (<= 100 lines) and well-defined so any advantage Opus has in larger contexts won't be apparent to me. They're also generally novel problems but NOT particularly challenging (anyone with a BS CS should be able to solve them in < 1hr). I have them working with mostly C++ and Clojure, a bit of Python, and Vimscript every once in a while. Both models are much better at Python and fairly bad at Vimscript. Clojure failure cases are mostly from invented functions and being bad at modifying existing code. I can't pick out a strong pattern in how they fail with C++, but there have been a few times where GPT4 ends up looping between the same couple unworkable solutions (maybe this indicates a poor understanding of prior context?).
      - 3y ago

ben_w 3y ago

We don't talk about Google "not having a moat" since their patents on PageRank expired. All the talk of OpenAI's moats (or lack thereof) since the memo, seems like humans being stochastic parrots.

ldjkfkdsjnv 3y ago
Google has a data moat on traditional search. Its really hard to train a traditional search system without google scale data. LLMs effectively side stepped that whole issue
- ben_w 3y ago
  Google's special sauce, is the data of all user's responses to the things Google's algorithm shows them. This description also works for ChatGPT, what it has which others do not (at anything close to the same scale). Google's search engine is a well understood bit of matrix multiplication on a scale many others can easily afford to replicate. This description also works for GPT-3.
  - ldjkfkdsjnv 3y ago
    No becuause ChatGPT was trained on the whole internet using a new algorithm. You needed perfect tagged data for the old algorithm (page rank). LLMs are a fundamental leap forward
    - ben_w 3y ago
      The GPT algorithm isn't a secret. The RLFH data — users giving a thumbs up or down, or regenerating a response, or even getting detectably angry in their subsequent messages — is. PageRank isn't a secret. Google's version of RLFH — which link does a user click on, do they ignore the results without clicking any, do they write several related queries before clicking a link, do they return to the search results soon after clicking a link — are also secret. That the Transformer model is a breakthrough doesn't make it a moat; that the Transformer model is public doesn't mean people using it don't have a moat. Hence why I'm criticising the use of "moat" as mere parroting.

mcbuilder 3y ago

Turboderp released a custom quantized version of dbrx-base and instruct a few days after the official model landed. https://huggingface.co/turboderp/dbrx-base-exl2 https://huggingface.co/turboderp/dbrx-base-exl2 You are right, LLMs weights are fast getting commoditized. As fast as compute is scaling now, I don't think it can continue forever, so the huge advantages big players have is going to get wiped. I'm sure they will always be able to deliver at scale, but a super GPT4 performing model available on your local (or small player type) hardware in the next few years seems more likely than a few big mega-players and mega-models.

xpe 3y ago

The comment above seems to be interpreting OpenAI’s goals in a way that is not consistent with its charter. Structurally, for OpenAI, capped profit is a means not the end. If the capped profit part of OpenAI is not acting in alignment with its mission, it is the OpenAI board’s responsibility to rein it in. — https://openai.com/charter https://openai.com/charter OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. To that end, we commit to the following principles: Broadly distributed benefits … Long-term safety … Technical leadership … Cooperative orientation