3 ms·

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the fro

by cubefox 1mo ago

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the frontier itself.

disgruntledphd2 1mo ago
I'm not really convinced that there's much secret sauce here, all the methods and data are public, the only real difference is how much compute it takes.
- Amekedl 1mo ago
  everybody do be cooking with water. Chinese Labs provided pretty good, primarily cost-reducing techniques, like the sparse attention patterns recently. I'd bet OpenAI and Anthropic use their variants of those too, so they can get greater margin on their tokens - not something they'd really want to / need to self-report.
- cubefox 1mo ago
  This is obviously false. There is "secret sauce" because in fact not all the methods and data are public.
  - disgruntledphd2 1mo ago
    Where does the secret sauce show up in the outputs, then? Like, (apart from tone), I find it hard to distinguish between the outputs of GPT/Claude/Kimi/GLM recently (I use cursor, and have been giving them the same prompt and comparing). If anything, I found that the non-Claude models were better in many cases, which definitely doesn't map to their pricing. > in fact not all the methods and data are public Probably not, but unless you work at a lab, I'm not sure that anyone can say (and if you do work at a lab, you should not be replying on this thread).
    - cubefox 1mo ago
      > Where does the secret sauce show up in the outputs, then? In benchmarks, revenue, and comments from a lot of people on Hacker News.
      - disgruntledphd2 1mo ago
        > revenue Maybe, I'm not sure this will continue. > In benchmarks All published benchmarks are useless, unfortunately.
- didroe 1mo ago
  All the methods and data are not public. We don't know what unpublished methods they're using. You can get most of the pre-training data publicly but they've probably spent a ton of money curating it and are now doing things like buying rare books. The RL training data is all (/mostly) proprietary though, and that's the real secret sauce part.
  - woctordho 1mo ago
    All the RL data are exactly public. There are huge amount of distilled data freely available, and that amount is more than enough to train a ~10T model.
    - cubefox 1mo ago
      > All the RL data are exactly public. Nope, because the big AI companies are paying billions for it. They wouldn't pay anything for public data.
      - woctordho 1mo ago
        There are 'transfer stations' and that's how exactly I use GPT and Claude in China. OpenAI and Anthropic do not sell in China, so we use their AI with a much lower price like 1% of the official API price. The largest transfer stations have TBs of traffic every day, and the traffic is eventually possessed by the open source community. Subscription engineering is a deep field. Neither OpenAI nor Anthropic have any technical advantage in this field.

by cubefox 1mo ago

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the frontier itself.

disgruntledphd2 1mo ago

I'm not really convinced that there's much secret sauce here, all the methods and data are public, the only real difference is how much compute it takes.

Amekedl 1mo ago
everybody do be cooking with water. Chinese Labs provided pretty good, primarily cost-reducing techniques, like the sparse attention patterns recently. I'd bet OpenAI and Anthropic use their variants of those too, so they can get greater margin on their tokens - not something they'd really want to / need to self-report.
cubefox 1mo ago
This is obviously false. There is "secret sauce" because in fact not all the methods and data are public.
- disgruntledphd2 1mo ago
  Where does the secret sauce show up in the outputs, then? Like, (apart from tone), I find it hard to distinguish between the outputs of GPT/Claude/Kimi/GLM recently (I use cursor, and have been giving them the same prompt and comparing). If anything, I found that the non-Claude models were better in many cases, which definitely doesn't map to their pricing. > in fact not all the methods and data are public Probably not, but unless you work at a lab, I'm not sure that anyone can say (and if you do work at a lab, you should not be replying on this thread).
  - cubefox 1mo ago
    > Where does the secret sauce show up in the outputs, then? In benchmarks, revenue, and comments from a lot of people on Hacker News.
    - disgruntledphd2 1mo ago
      > revenue Maybe, I'm not sure this will continue. > In benchmarks All published benchmarks are useless, unfortunately.
didroe 1mo ago
All the methods and data are not public. We don't know what unpublished methods they're using. You can get most of the pre-training data publicly but they've probably spent a ton of money curating it and are now doing things like buying rare books. The RL training data is all (/mostly) proprietary though, and that's the real secret sauce part.
- woctordho 1mo ago
  All the RL data are exactly public. There are huge amount of distilled data freely available, and that amount is more than enough to train a ~10T model.
  - cubefox 1mo ago
    > All the RL data are exactly public. Nope, because the big AI companies are paying billions for it. They wouldn't pay anything for public data.
    - woctordho 1mo ago
      There are 'transfer stations' and that's how exactly I use GPT and Claude in China. OpenAI and Anthropic do not sell in China, so we use their AI with a much lower price like 1% of the official API price. The largest transfer stations have TBs of traffic every day, and the traffic is eventually possessed by the open source community. Subscription engineering is a deep field. Neither OpenAI nor Anthropic have any technical advantage in this field.