6 ms·

Just reading the headline, I say good. A) These models are trained by ignoring IP. It is hypocritical and absurd to then try to assert IP over them. And I am f

Just reading the headline, I say good. A) These models are trained by ignoring IP. It is hypocritical and absurd to then try to assert IP over them. And I am for the destruction of IP on all ends. B) What this essentially means is that the Chinese labs are taking the work of these mega corporations into making it freely accessible to other labs and businesses, to serve inference, fine tune, and host privately on prem. That's clearly a good thing for competition in the market as a whole. C) I don't see why we should have to duplicate the massive energy and infrastructure investment of building foundation models over and over forever just because we want to preserve the IP rights of a few companies. That seems a shame and it seems better to me for everything to learn from everything else for the whole ecosystem to get better by topping each other and building off each other; that's also why publishing research into the architecture and training of these models is so much better than what the proprietary labs do (keeping everything a secret), although tbf Anthropic's interpretability research is cool. D) these Chinese models give 90% of the performance of frontier proprietary models at a 10th or 20th of the cost. That seems like a win for everyone. Not to mention the fact that this distilling also allows them to make much smaller local models that everyone can run. This is a win for actual democratization, decentralization, and accessibility for the little guy.

spudlyo 8mo ago
> And I am for the destruction of IP on all ends. While I'm not unsympathetic to the plight of creatives, and their need to eat, I feel like the pendulum has swung so far to the interests of the copyright holders and away from the needs of the public that the bargain is no longer one I support. To the extent that AI is helping to expose the absurdity of this system, I'm all for it. I don't think "burn it all down" is the answer, but I'd love to see the pendulum swing back our way.
- paxys 8mo ago
  Because copyright laws rarely serve small independent creatives, but rather corporations like Disney that are in the business of hoarding and monetizing culture.
  - logicprog 8mo ago
    Yeah, I would argue that, just systemically, intellectual property laws can't really do anything but overwhelmingly serve the interests of the wealthy and mega corporations. I also think they're ethically wrong and run counter to the kind of artistic and information culture that I would prefer, but those are arguments more people are likely to disagree on.
    - spudlyo 8mo ago
      I think most people would argue that dismantling intellectual property would mean the end of all new creative endeavors, as if humanity is only driven to create art for practical reasons. Schopenhauer, on the other hand, would argue that true art must serve absolutely no practical or utilitarian purpose, and that pecuniary concerns only corrupt artistic and intellectual labors leading to mediocrity and dishonesty.
      - paxys 8mo ago
        Copyright laws as we know them came into being sometime in the 18th century. The earliest recorded works of art produced by humans are from 40,000-45,000 BC. So it's hard to take the "we'll never have creative output without strict copyright!!" extremism seriously.
        rerdavies 8mo ago
        The original printing of the King James Bible had copyright protection (and still does, in the UK). 1611.
      - logicprog 8mo ago
        > Schopenhauer, on the other hand, would argue that true art must serve absolutely no practical or utilitarian purpose, and that pecuniary concerns only corrupt artistic and intellectual labors leading to mediocrity and dishonesty. As, similarly, would Bataille, one of the philosophers I'm interested in!
  - 8mo ago
jsheard 8mo ago
They're trying to kidnap what Anthropic has rightfully stolen! Jokes and complete lack of sympathy aside, it does complicate the narrative that these small labs are always on the heels of the big labs for pennies on the dollar, if they rely on distilling the big labs models. That means there still has to be big bucks coming from somewhere.
- Imustaskforhelp 8mo ago
  I don't see Z.ai (GLM 5) in the list though. I consider Qwen/Kimi to have a close relationship so I might not be sure but Qwen might be using Kimi data (I have written another comment in more depth) I still prefer kimi fwiw. It's one of the best models I have witnessed open source and when I tried GLM 5, it really was lacklustre for me on its launchday but I will have to see it for myself now comparing the two maybe as I do see GLM 5 do some good things in benchmarks but we all know how benchmarks should be less trusted. I still think that there is still some hope in chinese models even after this ie. they aren't completely dependent on the large models seeing GLM 5. I am seeing an accusation of GLM 5 doing Distillation[0] but I am not seeing any hard evidence of it. [0]: https://mtsoln.com/id/blog/wawasan-720/the-temu-fication-of-ai-the-truth-behind-glm-5-and-the-death-of-the-mid-market-5591 https://mtsoln.com/id/blog/wawasan-720/the-temu-fication-of-...
- nashadelic 8mo ago
  something about Bill Gates telling Steve Jobs he didn't steal from him, he stole from their rich neighbour Parc
  - ergocoder 8mo ago
    The detail was different where 24,000 accounts weren't created to pull the data..
    - nashadelic 8mo ago
      And they had hundreds of engineers reverse engineer the GUI smh
ashertrockman 8mo ago
A) The "IP" they're concerned about isn't the same IP you speak of. It's the investment in RL training / GPU hours that it takes to go from a base model to a usable frontier model. B) I don't think the story is so clean. The distilled models often have regressions in important areas like safety and security (see, for example, NIST's evaluation of DeepSeek models). This might be why we don't see larger companies releasing their own tiny reasoning models so much. And copying isn't exactly healthy competition. Of course, I do find it useful as a researcher to experiment with small reasoning models -- but I do worry that the findings don't generalize well beyond that setting. C) Maybe because we want lots of different perspectives on building models, lots of independent innovation. I think it's bad if every model is downstream of a couple "frontier" models. It's an issue of monoculture, like in cybersecurity more generally. D) Is it really 90% of the performance, or are they just extremely targeted to benchmarks? I'd be cautious about running said local models for, e.g., my agent with access to the open web.
- logicprog 8mo ago
  Fair points, and worth responding to for a more nuanced discussion! I hope you take these responses in that light :) A) Well, sure, yes, it's different specific IP being distilled on versus what was trained on. But I don't see why the same principles should not apply to both. If companies ignore IP when training on material, then it should be okay for other companies to ignore IP when distilling on material — either IP is a thing we care about or it isn't. (I don't). B) I'm really not sure how seriously I take the worries about safety and security RLing models. You can RLA amodel to refuse to hack something or make a bio weapon or whatever as much as you want, but ultimately, for one thing, the model won't be capable of helping a person who has no idea what they're doing. Do serious harm anyway. And for another thing, the internet already exists for finding information on that stuff. And finally, people are always going to build the jailbreak models anyway. I guess the only safety related concern I have with models is sychophancy, and from what I've seen, there's no clear trend where closed frontier models are less sychophantic than open source ones. In fact, quite the opposite, at least in the sense that the Kimi models are significantly less psychophantic than everyone else. C) This is a pretty fair point. I definitely think that having more base frontier models in the world, trained separately based on independent innovations, would be a good thing. I'm definitely in favor of having more perspectives. But it seems to me that there is not really much chance for diversity in perspectives when it comes to training a base frontier model anyway because they're all already using the maximum amount of information available. So that set is going to be basically identical. And as for distilling the RL behaviors and so on of the models, this distillation process is still just a part of what the Chinese labs do — they've also all got their own extensive pre-training and RL systems, and especially RL with different focuses and model personalities, and so on. They've also got diverse architectures and I suspect, in fact, very different architectures from what's going on under the hood from the big frontier labs, considering, for instance, we're seeing DSA and other hybrid attention systems make their way into the Chinese model mainstream and their stuff like high variation in size, and sparsity, and so on. D) I find that for basically all the tasks that I perform, the open models, especially since K2T and now K2.5, are more than sufficient, and I'd say the kind of agentic coding, research, and writing review I do is both very broad and pretty representative. So I'd say that for 90% of tasks that you would use an AI for, the difference between the large frontier models and the best open weight models is indistinguishable just because they've saturated them, and so they're 90% equivalent even if they're not within 10% in terms of the capabilities on the very hardest tasks.
impulser_ 8mo ago
It's greed, now that they have all the data and infrastructure they are pulling up the ladder. Why do you think not a single one of these labs have released an open source models distilled on their own SOTA model? They are all preaching they want to provide AI to everyone, wouldn't this be the best way to do this? Use your SOTA model to produce a lesser but open source model?
hereme888 8mo ago
I would not hire someone with your sense of judgment and moral compass.
- logicprog 8mo ago
  Well, it's a good thing I'm not looking for a job from you then
juleiie 8mo ago
Well if you think about AI as another manhattan project this is clearly not that good. Or maybe it is? Maybe it will zero sum into some kind of mutually assured AI superiority destruction. Maybe anything else would be too tempting for the nation with access to super intelligence to not instantly submit the opponent that doesn’t possess an equivalent? This is a bit overdramatizing. Let’s just think of it as strategic, tactical and planning superalgorithms. A nation without access to them becomes between severely disadvantaged and completely defenseless. Is it in the interest of the world to preserve the exiting status quo and balance of power? Or let’s say we make china severely behind this technology. Now they cannot defend themselves from a potential USA swift nuclear strike so precise that a chance of retaliation is zero. Is this a net benefit to the world.
lofaszvanitt 8mo ago
Free Open AI for everyone. Well, coding was mostly trained on FOSS projects.

spudlyo 8mo ago

> And I am for the destruction of IP on all ends. While I'm not unsympathetic to the plight of creatives, and their need to eat, I feel like the pendulum has swung so far to the interests of the copyright holders and away from the needs of the public that the bargain is no longer one I support. To the extent that AI is helping to expose the absurdity of this system, I'm all for it. I don't think "burn it all down" is the answer, but I'd love to see the pendulum swing back our way.

paxys 8mo ago
Because copyright laws rarely serve small independent creatives, but rather corporations like Disney that are in the business of hoarding and monetizing culture.
- logicprog 8mo ago
  Yeah, I would argue that, just systemically, intellectual property laws can't really do anything but overwhelmingly serve the interests of the wealthy and mega corporations. I also think they're ethically wrong and run counter to the kind of artistic and information culture that I would prefer, but those are arguments more people are likely to disagree on.
  - spudlyo 8mo ago
    I think most people would argue that dismantling intellectual property would mean the end of all new creative endeavors, as if humanity is only driven to create art for practical reasons. Schopenhauer, on the other hand, would argue that true art must serve absolutely no practical or utilitarian purpose, and that pecuniary concerns only corrupt artistic and intellectual labors leading to mediocrity and dishonesty.
    - paxys 8mo ago
      Copyright laws as we know them came into being sometime in the 18th century. The earliest recorded works of art produced by humans are from 40,000-45,000 BC. So it's hard to take the "we'll never have creative output without strict copyright!!" extremism seriously.
      - rerdavies 8mo ago
        The original printing of the King James Bible had copyright protection (and still does, in the UK). 1611.
    - logicprog 8mo ago
      > Schopenhauer, on the other hand, would argue that true art must serve absolutely no practical or utilitarian purpose, and that pecuniary concerns only corrupt artistic and intellectual labors leading to mediocrity and dishonesty. As, similarly, would Bataille, one of the philosophers I'm interested in!
- 8mo ago

jsheard 8mo ago

They're trying to kidnap what Anthropic has rightfully stolen! Jokes and complete lack of sympathy aside, it does complicate the narrative that these small labs are always on the heels of the big labs for pennies on the dollar, if they rely on distilling the big labs models. That means there still has to be big bucks coming from somewhere.

Imustaskforhelp 8mo ago
I don't see Z.ai (GLM 5) in the list though. I consider Qwen/Kimi to have a close relationship so I might not be sure but Qwen might be using Kimi data (I have written another comment in more depth) I still prefer kimi fwiw. It's one of the best models I have witnessed open source and when I tried GLM 5, it really was lacklustre for me on its launchday but I will have to see it for myself now comparing the two maybe as I do see GLM 5 do some good things in benchmarks but we all know how benchmarks should be less trusted. I still think that there is still some hope in chinese models even after this ie. they aren't completely dependent on the large models seeing GLM 5. I am seeing an accusation of GLM 5 doing Distillation[0] but I am not seeing any hard evidence of it. [0]: https://mtsoln.com/id/blog/wawasan-720/the-temu-fication-of-ai-the-truth-behind-glm-5-and-the-death-of-the-mid-market-5591 https://mtsoln.com/id/blog/wawasan-720/the-temu-fication-of-...
nashadelic 8mo ago
something about Bill Gates telling Steve Jobs he didn't steal from him, he stole from their rich neighbour Parc
- ergocoder 8mo ago
  The detail was different where 24,000 accounts weren't created to pull the data..
  - nashadelic 8mo ago
    And they had hundreds of engineers reverse engineer the GUI smh

ashertrockman 8mo ago

A) The "IP" they're concerned about isn't the same IP you speak of. It's the investment in RL training / GPU hours that it takes to go from a base model to a usable frontier model. B) I don't think the story is so clean. The distilled models often have regressions in important areas like safety and security (see, for example, NIST's evaluation of DeepSeek models). This might be why we don't see larger companies releasing their own tiny reasoning models so much. And copying isn't exactly healthy competition. Of course, I do find it useful as a researcher to experiment with small reasoning models -- but I do worry that the findings don't generalize well beyond that setting. C) Maybe because we want lots of different perspectives on building models, lots of independent innovation. I think it's bad if every model is downstream of a couple "frontier" models. It's an issue of monoculture, like in cybersecurity more generally. D) Is it really 90% of the performance, or are they just extremely targeted to benchmarks? I'd be cautious about running said local models for, e.g., my agent with access to the open web.

logicprog 8mo ago
Fair points, and worth responding to for a more nuanced discussion! I hope you take these responses in that light :) A) Well, sure, yes, it's different specific IP being distilled on versus what was trained on. But I don't see why the same principles should not apply to both. If companies ignore IP when training on material, then it should be okay for other companies to ignore IP when distilling on material — either IP is a thing we care about or it isn't. (I don't). B) I'm really not sure how seriously I take the worries about safety and security RLing models. You can RLA amodel to refuse to hack something or make a bio weapon or whatever as much as you want, but ultimately, for one thing, the model won't be capable of helping a person who has no idea what they're doing. Do serious harm anyway. And for another thing, the internet already exists for finding information on that stuff. And finally, people are always going to build the jailbreak models anyway. I guess the only safety related concern I have with models is sychophancy, and from what I've seen, there's no clear trend where closed frontier models are less sychophantic than open source ones. In fact, quite the opposite, at least in the sense that the Kimi models are significantly less psychophantic than everyone else. C) This is a pretty fair point. I definitely think that having more base frontier models in the world, trained separately based on independent innovations, would be a good thing. I'm definitely in favor of having more perspectives. But it seems to me that there is not really much chance for diversity in perspectives when it comes to training a base frontier model anyway because they're all already using the maximum amount of information available. So that set is going to be basically identical. And as for distilling the RL behaviors and so on of the models, this distillation process is still just a part of what the Chinese labs do — they've also all got their own extensive pre-training and RL systems, and especially RL with different focuses and model personalities, and so on. They've also got diverse architectures and I suspect, in fact, very different architectures from what's going on under the hood from the big frontier labs, considering, for instance, we're seeing DSA and other hybrid attention systems make their way into the Chinese model mainstream and their stuff like high variation in size, and sparsity, and so on. D) I find that for basically all the tasks that I perform, the open models, especially since K2T and now K2.5, are more than sufficient, and I'd say the kind of agentic coding, research, and writing review I do is both very broad and pretty representative. So I'd say that for 90% of tasks that you would use an AI for, the difference between the large frontier models and the best open weight models is indistinguishable just because they've saturated them, and so they're 90% equivalent even if they're not within 10% in terms of the capabilities on the very hardest tasks.

impulser_ 8mo ago

It's greed, now that they have all the data and infrastructure they are pulling up the ladder. Why do you think not a single one of these labs have released an open source models distilled on their own SOTA model? They are all preaching they want to provide AI to everyone, wouldn't this be the best way to do this? Use your SOTA model to produce a lesser but open source model?

hereme888 8mo ago

I would not hire someone with your sense of judgment and moral compass.

logicprog 8mo ago
Well, it's a good thing I'm not looking for a job from you then

juleiie 8mo ago

Well if you think about AI as another manhattan project this is clearly not that good. Or maybe it is? Maybe it will zero sum into some kind of mutually assured AI superiority destruction. Maybe anything else would be too tempting for the nation with access to super intelligence to not instantly submit the opponent that doesn’t possess an equivalent? This is a bit overdramatizing. Let’s just think of it as strategic, tactical and planning superalgorithms. A nation without access to them becomes between severely disadvantaged and completely defenseless. Is it in the interest of the world to preserve the exiting status quo and balance of power? Or let’s say we make china severely behind this technology. Now they cannot defend themselves from a potential USA swift nuclear strike so precise that a chance of retaliation is zero. Is this a net benefit to the world.

lofaszvanitt 8mo ago

Free Open AI for everyone. Well, coding was mostly trained on FOSS projects.