4 ms·
Training a model is very expensive and creates something no individual rights-holder could. Distilling a model copies this value add and captures it without be
by nonethewiser 3mo ago
Training a model is very expensive and creates something no individual rights-holder could. Distilling a model copies this value add and captures it without bearing the cost that created it.
- mosura 3mo agoWhat of the costs for creating the data that was used to train the model being distilled?
- yandie 3mo agoAt least 1.5B by stealing books, per a recent ruling. I don’t feel sorry for the model companies
- mosura 3mo agoThey only had to pay for storing the books on a server for later possible use. They did not have to pay anything for the training which was declared fair use.
- deleted 3mo ago[deleted]
- deleted 3mo ago[deleted]
- nonethewiser 3mo agoThis actually undermines the argument that distilling is harmless because its founded on the idea that Anthropic did the same thing and didn’t have any repercussions.
- applicative 3mo agoDo you people seriously believe that Moonshot and the Chinese personal-cult-state dont possess and train on the same torrents?
- mosura 3mo agoThey aren’t hypocritically crying foul about it, so no we don’t care if they do.
- deleted 3mo ago[deleted]
- nonethewiser 3mo agoWell of course. But they dont hate Chinese model companies.
- titzer 3mo agoAn incredibly ironic comment.
- jayd16 3mo agoI'm going to hope this was sarcasm and if it is, it's great.
- 8note 3mo agothe outputs of the model have no property protections, and training a model on the outputs of another model does the exact same thing - its expensive and creates new value over what was in the input - a set of documents.
- pandoro 3mo agoCould you give an example of the value that only training a model can create but none of the rights-holder could? I feel like if you got a direct, instant communication channel to any of the rights-holder that created the content in the training set of those models, you'd get more value than what the LLM could ever give you on any specific subject.
- nonethewiser 3mo ago>Could you give an example of the value that only training a model can create but none of the rights-holder could? Yeah. LLMs
- shlewis 3mo agoYes. I bet Moonshot paid for API access as opposed to pirating like Anthropic did.
- deleted 3mo ago[deleted]
- nonethewiser 3mo agoI understand you think Anthropic should have paid for the information it trained the models on. But im talking about all the costs to build a model. Do you think Anthropic didnt spend money to build these models? Did you not know that its actually very expensive?
- shlewis 3mo agoMoonshot pay for access at the price point Anthropic sees fit, potentially more, since they likely had to jump through multiple hoops? It's hard to feel sorry about the breach of their ToS, which ultimately is all Anthropic can argue, when they are constantly being sued by countless IP owners
- rvba 3mo agoA lot of those arguments could be said about writing a book, or a decent forum guide.
- verdverm 3mo agoThere is a lot more one has to be good at to make a fable level model. Distillation won't get you there, it will help refine some at the end