8 ms·
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less sa
by jaggederest 3mo ago
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.
- dylan604 3mo agoThis is why I don't give a shit that this is happening. It's actually kind of funny to me.
- azinman2 3mo agoUnless you’re from mainland China, you should.
- bilbo0s 3mo agoWhy? Serious question. I'm from the US, and I think it's hilarious.
- breppp 3mo agoOne example is that a totalitarian government known for erasing historical record of its crimes will control an arbitrator of truth.
- dylan604 3mo agoso what? this has nothing to do with them taking from those that took before them. china censoring information they do not like is nothing new and seems irrelevant to this conversation
- breppp 3mo agoFair enough you don't care, some people might want to know about the Uyghur happy camps, mass organ harvesting and such. In a world where information sources are only going to dwindle, it is not in anyone's interest to empower actors that will use these to manipulate perceptions
- dylan604 3mo agoThe people that care about that won't be using CCP products/services now will they?
- deleted 3mo ago[deleted]
- 4bpp 3mo agoAnd some people might want to know about [insert your favourite beyond-the-pale-in-the-US topic here, we're on a US forum after all]. I think it would be great if those people could turn to Chinese models, while anyone wanting to know about Uyghur camps can ask the US ones.
- customguy 3mo ago> beyond-the-pale-in-the-US topic Can you name one? It's an honest question, first of I'm not American, but also the stuff I do come up with (asking an LLM how to blow up a school or whatever) would also be handled similarly in China, so those would be a wash, and I can't think of any that aren't.
- antiamerican634 3mo agoFrom an outsiders point of view: - The Trail of Tears - The Tuskegee syphilis study - Use of Agent Orange in the Vietnam War - Open Air biological warfare testts in civilians eg. in 1950 San Francisco - The only use of nuclear weapons against civilians? - Coca Cola and "american culture" - Neoliberalist economy - Spreading blame for their sins to other "white" nations plus one: The text input method to HN comments :(
- deleted 3mo ago[deleted]
- queenkjuul 3mo agoThe US already decides what models get released by US companies...
- analognoise 3mo agoThe USA?
- sph 3mo agoA KGB spy and a CIA agent meet up in a bar for a friendly drink "I have to admit, I'm always so impressed by Soviet propaganda. You really know how to get people worked up," the CIA agent says. "Thank you," the KGB says. "We do our best but truly, it's nothing compared to American propaganda. Your people believe everything your state media tells them." The CIA agent drops his drink in shock and disgust. "Thank you friend, but you must be confused... There's no propaganda in America."
- breppp 3mo agoNo, there might be lesser known periods in USA history but none that are censored. Also generally the USA had never done crimes in the scale of the CCP (around 30+ million dead)
- azinman2 3mo agoFor a number of reasons. First, whether we like it or not (generally not), a fuck ton of money has been invested in US AI companies, data centers, RLHF datasets amongst other datasets, etc. If that were to go to 0 that’d be quite disastrous. Alternatively, if it goes well, it’s great for the US (and to a lesser extent allies) economy and global standing. Relatedly, tech has been a huge power house for the US economy for decades now. If the main driver of growth goes to China, what replaces it? Along with all the potential tax money, foreign investment, etc? Next, patriotism / nationalism. This is very much a zero-sum game that defines who owns the future. Would you rather your country win or lose this? Lose this and you start losing talent, money, global standing, etc. That furthers a cascading effect that’s very negative. It also will likely create social strife with the knock on effects leading to even more bad populist ideas that just further diminish the country and tear society’s fabric apart. None of this is hilarious.
- vinyl7 3mo ago> This is very much a zero-sum game that defines who owns the future. Would you rather your country win or lose this? Lose this and you start losing talent, money, global standing, etc. That furthers a cascading effect that’s very negative. This process is already well underway. The current massive over investment into the AI hype bubble is the final death throes of a failed economy trying to keep its head above water.
- xbmcuser 3mo agoNo you should not care unless you are a shareholder in one of these Ai ponzi companies. For the rest of the world Chinese companies matching and open sourcing llm's will keep 100 or so tech oligarchs taking over all the world economic output for themselves as all the idiot politician are unwilling to tax wealth.
- HDBaseT 3mo agoEveryone is technically (directly or indirectly) a shareholder of one of these AI or AI affiliated companies.
- cowboy_henk 3mo agoWe're also shareholders in all other publicly traded stocks (assuming you're referring to pension schemes etc), which means competition in the AI model market is better than a few winners making everyone paying through their nose for access. Even better, open weights models which cannot be disabled on a whim means the entire world economy can benefit.
- stuaxo 3mo agoThe majority being indirect and having no input to how these companies run if we can't get some strong regulation going.
- VulgarExigency 3mo agoWhy? Should we be held hostage to the whims of these companies and all the investors in the throes of AI psychosis, and let them do whatever the fuck they want, because if we don't then the economy will crash?
- owebmaster 3mo agoWhy so? If I'm from Europe or South America, should I hope Anthropic/Openai win?
- RazorBucksICO 3mo agoYes. They’re flawed, but you don’t want a global police state administered by the PRC.
- owebmaster 3mo agoWho said so? I don't want the current global police state administered by the US government and oligarchies
- SZJX 3mo agoNo matter your perception of PRC, I don't see how competition around open-weight models has much to do with a "global police state".
- wasfgwp 3mo agoThe better question is why would any rational consumer would want anyone to “win”? Disappearance of competition is the worst outcome for them (I guess being squeezed by a monopolistic company from your pwn country is slightly nicer)
- m4rtink 3mo agoI think it is only rational to want them to fail & to fail hard, to make an example. To avoid future situations where money is invested on hype only with no regards to what societal disruption it causes.
- azinman2 3mo agoAnswered in a sister thread. Are you from Europe or South America? Or just asking on their behalf? They are allies of the US, which includes economic spheres of influence. The premise of the petrodollar is that allies do better by being a part of it than not, which has been true for many, many decades now. China’s rise is providing an alternative for the first time in 70+ years, but so far it’s very unclear if these benefits will truly extend to allies of China or not. For example, China sends their own laborers when building infrastructure in Africa. Sure there’s new infrastructure but also debt to CCP without any benefits of knowledge transfer or local employment.
- NuclearPM 3mo agoWhy?
- remus 3mo agoWhile I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point was just that training an LLM takes more resources and expertise than distilling from an existing LLM so I don't think the equivalence between training and distilling is entirely justified.
- ryandvm 3mo agoI don't know man. This reads like "yeah we stole your grain, but making bread is hard."
- Muromec 3mo agoIt sure is, but it doesn't matter. Whatever position that generates more economic activity is declared legal using some nonsense retconned logic "because we said so".
- darod 3mo agoYou can argue that reverse engineering anything is as hard if not harder than engineering something. I can’t imagine distillation is any different.
- some_random 3mo agoDistillation is objectively easier than training a model from scratch, that's why all these Chinese labs are doing it.
- recursive 3mo agoTraining a model is objectively easier than generating the sum total of human creative output prior to 2020. That's why the big labs are doing it. What's the difference here?
- petilon 3mo agoI disagree that LLM models are the product of enormous quantities of copyright infringement. The recent announcement that AI-assisted research produced a counterexample to the Jacobian conjecture--a long-standing open problem in algebraic geometry--shows the original value AI can create. The result was not copied from a textbook; it emerged from AI learning from existing material, much as a human does, and then applying that knowledge in a new way. If that's a violation of copyright, then a human doing the exact same thing would be a copyright violation too. But it isn't.
- ilovecake1984 3mo agoThe didn’t pay for the books. It’s massive copyright infringement. The human buys the books.
- petilon 3mo agoDid they borrow the book? If I learn from a borrowed book is that copyright infringement?
- foxglacier 3mo agoIt's worth keeping in mind the purpose of copyright. It's a pragmatic tool to encourage investment in creative work for the benefit of everybody/consumers. We may be entering a time where there's less need to incentivize people to write books. At least not non-fiction books which are simply a collection of existing knowledge presented in an a way that's suitable for human readers. A lot of the value those authors provided can now be done by AI. Yes, the AI trained on their work, but now that it's here, we don't need new non-fiction authors quite as much as we used to. I wouldn't want to live in a world where technology or general people's wellbeing was held back by obsolete laws that ended up lingering on just to protect undeserving special people at the expense of the rest of society. Remember guilds for tradesmen? They were also a monopoly given by the government to special people. They had their purpose but nowadays we have different ways to keep tradesmen working effectively like license requirements and insurance. Just to be clear, I think we do still need copyright, but that we might be in a transition period where it has to be redesigned to adapt to AI.
- aunty_helen 3mo agoNo, they settled that yesterday, so all is forgotten. Press releases were queued for today so just in the nick of time.
- RazorBucksICO 3mo agoSo Moonshot is next for a lawsuit or do lawyers only care about Anthropic?
- anon373839 3mo agoBetween the two of them, only one firm is trying to pull up the ladder so that no one else can have what it took.
- janderson215 3mo agoI think that was specifically on the piracy aspect.
- bluegatty 3mo agoNo - distillation is not data inputs. Raw materials vs. Value add. They are different things, like ore and metal. Distillation is a new thing we need to understand, it's probably closer to IP than not.
- tikhonj 3mo agoThe "data inputs" were also, very much, somebody's "value added" IP. We're talking about things like text people wrote, not some kind of raw data floating out in the ether.
- bluegatty 3mo agoDid I say there was no value add in the inputs? Ore has value, a different kind of value than the output of the refinery.
- sho_hn 3mo agoAre you suggesting data input is further from IP than distillation? That would stun me, but it's a little hard to read.
- mkehrt 3mo agoDo you think writing books (and Wikipedia articles, and stack overflow articles, and github repos, and, and, and, and ...) is not a value add?? What terrible claim.
- dijksterhuis 3mo agodistillation has been around for 12 years. it's not new in terms of ML techniques. https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531 although i doubt there has been a legal case over it yet in the context of the legality of stealing shit but IANAL.
- bluegatty 3mo agoYes, I get that, but it's only now the issues are coming into the commons in a way that industry / society needs the regulatory clarity. It's completey insane that we still don't know how Open Source would work, that the laws are vague and we're still technically waiting for the courts to decide on cases. The government should a) legislate and b) create test cases and run them through the courts so that we can have clarity.
- 8note 3mo agofair use, both in the original training and in distillation, or rather, anthropic has no copyright at all over the output tokens