14 ms·
OpenAI suspends ByteDance's account after it used GPT to train its own AI model
- sgu999 3y agoSilly, they should have hooked it to a different model that gives harmful training data instead :D
- muragekibicho 3y agoI wonder how this affects US-China relations
- vertis 3y agoI have occasionally wondered how easy it would be to bootstrap your own models using models from the big players. I remember first thinking about this when IBM Watson was the newest hottest thing on the block. Then OpenAI came along and it seemed like it wouldn't be necessary any more because they were releasing their models anyway, except for when they suddenly decided that they wouldn't any more. But others have been carrying that torch since (Llama and many others). Still it seems an interesting way of enhancing a model.
- j-a-a-p 3y ago> I have occasionally wondered how easy it would be to bootstrap your own models using models from the big players. I guess OpenAI did the same. If you read their API terms one of the first things they prohibit is to use it to train a competing model. Maybe they do the same tricks internally and know how powerful it can be?
- londons_explore 3y ago> While ByteDance’s use of our API was minimal Aka... some dev fired off a handful of test queries and never used the account again, so OpenAI decided to suspend it to look good in the US press.
- repeekad 3y agoAlternatively, they caught the action promptly and kept use to a minimum Also, what does “minimal” even mean? I’m sure they monitor accounts that max out their API request limits, or even just request programmatically (i.e. request patterns that don’t match a natural human use pattern, like slowing down during a time zones lunch hours). Maybe this was a couple days worth of traffic
- vineyardmike 3y agoOr the media reported on their usage, and the “minimal” clause is to reassure investors that their model is still protected and special.
- deleted 3y ago[deleted]
- zeruh 3y agoWouldn't it result in overfitting?
- kevsim 3y agoI would also think it'd be an incredibly expensive way to train a model.
- ReptileMan 3y agoDepends. I wonder what is the minimum reasonable amount of different tokens needed to lift up the weights.
- Jensson 3y agoYou store the output from ChatGPT, you don't run it again every time you do a training step. Generating millions of examples to add to your own training wont cost much at all relatively.
- triyambakam 3y agoThe details on how exactly they may have used it to train their model is vague. I believe transfer learning or knowledge distillation are valid techniques based on the inference from other models.
- seydor 3y agoConsidering the extent to which youtube is blatantly encouraging people to copy their videos, and no doubt that openai haw user tiktok content for their models, this is like slapping a child.
- hipadev23 3y agoNew Pied Piper
- bakugo 3y agoThey also have their own web scraper called ByteSpider that scrapes websites with lots of text very aggressively and ignores robots.txt. I've had to block it by useragent on one of my sites.
- tech2 3y agoI don't think it ignores robots.txt, I think it just doesn't have a very good parser and you need to give them their own user-agent block. I had a similar level of frustration. https://www.feitsui.com/en/article/32 https://www.feitsui.com/en/article/32
- codetrotter 3y agoAfter all, if they wanted to completely ignore the wishes of the website owners they probably would not announce their spider as such in the user agent. They’d just pretend to be a web browser.
- dotancohen 3y agoIt is trivial to detect a spider from human traffic based on requests alone. Lying about the UA would just be bad press for them.
- kccqzy 3y agoIf it's really trivial as you say, Google's reCAPTCHA and similar products like hCAPTCHA would instantly have no reason to exist.
- jpc0 3y agoBot intentionally trying to look human =! Spider A spider will generally have a pretty predictable route through a web site.
- dotancohen 3y agoThe various CAPTCHA implementations are primarily designed to prevent bot submissions, not spiders.
- Al-Khwarizmi 3y agoI still don't understand how they can keep a straight face claiming that training on all human-written material (copyrighted or not) that can be found on the Internet is perfectly fine, but training on ChatGPT output is not (or in other words, that human writers cannot have a choice on whether their output is used, but bot owners can).
- Espressosaurus 3y agoEasily. Pulling up the ladder behind you is classic self-interested, antisocial behavior. It doesn't have to be internally consistent, it just has to make them money.
- falcor84 3y agoIndeed. Setting up "Barriers to entry" is one of the fundamental things taught on an MBA.
- jomohke 3y agoTo be fair, this isn't a ladder OpenAI used themselves, right? (The seemingly-extreme shortcut of training from an external LLM model) Their ladder (using public data, and hiring humans to classify to taste) is still available I believe.
- matwood 3y ago> Their ladder (using public data, and hiring humans to classify) is still available I believe. Not really. Once chatGPT came out, many sites changed their terms and/or significantly increased their API access costs to prevent/limit/make cost prohibitive future scraping.
- jomohke 3y agoGood point. Though that affects OpenAI too for new data. I had assumed most of their web content was from Common Crawl, and the older pre-ChatGPT Common Crawl datasets used would still be available. But it looks like Twitter, for one, was not in Common Crawl.
- deleted 3y ago[deleted]
- muzani 3y agoFrom OpenAI's terms of use: https://openai.com/policies/terms-of-use https://openai.com/policies/terms-of-use ====== What You Cannot Do. You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: Use our Services in a way that infringes, misappropriates or violates anyone’s rights. Modify, copy, lease, sell or distribute any of our Services. Attempt to or assist anyone to reverse engineer, decompile or discover the source code or underlying components of our Services, including our models, algorithms, or systems (except to the extent this restriction is prohibited by applicable law). Automatically or programmatically extract data or Output (defined below). Represent that Output was human-generated when it was not. Interfere with or disrupt our Services, including circumvent any rate limits or restrictions or bypass any protective measures or safety mitigations we put on our Services. Use Output to develop models that compete with OpenAI. ======
- 3cats-in-a-coat 3y agoIt makes sense it'd say that. Of course, GPT is built on everyone's output itself. So throwing around statements like "we suspended ByteDance to ensure GPT is used for good" are hypocritical at best. They're not the pope, they have no monopoly on good.
- deleted 3y ago[deleted]
- lelanthran 3y agoIt's going to be stupid for openai to argue that those terms are binding when they've already argued in court that those terms are nonbinding when they scraped other people's dat.
- shikon7 3y ago> Automatically or programmatically extract data or Output (defined below). What does that mean? Isn’t that just the definition of API use?
- 3cats-in-a-coat 3y agoSo. What about Grok?
- throwawayscrape 3y agoWhat’s a competing model? Only LLM’s or images too? Kinda vague. I hope only big spenders get pinged because I’ve been using the vision api to help auto-annotate some training data.
- Vektorceraptor 3y agoIt's probably a technical term for models "we don't like".
- gmerc 3y agoHave they suspended Elon for the same? Stanford for Alpaca?
- Vektorceraptor 3y agoEvery time someone uses 'the good' to justify their cause, I know that they are lying and are not who they are generally thought to be. And I'm furious and tired that it's always the 'intelligent' people who are guilty of this. I find this hypocrisy very difficult to deal with.
- leeeeeep101 3y ago[dead]
- w10-1 3y agoAside from copyright, this raises anti-trust issues. Courts generally uphold provisions against reverse-engineering (as protecting internal, proprietary knowledge) but are more welcoming to copying interfaces (as encouraging market substitutes). So one question would be whether OpenAI can restrict use of the output of their tool in this manner, since the output itself is manifestly open (to the customer). That seems novel. The only analogy I know of is database licensing that prevents customers from publishing comparisons, which seems anti-competitive. Anti-trust policy is motivated mainly in mature markets, where one player has fairly (by hypothesis) grown to dominate. The law and courts apply special scrutiny to identify ordinarily-acceptable market practices that extend the market power of the dominant player. But is it the same analysis in a growing market? It seems like even if OpenAI (or especially, OpenAI+Microsoft) is dominant, if the market is growing quickly, the concern might be relaxed since the dominance is uncertain. Conversely, if the market is particularly susceptible to capture, early leaders might warrant heightened scrutiny. But aside from monopoly's first-order effect on reducing competition, the second-order effect is to reduce investments in competitors, which has anti-competitive effects. That concern could be highest in the early stages of a market. Are there any good lawyer blogs on point?
- avsteele 3y agoThe terms against using the output to develop a competing product seem the same as reverse engineering to me. Competitors can't get OpenAI's model weights but use its outputs to produce a functionally similar model. It's like if you had a competitor's engine and couldn't open it up but could still see the outputs: torque, rpm, ..., and could control the inputs: fuel intake, air mixture, etc... Then you make an engine by inferring back from these measurements. Wouldn't that be reverse engineering?
- matei88 3y agoNo, it will be an engine that produces the same output if given the same input. It's like saying Google reverse engineered Yahoo and built a search engine
- deleted 3y ago
- wangii 3y agoI thought OpenAI's mission was to ensure technology to benefit all humanity, not to fall into the hands of big governments and big business.
- nicce 3y agoMaking better AIs with it should benefit the humanity. They should change name already.
- deleted 3y ago[deleted]
- smashah 3y agoWhen is OpenAI going to suspend or fire Tal Broda?
- joigreh 3y ago[flagged]
- 3xnl 3y agoSeems everyone is trying to catch up with OpenAI. The other day Grok was spitting out OpenAI terms and conditions
- anon373839 3y agoThis must be for show. If ByteDance or anyone else is sufficiently motivated to distill OpenAI’s models, it can’t be prevented. You can simply pay other people to collect the data for you.
- blueboo 3y ago“Simply”? It would be quite a project to do that at sufficient scale undetected, no?
- anon373839 3y agoHow do you figure? It would look like people from all over the place running API workloads on a wide range of tasks and topics (if the goal is to distill the model generally, that’s what you’d need). So many people are already using GPT-4 to synthesize fine-tuning datasets, I think it would be invisible. However, you might wonder what the goal is. This “API distillation” is good for teaching a pretrained model how to do lots of things. But the end result is always constrained by the quality of the pretrained base model, and API outputs don’t help at all with that part.
- yard2010 3y agoWith the backing state power they don't really need to pay anyone
- hoofhearted 3y agoYou just block their IP’s; Gain intel from internal employees; Have the expensive law firm start sending C&D’s and prepping that law suit. If you tell someone that they can’t use your product in a certain manner, and then they go to extra lengths to circumvent your measures to gain a profit for themselves, then there is going to be big civil and criminal legal problems. State sponsored or not; Bytedance has a bank account and business license here in the U.S.
- renonce 3y agoThey can outsource their API requests to third party “AI startup” who proxy their requests to OpenAI for a small fee
- spoiler 3y agoIsn't it a bit hypocritical of them to use other people's copyrighted work or "output" they weren't given license/permission to use, but deny people the same opportunity with their work/output? I'm genuinely wondering how this is different from them using others work without consent, but IANAL so maybe I'm just confusing mortality and legality
- muzani 3y agoYes, but the original intent was to help other companies create ethical AI models. If they've already turned their back on those core values, a bit more hypocrisy won't stop them.
- fiips 3y agoStated intent and actual intent are usually different, I expect they intended the opposite all along but were just riding the open source ethical AI wave to profit.
- omeze 3y agoIve talked to a few people formerly at OpenAI a few years ago - the deviation from the original mission was a “boiling frog” process that is best demarcated when the Anthropic founders left. The core team doing the actual science and engineering work did legitimately believe in the open research mission. But when funding was hard to find, things kind of broke.
- muzani 3y agoI strongly doubt it. Sam was pretty damn rich before OpenAI. He's written a lot of stuff about AI and how he worried someone would do it. The original plan was roughly build the basis for ethical AI, have someone else build on top of it, which somehow pivoted into everyone using their API. If they just wanted money, they could have cut out a lot of the ethical filters and choke out competitors. Google didn't stop NSFW. Twitter, Reddit, Tumblr, etc didn't. AI is bound to be used for NSFW among other things, but they've set the standards to make it ethical. I think eventually they did let loose to try to keep ahead of competitors. This probably pissed off the board and led to the drama recently? Just speculation. Because the new models are nowhere near as anal as the initial release.
- croes 3y agoBlocking competitors, cooperating with Germany's worst news paper, MS has a seat on OpenAI's board. Altman's return is really working out
- andai 3y ago>cooperate with Germany's worst news paper >As part of the deal, ChatGPT users will receive summaries of news stories from Axel Springer’s brands, including Politico, Business Insider, Bild and Welt, with attribution and links to the original sources of reporting, the companies said Wednesday. The agreement will allow OpenAI’s models to take advantage of the publisher’s higher quality and more current information in its chatbots’ answers. https://www.cnn.com/2023/12/13/tech/open-ai-axel-springer-chatgpt/index.html https://www.cnn.com/2023/12/13/tech/open-ai-axel-springer-ch...
- ekianjo 3y ago> publisher’s higher quality Lol is this a joke? Those are all tabloids.
- oezi 3y agoWorse, it is the Fox News of Germany. Actively trying to create dissent and undermining democracy.
- naiv 3y agoYou forgot the /s on your post. I hope at least.
- deleted 3y ago[deleted]
- croes 3y agoNope, he didn't, Axel-Springer regularly campaigns against people they don't like, including private people. Let me quote one of Germany's best cabaret artists, Volker Piepers: "Bild-Zeitung..... this filthy newspaper that is so disgusting that you insult dead fish if you wrap it in it!" There is a reason Heinrich Böll wrote a book about them: https://en.m.wikipedia.org/wiki/The_Lost_Honour_of_Katharina_Blum https://en.m.wikipedia.org/wiki/The_Lost_Honour_of_Katharina...
- mirekrusin 3y agoThey should fire CEO and become open or rename itself to ClosedAI. It's like if Monsanto was called Organic Cooperative.
- Alifatisk 3y agoI usually write ClosedAi (OpenAi) whenever I mention them, just to clarify.
- dimask 3y agoI use this firefox extension that converts `OpenAI` to `"Open"AI` as part of my sanity-preserving toolkit. https://addons.mozilla.org/en-US/firefox/addon/openai-is-not-open/ https://addons.mozilla.org/en-US/firefox/addon/openai-is-not...
- imjonse 3y agoBTW, Monsanto got rid of their infamous name by being bought and rebranded by Bayer.
- probably_wrong 3y agoI heard it told as "it's not OpenAI, it's ClosedAF"
- bawana 3y agoHow can chatGPT be used to do this? Did they ask it to just produce training data?
- Jensson 3y agoYou can use it to produce the fine tuning examples. LLMs are very good at learning styles and answer flows, so you ask ChatGPT a million standard questions and problems and then record the outputs, that data will be very similar to the finetuning data used by OpenAI. Then put that data into your own LLMs fine tuning step and voila, you have an LLM that behaves very similarly to ChatGPT! OpenAI knows this which is why they put that TOS there, they spent a lot of money to create that finetuning data so they don't want to give all of that away for cheap.
- selfhoster11 3y agoThey likely didn't use (just) ChatGPT, but the GPT-3.5/GPT-4 API. Both can be used to create training data quite successfully. This technique has been used in the past to create synthetic post-training (fine tuning) datasets like Orca, Samantha, and so on.
- seydor 3y agoI hope bytedance sues so this can be debated in court
- Zambyte 3y agoWhy?
- elforce002 3y agoI think because openai took advantage of everyone and know doesn't want anyone doing the same thing.
- mkl95 3y ago> While ByteDance’s use of our API was minimal, we have suspended their account while we further investigate I don't like ByteDance at all, but I hope OpenAI are aware both shady and legit companies are making some serious cash using their API. > All API customers must adhere to our usage policies to ensure that our technology is used for good. "For good" sounds like the philosophy of companies like Stripe and Twitch. There always is a grey area with stuff that is good for many people, but other people see as evil.
- kranke155 3y agoOpenAI can use the world's images and text as training data, but as soon as its in their system, it can't be used by competitors. Incredible. The most broken version of copyright imaginable.
- aik 3y agoAm not justifying what OpenAI did, but nobody is stopping ByteDance from doing what OpenAI did. They can also use the world’s information. Instead, since OpenAI has “cleaned” the data, they are trying to use OpenAI’s cleaned dataset. After OpenAI spending endless amounts of money on that, am not surprised they don’t want others to steal their “cleaned” dataset.
- graypegg 3y agoI think GP is pointing out that someone that spends years building a large online gallery of their artwork, only for it to be smushed into a pool of vector mush, has the same reasoning to prevent openAI from using their work as openAI does to prevent competitors from using their artisanaly laundered dataset. Doesn’t matter how much endless amounts of money they spend, they’re going to have to contend with the fact that the value they ship is derived from other’s work. It’s just diluted to the point of it becoming “data” rather than “artworks”.
- FpUser 3y ago>"After OpenAI spending endless amounts of money on that, am not surprised they don’t want others to steal their “cleaned” dataset." And let's say I do not want them to clean up and then use my data for profit.
- endisneigh 3y agoThis makes no sense lol. The information openAI is using is cleaned to begin with
- aik 3y agoRaw text from a website including header text and footers and links and images etc is very dirty stuff.
- Simon_ORourke 3y agoI suspect they'll see a new account setup quite quickly called "NotByteDance" or something similar.
- batch12 3y agoNow someone else just needs to make a company in the middle that will generate the training data for you.
- virgildotcodes 3y agoI wonder if we’ll eventually see an open training set composed entirely of AI generated materials by OAI and others’ models. This API restriction is only an issue until that point, I assume, or is the distilling being done in these situations much more dynamic in nature?
- emsign 3y agoThat's so cliché
- nojvek 3y agoPhi2 trained on synthetic datasets from OpenAPI. Textbook is all you need paper also talks about using output from ChatGPT to build training models. So Bytedance gets account revoked but tons of other models are allowed. China bans US companies, US does same to China. We are both almost the same. Sure they have a more authoritarian government but ours isn’t quite a liberal democracy either. It’s competition all the way down.
- chriskanan 3y agoPhi2 was created by Microsoft, that's probably why that was acceptable.
- 2OEH8eoCRo0 3y agoMy guess is that they don't want their technology to be used to modernize the PRC's military. Just the other day (PDF warning): https://selectcommitteeontheccp.house.gov/sites/evo-subsites/selectcommitteeontheccp.house.gov/files/evo-media-document/reset-prevent-build-scc-report.pdf https://selectcommitteeontheccp.house.gov/sites/evo-subsites... > Pillar II: Stem the Flow of U.S. Capital and Technology Fueling the PRC’s Military Modernization and Human Rights Abuses > U.S. export controls have been slow to adapt to rapid changes in technology and attempts by adversaries to blur the lines between private and public sector entities, particularly the PRC’s strategy of Military-Civil Fusion. > “The military is for civilian use, the civilian is military, and the military and civilian are fused.” In other words, no line exists between civilian and military technological development. U.S. export controls have yet to adapt to this reality
- deleted 3y ago[deleted]
- chollida1 3y agoCan someone walk me through how you'd use what open ai offers to train your own model?
- jumpCastle 3y agoUse model output as training data. For better performance you can get some top log probs and minimize kl divergence.
- TheCaptain4815 3y agoI guess they could figure it out via pattern repetition right? I thought they leave API users alone, especially corporate users… Or are they still monitoring corporate users? Sounds like a major security issue if I’m a CEO. Does Microsoft read and ban people from Microsoft Word/365? Etc
- daniel-thompson 3y agorules for thee, but not for me
- dkjaudyeqooe 3y agoOpenAI: closed in almost every possible way.
- gwern 3y ago> Employees involved are well aware of the implications; I’ve seen conversations on Lark, ByteDance’s internal communication platform for employees, about how to “whitewash” the evidence through “data desensitization.” The misuse is so rampant that Project Seed employees regularly hit their max allowance for API access. If you ever wondered how all these models could seem to catch up to GPT-3.5 so quickly, but then struggled to do noticeably better (much less exceed GPT-4), while not talking about their data or saying they definitely weren't simply training on GPT-3.5 outputs, remember this: they might just be lying.
- crazygringo 3y agoCan someone here explain how an existing model like ChatGPT, that you only have API access to, can be used to train your own model and somehow copy aspects of it? That's entirely non-intuitive to me how that would work. Like are they just asking it questions about every topic under the sun, and then creating training materials out of the answers? How would you even begin to assemble a list of prompts that would cover all of knowledge? And how could you ever distinguish useful outputs from nonsense hallucinations? I feel like I'm missing some key details here that the article and its links don't explain at all.
- Jensson 3y ago> How would you even begin to assemble a list of prompts that would cover all of knowledge? They don't need to, all that knowledge is already in public training sets from scraping the internet. What is harder to get is the answer patterns to ensure good user experience. You want a lot of such answer patterns so the model knows what structure to use for different kinds of questions. That structure isn't to make the result formatted for humans, but contains reasoning paths the LLM takes in order to arrive at reasonable answers. Since an LLMs thinking is the words it writes the structure of how it responds thus corresponds to thinking patterns, and you want the LLM to learn a lot of those, internet data wont contain that but ChatGPT responses will. TLDR: The words an LLM writes is also what it thinks, since it doesn't hide thoughts, so by training on what ChatGPT writes you also train on how it thinks. It isn't the facts you want but the thinking patterns.
- gpjt 3y agoNot an expert but I suspect it would be most useful for the instruction tuning and fine-tuning phases. They would pre-train the model on some huge dump of stuff from the internet like the Pile -- Wikipedia, books, Reddit scrapes and so on, and the result would be something that can predict next words really well but doesn't know it's meant to answer questions. That's the pre-train. The next step is to train that on transcripts of chats so that it specialises in those, and finally on chats that have the balance of helpfulness and harmlessness that you want. Those latter two need significantly less data, but still more than you can cheaply produce with humans, so they could use OpenAI to generate it. But perhaps it's a small enough amount that even if you need an LLM to generate it economically, it's affordable to have humans read through it and throw away anything excessively hallucinated. Or perhaps you can restrict the scope of the conversations to ones where hallucinations are unlikely, and trust that fine tuning on them won't prevent the LLM from digging into its deeper knowledge in use.
- yawz 3y agoHow smart is this anyway? Isn't this what causes the Model Collapse?
- whywhywhywhy 3y agoIf I were Bytedance I’d retaliate by opening the result of what I trained off them and the training data I milked from their model. What they gonna do, double ban you.
- karmakaze 3y agoInteresting to see that capitalism is inherently inefficient. Freely sharing a trained model would be the most efficient and beneficial to progress, but the investment in training was done by private company to turn profits. In this light the wasted energy of cryptocurrency is just another inefficiency.
- mmaunder 3y agoOh this is priceless: “ All API customers must adhere to our usage policies to ensure that our technology is used for good.” Not violating terms. Not abiding by terms of use. But good vs evil. It screeches “we occupy the moral high ground and are the arbiters of what is ethical and what is evil.” The entire brand game plan around “Open”AI is to position themselves as first doing no evil, just like google did in the early days, to enable doing a lot of evil.
- aorth 3y agoRelated anecdote: last web a web server of mine was under very heavy load. The site was flapping and I could barely even SSH to the host. I did some basic analysis of the httpd logs and found that there were 13,000 unique IPs within the past week and 12,000 belonged to one subnet on Amazon EC2. The user agent for all those requests was ByteSpider. Millions of requests over a few days. I'm into freedom of information and what not but come on!!!
- oblib 3y ago"OpenAI suspends ByteDance's account after it used GPT to train its own AI model" That's truly hilarious... It's akin to a child who stole a candy bar in a candy store complaining about their sister stealing that same candy bar from them.
- nwiwatman 3y agoIn my view is: what perspective we wanna use to judge? philosoptical, religious, ethical, legal, or business? 1.philo: who gives a damn it’s meaningless in the cosmic scheme, no meaningful impact on our species! 2. Religious & Ethical : wrong 3.legal : our/internet content isn’t protected by any usage Terms (license) OpenAI’s content is . So if ByteDance violated terms OpenAI is right 4.OpenAI took content unethically and got away with it , ByteDance tried n was caught! There could be other Dimensions of analysis, make ur pick!
- nwiwatman 3y agoIt depends which perspective we want to analyze this: philosophical, religious, ethical, legal, business, *? 1. Philosophically: who gives a damn it’s meaningless at the cosmic level with no impact on our species trajectory 2.Religious n Ethical: Wrong 3.legal: OpenAI took Internet n public content not protected by any licensing agreement and got away with it! ByteDance took OpenAI’s content protected by their L Terms, so those terms are applicable here. 4.Business view: OpenAI user others data and got away with it, ByteDance tried n was caught! 5. Other Dimensions of analyses are possible! Your view = f(your perspective) make ur pick!
- johhns4 3y agoWhat exactly did they do? Just transform data or generate completely new data for training? I've seen plenty of people that have used GPT-4 to help transform data, i.e. using GPT-4 to generate summaries of texts and then using that to train smaller models. Not sure why this wouldn't be allowed, as it's not technically data that is coming from GPT-4. If they don't allow this it just seems like they are trying to prevent people from building smaller cheaper models that will perform better for a specific use case and gobbling up the market for as long as they can.