10 ms·
Is Grok Basically Just an OpenAI Wrapper?
- berkes 3y agoWhat's grok? (I dropped off Twitter/X the moment the inmates started running that asylum again. It appears to have to do with X?)
- ronsor 3y agoThis isn't really that newsworthy. Grok was probably trained on ChatGPT logs, just like the litany of other chat-oriented LLMs that are also open source. They still should've filtered out the OpenAI canned responses (there are datasets with that filter).
- gtirloni 3y agoThey shouldn't have done that though. Filtering is not the solution. OpenAI terms of service explicitly say it cannot be used to create a competing product. Is it hard to filter ChatGPT output from random chats shared in articles? Yes, it is. That's their problem though.
- nrb 3y agoCan they keep a straight face when trying to justify this clause of their terms? Isn't training on content without permission kind of their whole business model?
- dragonwriter 3y ago> Can they keep a straight face when trying to justify this clause of their terms? Probably. > Isn't training on content without permission kind of their whole business model? Right, their business model rests on the argument that copyright law doesn't require permission from copyright holders. They aren't relying on copyright law, but contract law (which only binds direct users, but applies to material not covered by copyright and applies in circumstances where even if material was covered by copyright use would be allowed as, e.g, fair use.) Is it an attempt to pull up the ladder? Obviously. But both the safety mission of the charity and the commercial competitive interests of the for profit provide rationales for this, so there is no reason to think that just because it hobbles others in trying to catch up that it would be difficult for them to do it with a straight face.
- WillPostForFood 3y agoIf they aren't OpenAI customers so they aren't subject to the terms. There is a meaningful distinction between using ChatGPT directly, putting in the prompts and taking the results, and just finding random output that other people have published in the internet and using that.
- gtirloni 3y agoThat's actually a very good point I hadn't considered.
- ilikehurdles 3y ago> OpenAI terms of service apply only to people who agree to OpenAI's terms of service (its users)
- dragonwriter 3y ago> This isn't really that newsworthy. Grok was probably trained on ChatGPT logs If Musk's "anti-woke" AI was just trained on logs from what he calls "woke AI", that would, actually, be particularly newsworthy. (The explanation they have put forward is that this is not the case, but they just failed to realize that blindly ingesting web content would bring in a lot of ChatGPT-derived content.)
- bundie 3y agoExactly.
- skygazer 3y agoI have a hard time believing that just pulling publicly available web data would instill a professed identity of an openAI creation, rather than a gen-x hipster or zoomer, or whatever. Seems so much more likely they trained predominately on gpt-3.5 output on the sly. And I’m not even sure that would do it — how often does open chatGPT identify openAI in normal responses? And OpenAI instills that sense of identity with fine tuning with synthetic data.
- sschueller 3y agoI believe it is. I've had people send me screenshots where it refers to itself as chatGPT.
- simonw 3y agoNever trust a model to talk accurately about it itself. It's trained on data that was created before the model existed.
- timeon 3y ago> Never trust a model to talk accurately about it itself. *about anything
- 3cats-in-a-coat 3y agoTell me how frequently does ChatGPT refer to itself as anything except ChatGPT? Basically never. The raw GPT model would be anything. My favorite was "I'm a young female lawyer with ambitions." But a tuned model knows what it is. Grok keeps confusing itself with ChatGPT because it was trained via ChatGPT, directly and explicitly. And apparently the team sucks at its job, filtering the data.
- dools 3y agoOr because it’s just a ChatGPT API wrapper
- 3cats-in-a-coat 3y agoI doubt, because OpenAI would figure that out and cut their access at will (and wrapping the API in this particular way by claiming it's another model is against their ToS). Also, despite it's clearly modeled after GPT, it also often does things GPT wouldn't say. In particular, it can get very repetitive, repeating itself over and over when asked to clarify a question, which curiously is also what Bing's model does. Bing is powered by multiple models, an early version of GPT-4 currently to be replaced with GPT-4 Turbo, and another model Microsoft made... which is the repetitive one. It seems part of OpenAI's secret sauce is to avoid the model getting stuck in loops, which naive efforts like xAI's and Microsoft have not independently resolved.
- simonw 3y agoGrok is a new model. Its training data has seen enough examples of OpenAI generated content that it occasionally spits out text like this that makes it look like it's by OpenAI. This is common issue across all sorts of other alternative models too. It's not particularly surprising.
- empath-nirvana 3y agoMultiple people have said something like this, but can you give me an example of another trained AI model that says it's an openai model? Every example of grok transcripts I've seen so far very much has the feel of using the gpt4 api with custom prompts (ie, not chatgpt). It wouldn't surprise me in the slightest if they're faking it.
- simonw 3y agoA lot of the models I've downloaded from a Hugging Face have exhibited this kind of mistake.
- smrtinsert 3y agoDo you remember which ones? Wondering if grok used one of those as its base
- simonw 3y agoThe team behind Grok look legit enough to me that I believe them when they say they trained a new model. Training models isn't actually that hard these days, provided you have the capital to do it. There's a LOT of good literature now, and an increasing pool of people with experience of the process.
- andy_xor_andrew 3y agoIt's far, far more likely that Grok was just trained on data that includes text generated by GPT. This is super common in the open-source/local AI area. Many models are trained on output from GPT. The better models will filter out anything that mentions GPT or OpenAI. Seems like Grok is not one of the better models.
- john-radio 3y agoI know that's X's story, but why do you consider it "far, far more likely" for that to be true?
- simonw 3y agoBecause the alternative is that Grok are reselling OpenAI API access fraudulently. OpenAI would know that they were doing this. Reselling like this is against their terms of service: https://openai.com/policies/terms-of-use https://openai.com/policies/terms-of-use
- john-radio 3y agoMy assumption was that they simply have a contract with openai.
- dools 3y agoWhere does it say that goes against their terms of service? I can’t see anything that would preclude Twitter from just plugging in the API and using it directly
- aprilthird2021 3y agoOccam's razor probably
- VirusNewbie 3y agoI think it's far more likely they're building off some open source foundational models, rather than doing one from scratch OR using GPT under the hood. If it was the latter, don't you think we'd hear about how "Grok is on par with GPT"? But you don't, people think it's not as good (but funnier?)
- hn_throwaway_99 3y agoI feel like comments defending Twitter (I'm still calling it that until they change their domain) are half missing the point. True, it's not necessarily an "OpenAI API wrapper", but the fact that it was trained on ChatGPT's logs basically means it's still an "OpenAI wrapper" of sorts, and it's going to be inferior in nearly every way (I always thought a huge problem with LLMs going forward was the risk that they would be "contaminated" with non-human training data). I can't think of any reason anybody would use Grok over ChatGPT besides political tribe signalling.
- Phil_Latio 3y agoThis is what they say in their announcement of Grok: > A unique and fundamental advantage of Grok is that it has real-time knowledge of the world via the 𝕏 platform. You disagree with that?
- empath-nirvana 3y agoChatgpt has done internet searches in responses for a while now.
- Imnimo 3y agoI would disagree that that constitutes a "fundamental advantage"
- dragonwriter 3y agoIt's technically unique since the kind of access it has to X is obviously proprietary. It's not a fundamental advantage for most AI model uses or a particularly good source of knowledge of the world.
- qarl 3y agoI wonder why they decided not to filter-out references to OpenAI and ChatGPT.
- anonymouskimmer 3y agoBecause people may ask how X and Grok are superior to OpenAI and ChatGPT?
- LorenDB 3y agoI had self hosted Llama 2 call itself Bard the other day, so I'd take this with a grain of salt - or maybe even the whole shaker.
- gafage 3y agoIt makes no sense to me that one would train a chatbot on chatgpt conversations and not filter strings that literally say "openai" and "chatgpt". Extreme incompetence.
- sertbdfgbnfgsd 3y agoElon Musk playing 5 dimensional chess to save freedom of speech.
- spacecadet 3y agolol
- thorncorona 3y agoExcluding OpenAI/ChatGPT generated content without excluding discourse that mentions OpenAI / ChatGPT such as news articles and industry papers seems like a nontrivial problem to solve at scale.
- qarl 3y agoHandling the general case is hard. Handling the huge number of cases where ChatGPT says something like "As a large language model created by OpenAI" would be very simple.
- minimaxir 3y agoAn xAI engineer replied: > The issue here is that the web is full of ChatGPT outputs, so we accidentally picked up some of them when we trained Grok on a large amount of web data. This was a huge surprise to us when we first noticed it. For what it’s worth, the issue is very rare and now that we’re aware of it we’ll make sure that future versions of Grok don’t have this problem. Don’t worry, no OpenAI code was used to make Grok. https://twitter.com/ibab_ml/status/1733558576982155274 https://twitter.com/ibab_ml/status/1733558576982155274
- timdorr 3y agoIt was a huge surprise that ChatGPT is popular?
- thorncorona 3y agoThe amount of AI generated SEO spam.
- dools 3y agoHow much SEO spam has chatGPT refusing to do things in it though?
- voiceblue 3y agoIsn’t this tantamount to an admission of willfully flaunting copyright law? Edit: I'm not 100% sure, but I think my comment is being interpreted uncharitably. I would like to grab your finger, which is pressed squarely atop "output of other LLMs", and move it over a bit to the label of "website with unknown and almost certainly copyright content". And then, I would like to point out that there is a difference between training on such data with the benefit of plausible deniability (e.g. some third party), versus going specifically after data you know to be copyrighted, and then claiming that this is precisely the reason why your model will spit such content out verbatim. At that point, you might as well indict yourself, no?
- WillPostForFood 3y ago
- ineedaj0b 3y ago[flagged]
- alsodumb 3y agoA simple explanation is that Grok probably used GPT3.5 or GPT4 API to generate synthetic data, most likely for RLHF.
- deleted 3y ago[deleted]