8 ms·
For those unaware - GPT stands for Generative pre-trained transformers (GPT). It's a type of a deep neural network architecture that builds on top of Transforme
by amrrs 4y ago
For those unaware - GPT stands for Generative pre-trained transformers (GPT). It's a type of a deep neural network architecture that builds on top of Transformers which was released by Google researchers. I'm wondering what would happen if someone copyrights "Transformers" now :-/
- LeoPanthera 4y agoHasbro might have something to say about that.
- neilv 4y agoEmergent behavior is more than meets the eye.
- PeterisP 4y agoNot really, trademarks are industry-specific, and having a registered trademark in another industry where there is no reasonable confusion is not an obstacle. For example, "Oracle" is a valid registered trademark for cigars and for archery bow sights and for many, many other things, despite also being a trademark for one of the most litigious firms in the software industry.
- galleywest200 4y agoAre they industry specific? Monster Energy has a history of suing for trademark infringement over the word "Monster" in video game titles. Companies have caved to their requests. Unsure if that was due to validity or lack of willing to pay legal fees. https://news.yahoo.com/monster-energy-tried-pok-mon-224500250.html https://news.yahoo.com/monster-energy-tried-pok-mon-22450025...
- kube-system 4y agoYes, they are definitely industry specific. Which is why both Best Buy and your grocery store can sell Apples. Monster has been involved in esports previously. https://www.monsterenergy.com/gaming https://www.monsterenergy.com/gaming
- ska 4y agoYes, they are. Sometimes the lines become problematic. A famous example is Apple Computer and Apple Corps (the Beatles music holding company) with a long history of arguing about lines: https://en.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer https://en.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer
- kalleboo 4y agoWhat is it with companies named Monster and over-reaching trademark lawsuits? I remember the days when Monster Cable sued monster.com, Disney (Monsters, Inc), some tiny mini golf outfit...
- kube-system 4y agoThere are some Transformers games, I haven’t looked it up but I wouldn’t be surprised if Hasbro had one for “computer software”.
- jameshart 4y agoThey probably have some claim to trademark rights in the fields of robotics and cybernetics.
- kkielhofner 4y agoThey are applied for in various bewildering categories so yes, kind of industry specific. I've always found the categories to be pretty bizarre and of course you can apply in multiple categories. The key with trademarks is whether or not use/application of a trademark is "likelihood of confusion"[0]. So yeah, I can call my hardware store GPT and OpenAI can have GPT trademarked because there's no way someone could confuse them as the same thing. That's of course an easy example and if this trademark is issued anything even remotely touching AI/ML using the three letters "GPT" is probably infringing. What's really ridiculous about this is the execs at OpenAI didn't see this coming - I think trademark application for GPT is going to be challenging while simultaneously making them look like jerks. Plus, GPT is a non-sensical acronym to 99% of their market so other than the recognition they've created in calling it that there's no reason for this issue. They would have been much, much better off calling their GPTs and GPT-related products something (anything) that's actually unique and easily trademarked. [0] - https://www.uspto.gov/trademarks/search/likelihood-confusion https://www.uspto.gov/trademarks/search/likelihood-confusion
- jjtheblunt 4y agothat term has been in use for decades e.g., https://www.digikey.com/en/products/category/transformers/11 https://www.digikey.com/en/products/category/transformers/11
- kube-system 4y agoDoesn’t matter. Trademarks are product/service specific.
- breck 4y agoNot that it matters much, but I'm curious who first coined the specific "GPT" term? The first occurrence of that particular term I can find on Google Scholar is in 2019 ("Documentation is all you need" https://scholar.google.com/scholar?q=%22Generative+pre-trained+transformers%22&hl=en&as_sdt=0%2C5&as_ylo=2010&as_yhi=2019 https://scholar.google.com/scholar?q=%22Generative+pre-train...) Edit: it looks like it was OpenAI. (Radford et al, 2018).
- anxrn 4y agoYes, this paper [1] introduced the term. https://scholar.google.com/citations?view_op=view_citation&hl=en&user=dOad5HoAAAAJ&citation_for_view=dOad5HoAAAAJ:W7OEmFMy1HYC https://scholar.google.com/citations?view_op=view_citation&h...
- runnerup 4y agoWeird. The term “Generative pretraining” seems to also exist in this 2016 proceeding from 2 years prior: https://ui.adsabs.harvard.edu/abs/2016arXiv160202644D/abstract https://ui.adsabs.harvard.edu/abs/2016arXiv160202644D/abstra... Google Trends shows it first got searched for in measurable quantity back in 2008.
- reaperman 4y agoHere's "generative pretaining" in an IEEE publication from 2012. https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf Geoffrey Hinton wrote it. I wonder if he'll have anything to say about OpenAI trademarking the term 11 years after he wrote this. Here's an earlier publication from January, 2012 talking about Hinton's 2011 "Generative pretraining" work: https://www.microsoft.com/en-us/research/publication/three-classes-of-deep-learning-architectures-and-their-applications-a-tutorial-survey/ https://www.microsoft.com/en-us/research/publication/three-c...
- hn_throwaway_99 4y ago> Geoffrey Hinton wrote it. I wonder if he'll have anything to say about OpenAI trademarking the term 11 years after he wrote this. But they're not applying to trademark "generative pretraining". They're only applying to trademark GPT, which seems (I have very limited knowledge if there are other competing uses of the term) reasonable given that most people associate "GPT" with specific AI implementations created by OpenAI. There were other "business machine" companies that existed 100 years ago, but that is very different from trademarking "IBM".
- chrisco255 4y agoIt's a trademark application, not a copyright application. Quite different. You see copyrighted material is that which a GPT has been trained on, without any regards to those rights or protections. While a GPT trademark, on the other hand, is an underhanded attempt to leverage that same disregarded legal protection to secure an exclusive brand for itself, in spite of existing uses of the generic acronym.
- matemp 3y agohttps://www.lexology.com/library/detail.aspx?g=763049f7-7ef8-4a68-bdb1-2e4fa194b7ad https://www.lexology.com/library/detail.aspx?g=763049f7-7ef8...
- tick_tock_tick 4y agoSo the first one is a positive while the second is a negative? And they fully considered the rights of ever bit of copyrighted material they consumed. It's not their fault random people on the internet don't understand what those rights are.
- ben_w 4y agoThey do different things. Trademarks protect consumers from rip-off copies pretending to be what they want, copyright protects content creators from rip-off copies that genuinely are what consumers want.
- bombolo 4y ago> And they fully considered the rights of ever bit of copyrighted material they consumed. Yes, they considered and decided that they are big enough to violate and do what they want, since they aren't violating rights of other big entities, just of nobodies.
- tick_tock_tick 4y agoFeel free to cite what they violated. People are really quick to claim this but can't back it up or grab one of two sentences from one of the copyright statutes and claim it's a clear violation without any understanding of what they just copied.
- forrestthewoods 4y agoI hope someone trademarks “Transformer”. It’s the stupidest name ever. The entire purpose of all code is to transform data. See 15 years of data oriented design discussion. If I ever invent a specific ML design I’m going to call it “computer”. Because that’s as terrible of a name as “transformer”.
- ben_w 4y ago"Computer" has been trade marked several times, for example: https://trademarks.ipo.gov.uk/ipo-tmcase/page/Results/1/UK00003469549 https://trademarks.ipo.gov.uk/ipo-tmcase/page/Results/1/UK00... As has "Transformer": https://trademarks.ipo.gov.uk/ipo-tmtext/page/Results https://trademarks.ipo.gov.uk/ipo-tmtext/page/Results Or, y'know, "Apple", "Orange", "Blackberry", "Windows", etc.
- forrestthewoods 4y agoAnd your point is what exactly? Literally every ML model is a transformer. Every single one of them. All of them are transformers. Using the word “transformer” to refer to a specific architecture is therefore absurd. This is very different than naming a computer company Apple or Blackberry.
- ben_w 4y agoNo stranger than Windows, given WIMP, even if you want to disregard the link to the trademarks for "transformer" (which I now realise doesn't work because the URL doesn't contain the search term, so here's one item from that list: https://trademarks.ipo.gov.uk/ipo-tmcase/page/Results/1/UK00003423820 https://trademarks.ipo.gov.uk/ipo-tmcase/page/Results/1/UK00...) Trademarks are just like this, always have been, probably always will be.
- forrestthewoods 4y agoYou’re missing my point. That’s on me. I think Transformer is a stupid name. I think the researchers who chose the name Transformer are big poopyheads for picking a bad name. I’d rather they have named it after themselves, an anagram of their names, or whatever. I wish everyone would stop using the name Transformer. I hope it does get trademarked for the sole purpose of reducing its usage. Ideally to zero. I hate analogies. I think it would be similar stupid if Epic Games renamed “Unreal Engine” to “Game Engine” or if Toyota renamed the Camry to “Car”. It’s extremely different from Apple. It’s vaguely similar to Windows. Except when I write the words “window” in code no one will confuse it for the Operating System. Meanwhile I have written quite a lot of ML code and have regularly used the terms “transform” and “transformer”. None of which have had anything to do with the “Transformer” architecture.
- PeterisP 4y agoIt's not copyright, it's trademark. It doesn't affect the ability of others to build GPT-like models, but it would prohibit others from calling whatever they build 'somethingGPT' or 'GPT-5'. And if someone manages to trademark "transformers" (which IMHO is totally unlikely to succeed for multiple reasons), well then we'd shrug and call them something else, that wouldn't prohibit anyone from making or using transformer models.
- lrei 4y agoTransformer is the architecture. "Generative Pretrained" is just a term made up by the author to mean what everyone called for decades before and will call for decades after "Language Modelling". It was just a new way of saying "Language Modelling Transformer" that sounded cooler to the author and gave it cool initials. Coming up with cool names for models is hard.
- thesuperbigfrog 4y ago"Transformers" is already trademarked by Hasbro: https://tsdr.uspto.gov/#caseNumber=76051283&caseType=SERIAL_NO&searchType=statusSearch https://tsdr.uspto.gov/#caseNumber=76051283&caseType=SERIAL_... Maybe they will create ChatGPT-powered Autobot and Decepticon toys?
- HeckFeck 4y agoLawsuits in disguise.
- exitb 4y agoAnd IBM stands for International Business Machines. It didn’t make selling business machines internationally illegal. As far as I can see, OpenAI invented that term. Don’t see why they couldn’t trademark it.
- seizethecheese 4y agoYep. This is pearl clutching. Not HN at its best. Most commenters here don’t seem to really have any grasp of trademark law at all.
- ChatGTP 4y agoIt first came from Google. They didn’t invent it.
- TallGuyShort 4y agoGoogle released the paper on transformers. Didn't OpenAI coin the term for generative, pre-trained ones?
- im3w1l 4y agoCan you point to where Google used the abbreviation "GPT" to refer to one of it's models or the architecture in general?
- ChatGTP 4y agoGoogle it.
- gremlinsinc 4y agothis is like MacOS trying to trademark OS. Did openai even discover the tech, pretty sure it was based on a research paper.