7 ms·

I thought OpenAI was the first to develop and use the term GPT[1]. So maybe a more apt comparison than RNA would be PCR, which was a technique developed in a pr

I thought OpenAI was the first to develop and use the term GPT[1]. So maybe a more apt comparison than RNA would be PCR, which was a technique developed in a private lab and patented, even though it was published in scientific journals etc. 1. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised...

blagie 3y ago
Doesn't matter either way who invented it. This is trademark, not patent. If they wanted a trademark, that needed to appear on copy from day 0. The paper is titled: "Improving Language Understanding by Generative Pre-Training" and not "Improving Language Understanding by Generative Pre-Training (TM)" They ALSO would have needed to use a different generic term to refer to the technique. For example, Velcro always says things like "Velcro-brand hook-and-loop," and ALWAYS uses a generic term to refer to hook-and-loop when talking across brands. It ONLY uses "Velcro(tm)" to talk about their product specifically. OpenAI started using GPT generically. Publications like this one, if anything, undermine OpenAI's case since they're using the term generically. The decision is correct. Trademark law is used precisely to prevent this sort of thing: OpenAI can have a generic term or a trademark. The law is set up precisely to prevent a company from building their brand value by retroactively trademarking a generic term once relying on a community to get it established. To be more blunt: I use "GPT" generically because they encouraged me to do so. That should not contribute brand value to OpenAI GPT-3 or OpenAI GPT-4.
- rubyfan 3y ago
  Also, why would a “non-profit” need to trademark it?
  - Anduia 3y ago
    Many do so in order to maintain their integrity and reputation. For example, the name Wikipedia and the Wikipedia logo are registered trademarks. In the case of fundrising and sponsorship, it assures to sponsors the legitimity of that campaign, because there is no conflict with other organisation or company using the same name. I could go on, but I hope you get the point.
  - numbsafari 3y ago
    Non-profits seek trademarks for all sorts of reasons that have nothing to do with profit.
    - dumbfounder 3y ago
      And, non-profits doesn't mean they don't do things that they profit from, it just means they aren't trying to achieve profits for their owners. Endowments are non-profits, but they most certainly are making investments to achieve profits for the purpose of pushing that money into charitable vehicles.
      - numbsafari 3y ago
        A non-profit may also want to prevent others from profiting from their work by using their trademarks without approval for private gain. Having a trademark just means you are protecting a name, not that you are seeking to extract value, per se.
  - fnordpiglet 3y ago
    Non profit is a business structure not a designation meaning “no business acumen.” Many non profits are actually quite profitable and active businesses, but they have no beneficial owners, pay no dividends, etc, and often have some mission that’s in some way broadly beneficial. This exempts them from certain taxation and other benefits. But it absolutely doesn’t mean everything they do is done altruistically without material consideration and definitely doesn’t mean with no legal claims or recourse for protecting their works or identities.
    - dizhn 3y ago
      Plus they still get to pay huge salaries.
      - jameshart 3y ago
        Maybe because they need expensive skills
        smallmancontrov 3y ago
        I'm sure the highly paid charity execs would say so. Trouble is, the charity space often suffers from a broken market feedback mechanism, where the people paying for the product are not the people consuming the product, and this can lead to a business structure that looks like a pure-play marketing machine hooked up to exec pockets with occasional leakage into a small amount of actual charity work. "But the situation occurs in regular business too! Monopolies, oligopolies, etc happen when the market feedback mechanism breaks!" Yeah, and we should go after those too. It's really astonishing the lengths to which people go to defend bad behavior.
        fnordpiglet 3y ago
        Non profit doesn’t necessarily mean charity. For instance, the life line company (“help me I’ve fallen and can’t get up”) is (or was) organized as a non profit. They sold devices and services at a decent margin. Their excess revenues went back to employees in wages and perks. Executives and founders especially enjoyed extravagant life styles.
  - Ensorceled 3y ago
    So other people can't pretend to be them or their products and do things like raise funds. Which is also the point of "for profit" trademarks
  - ArnoVW 3y ago
    For the same reason Wikipedia needs a trademark, or any successful open source project. To prevent grifters from hitching a ride off your word and muddying the water / confusing users of your service or product
  - HWR_14 3y ago
    To prevent for profit companies from using it. The Olympic Rings and the Red Cross's Red Cross logo are both trademarked.
- jasonjayr 3y ago
  Velcro has an amusing campaign regarding their trademark: https://www.velcro.com/original-thinking/the-velcro-brand-trademark-guidelines/dont-say-velcro/ https://www.velcro.com/original-thinking/the-velcro-brand-tr...
  - kristofferR 3y ago
    This is seriously the type of language that will get men to want to "abuse" the Velcro trademark more. https://torrentfreak.com/iptv-anti-piracy-threats-may-increase-male-motivation-to-pirate-by-30-240202/ https://torrentfreak.com/iptv-anti-piracy-threats-may-increa...
    - jrockway 3y ago
      Unlikely. The Velcro trademark has been so far gone for decades that I doubt there is a single person on the planet that knows it's the name of a company and not a generic term for hook and loop fastener. The next company you'll see in this position is probably Google. I think that verb will outlive Google Search.
      - smolder 3y ago
        > I doubt there is a single person on the planet that knows ... Are we not persons on the planet?
        elzbardico 3y ago
        Speak for yourself, human.
      - singingboyo 3y ago
        More importantly, knowing and caring are two different things. I know that Velcro/Kleenex/Google are specific brands, but I don't really care - the common usage is so far gone that there's rarely a reason to use hook & loop fastener/tissue paper/internet search instead. Hell, for some people, "iPad" is a semi-generic term for a tablet. (Though I don't get that one, personally.)
- technothrasher 3y ago
  > If they wanted a trademark, that needed to appear on copy from day 0. The paper is titled: "Improving Language Understanding by Generative Pre-Training" and not "Improving Language Understanding by Generative Pre-Training (TM)" While the (TM) is useful to alert people to your trademark claim, you are not required to use it to establish your trademark. Simply using a unique mark to identify your goods or services and being the first to do so is enough.
  - gcr 3y ago
    Then why did openAI fail here, in your view?
    - bluGill 3y ago
      Because they didn't make any effort to treat this like a trade mark until after the term itself become generic to the public. If you want a trademark you need to be careful how you use it and how others use it to ensure it doesn't become a generic term. Once something is a generic term it is almost impossible to get it back. (it has been done: Xerox used to be the generic term people used for make a copy - but most of the effort was their competitors who for obvious reasons didn't want to use their competitors company name as a generic term) If you want a trademark you need to defend it. That means you know the generic term and use that when required. That means when anyone uses your trademark in a generic way your lawyers are immediately sending letters. Check with a lawyer - there are a lot more details you have to get right.
      - CactusOnFire 3y ago
        Yeah, just because OpenAI has gotten away with using open source ideology for regulatory capture in the AI space, but that doesn't mean they can get away with it in the intellectual property space.
      - autokad 3y ago
        is it a generic term? when I think gpt I only think openAI's work
nicklecompte 3y ago
RNA, PCR, and GPT all have precisely the same problem: the acronyms are purely technical. - ribonucleic acid - polymerase chain reaction - generative pretrained transformer You can't trademark the phrase "polymerase chain reaction" because there are many types of polymerase chain reactions, not just the ones used for copying DNA. [ETA: specifically I mean "copying DNA at commercial scale" i.e. the patented biochemical engineering techniques.] Likewise a generative neural network using pre-training with a transformer architecture is simply too generic to trademark "generative pretrained transformer." And if you can't trademark the phrase you can't trademark the acronym.
- snitty 3y ago
  I endorse this analysis. Sincerely, Your local IP attorney.
  - jacobsimon 3y ago
    Is this actually true though? KFC is able to trademark their name and acronym (I believe their trademark is for the acronym officially), even though they’re selling Kentucky-style fried chicken?
- refurb 3y ago
  What about International Business Machine?
  - nicklecompte 3y ago
    IBM doesn't sell products called "international business machines" and hold a trademark on that product name, which would force competing manufacturers to call them "devices for transcontinental enterprise" etc. IBM is a business name, that's a different thing. More importantly, nobody calls computers "international business machines," that name reflects IBM's 19th-century origins, so having a trademark on "International Business Machines" has no impact on competing businesses. A hypothetical 1930s computing company called Central Processor Units would have probably changed its name in the 1950s, since "CPU" wouldn't be trademarkable and their brand name wouldn't be worth very much. (A trademark on "CPU" would be needlessly detrimental to other computer manufacturers, or at best confusing, especially after the von Neumann architecture became the universal standard.) Of course in this alternate universe maybe CPUs would have been called something else, and maybe if IBM never existed we would be calling computers "business machines." IANAL but I think it's a mistake to apply "If A then B" rules to this stuff and try to invalidate reasonable guidelines based on specific counterexamples. Judges need to consider how language is actually used in context by the people working in those areas. This is why the USPTO cited so many businesses and practitioners using GPT in a generic context.
bhickey 3y ago
The term 'GPT' was first used in the BERT paper to refer to Generic Pre-trained Transformers. [0] > The fine-tuning approach, such as the Generative Pre-trained Transformer (OpenAI GPT) (Radford et al., 2018), introduces minimal task-specific parameters, and is trained on the downstream tasks by simply fine-tuning all pre-trained parameters. [0] https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805
- niutech 3y ago
  BERT paper is from 11 Oct 2018 and the OpenAI paper it was referring to "Improving Language Understanding by Generative Pre-Training" is from 11 Jun 2018: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised...
  - lolinder 3y ago
    Yes, but that paper that you link to doesn't ever call it GPT or Generative Pretrained Transformers. It talks about training Transformers with Generative Pretraining, both of which are pre-existing concepts by this point. I also looked on the OpenAI website in Sep 2018 and could find no reference to GPT or Generative Pretrained Transformers, so I think OP might be right about BERT using it first. http://web.archive.org/web/20180923011305/https://blog.openai.com/language-unsupervised/ http://web.archive.org/web/20180923011305/https://blog.opena...
lolinder 3y ago
That paper is using a pre-existing term "Generative Pretraining" [0] and applying it to Transformers (a Google innovation [1]). As far as I can see from a search, they don't even use the term GPT or Generative Pretrained Transformers in that paper, and they don't in the accompanying blog post either [2]. A sibling [3] claims that the BERT paper in Oct 2018 was the first to use the term GPT to describe what OpenAI built, and that sounds reasonable since a cursory look through the Sep 2018 archive of openai.com turns up nothing. [0] See this 2012 example: http://cs224d.stanford.edu/papers/maas_paper.pdf http://cs224d.stanford.edu/papers/maas_paper.pdf [1] https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf https://proceedings.neurips.cc/paper/2017/file/3f5ee243547de... [2] http://web.archive.org/web/20180923011305/https://blog.openai.com/language-unsupervised/ http://web.archive.org/web/20180923011305/https://blog.opena... [3] https://news.ycombinator.com/item?id=39381802 https://news.ycombinator.com/item?id=39381802
kurthr 3y ago
Yeah, but my 2 year old Generative Potty-Trained Transformer needs an acronym too! Seriously, like TLA domain names and other TLA acronyms, there just aren't enough to allow trademark. Imagine, if 5 letter LASER had been trademarked? Would that have also covered LASIK (Laser assisted sub-epithelial keratectomy)?

blagie 3y ago

Doesn't matter either way who invented it. This is trademark, not patent. If they wanted a trademark, that needed to appear on copy from day 0. The paper is titled: "Improving Language Understanding by Generative Pre-Training" and not "Improving Language Understanding by Generative Pre-Training (TM)" They ALSO would have needed to use a different generic term to refer to the technique. For example, Velcro always says things like "Velcro-brand hook-and-loop," and ALWAYS uses a generic term to refer to hook-and-loop when talking across brands. It ONLY uses "Velcro(tm)" to talk about their product specifically. OpenAI started using GPT generically. Publications like this one, if anything, undermine OpenAI's case since they're using the term generically. The decision is correct. Trademark law is used precisely to prevent this sort of thing: OpenAI can have a generic term or a trademark. The law is set up precisely to prevent a company from building their brand value by retroactively trademarking a generic term once relying on a community to get it established. To be more blunt: I use "GPT" generically because they encouraged me to do so. That should not contribute brand value to OpenAI GPT-3 or OpenAI GPT-4.

rubyfan 3y ago
Also, why would a “non-profit” need to trademark it?
- Anduia 3y ago
  Many do so in order to maintain their integrity and reputation. For example, the name Wikipedia and the Wikipedia logo are registered trademarks. In the case of fundrising and sponsorship, it assures to sponsors the legitimity of that campaign, because there is no conflict with other organisation or company using the same name. I could go on, but I hope you get the point.
- numbsafari 3y ago
  Non-profits seek trademarks for all sorts of reasons that have nothing to do with profit.
  - dumbfounder 3y ago
    And, non-profits doesn't mean they don't do things that they profit from, it just means they aren't trying to achieve profits for their owners. Endowments are non-profits, but they most certainly are making investments to achieve profits for the purpose of pushing that money into charitable vehicles.
    - numbsafari 3y ago
      A non-profit may also want to prevent others from profiting from their work by using their trademarks without approval for private gain. Having a trademark just means you are protecting a name, not that you are seeking to extract value, per se.
- fnordpiglet 3y ago
  Non profit is a business structure not a designation meaning “no business acumen.” Many non profits are actually quite profitable and active businesses, but they have no beneficial owners, pay no dividends, etc, and often have some mission that’s in some way broadly beneficial. This exempts them from certain taxation and other benefits. But it absolutely doesn’t mean everything they do is done altruistically without material consideration and definitely doesn’t mean with no legal claims or recourse for protecting their works or identities.
  - dizhn 3y ago
    Plus they still get to pay huge salaries.
    - jameshart 3y ago
      Maybe because they need expensive skills
      - smallmancontrov 3y ago
        I'm sure the highly paid charity execs would say so. Trouble is, the charity space often suffers from a broken market feedback mechanism, where the people paying for the product are not the people consuming the product, and this can lead to a business structure that looks like a pure-play marketing machine hooked up to exec pockets with occasional leakage into a small amount of actual charity work. "But the situation occurs in regular business too! Monopolies, oligopolies, etc happen when the market feedback mechanism breaks!" Yeah, and we should go after those too. It's really astonishing the lengths to which people go to defend bad behavior.
        fnordpiglet 3y ago
        Non profit doesn’t necessarily mean charity. For instance, the life line company (“help me I’ve fallen and can’t get up”) is (or was) organized as a non profit. They sold devices and services at a decent margin. Their excess revenues went back to employees in wages and perks. Executives and founders especially enjoyed extravagant life styles.
- Ensorceled 3y ago
  So other people can't pretend to be them or their products and do things like raise funds. Which is also the point of "for profit" trademarks
- ArnoVW 3y ago
  For the same reason Wikipedia needs a trademark, or any successful open source project. To prevent grifters from hitching a ride off your word and muddying the water / confusing users of your service or product
- HWR_14 3y ago
  To prevent for profit companies from using it. The Olympic Rings and the Red Cross's Red Cross logo are both trademarked.
jasonjayr 3y ago
Velcro has an amusing campaign regarding their trademark: https://www.velcro.com/original-thinking/the-velcro-brand-trademark-guidelines/dont-say-velcro/ https://www.velcro.com/original-thinking/the-velcro-brand-tr...
- kristofferR 3y ago
  This is seriously the type of language that will get men to want to "abuse" the Velcro trademark more. https://torrentfreak.com/iptv-anti-piracy-threats-may-increase-male-motivation-to-pirate-by-30-240202/ https://torrentfreak.com/iptv-anti-piracy-threats-may-increa...
  - jrockway 3y ago
    Unlikely. The Velcro trademark has been so far gone for decades that I doubt there is a single person on the planet that knows it's the name of a company and not a generic term for hook and loop fastener. The next company you'll see in this position is probably Google. I think that verb will outlive Google Search.
    - smolder 3y ago
      > I doubt there is a single person on the planet that knows ... Are we not persons on the planet?
      - elzbardico 3y ago
        Speak for yourself, human.
    - singingboyo 3y ago
      More importantly, knowing and caring are two different things. I know that Velcro/Kleenex/Google are specific brands, but I don't really care - the common usage is so far gone that there's rarely a reason to use hook & loop fastener/tissue paper/internet search instead. Hell, for some people, "iPad" is a semi-generic term for a tablet. (Though I don't get that one, personally.)
technothrasher 3y ago
> If they wanted a trademark, that needed to appear on copy from day 0. The paper is titled: "Improving Language Understanding by Generative Pre-Training" and not "Improving Language Understanding by Generative Pre-Training (TM)" While the (TM) is useful to alert people to your trademark claim, you are not required to use it to establish your trademark. Simply using a unique mark to identify your goods or services and being the first to do so is enough.
- gcr 3y ago
  Then why did openAI fail here, in your view?
  - bluGill 3y ago
    Because they didn't make any effort to treat this like a trade mark until after the term itself become generic to the public. If you want a trademark you need to be careful how you use it and how others use it to ensure it doesn't become a generic term. Once something is a generic term it is almost impossible to get it back. (it has been done: Xerox used to be the generic term people used for make a copy - but most of the effort was their competitors who for obvious reasons didn't want to use their competitors company name as a generic term) If you want a trademark you need to defend it. That means you know the generic term and use that when required. That means when anyone uses your trademark in a generic way your lawyers are immediately sending letters. Check with a lawyer - there are a lot more details you have to get right.
    - CactusOnFire 3y ago
      Yeah, just because OpenAI has gotten away with using open source ideology for regulatory capture in the AI space, but that doesn't mean they can get away with it in the intellectual property space.
    - autokad 3y ago
      is it a generic term? when I think gpt I only think openAI's work

nicklecompte 3y ago

RNA, PCR, and GPT all have precisely the same problem: the acronyms are purely technical. - ribonucleic acid - polymerase chain reaction - generative pretrained transformer You can't trademark the phrase "polymerase chain reaction" because there are many types of polymerase chain reactions, not just the ones used for copying DNA. [ETA: specifically I mean "copying DNA at commercial scale" i.e. the patented biochemical engineering techniques.] Likewise a generative neural network using pre-training with a transformer architecture is simply too generic to trademark "generative pretrained transformer." And if you can't trademark the phrase you can't trademark the acronym.

snitty 3y ago
I endorse this analysis. Sincerely, Your local IP attorney.
- jacobsimon 3y ago
  Is this actually true though? KFC is able to trademark their name and acronym (I believe their trademark is for the acronym officially), even though they’re selling Kentucky-style fried chicken?
refurb 3y ago
What about International Business Machine?
- nicklecompte 3y ago
  IBM doesn't sell products called "international business machines" and hold a trademark on that product name, which would force competing manufacturers to call them "devices for transcontinental enterprise" etc. IBM is a business name, that's a different thing. More importantly, nobody calls computers "international business machines," that name reflects IBM's 19th-century origins, so having a trademark on "International Business Machines" has no impact on competing businesses. A hypothetical 1930s computing company called Central Processor Units would have probably changed its name in the 1950s, since "CPU" wouldn't be trademarkable and their brand name wouldn't be worth very much. (A trademark on "CPU" would be needlessly detrimental to other computer manufacturers, or at best confusing, especially after the von Neumann architecture became the universal standard.) Of course in this alternate universe maybe CPUs would have been called something else, and maybe if IBM never existed we would be calling computers "business machines." IANAL but I think it's a mistake to apply "If A then B" rules to this stuff and try to invalidate reasonable guidelines based on specific counterexamples. Judges need to consider how language is actually used in context by the people working in those areas. This is why the USPTO cited so many businesses and practitioners using GPT in a generic context.

bhickey 3y ago

The term 'GPT' was first used in the BERT paper to refer to Generic Pre-trained Transformers. [0] > The fine-tuning approach, such as the Generative Pre-trained Transformer (OpenAI GPT) (Radford et al., 2018), introduces minimal task-specific parameters, and is trained on the downstream tasks by simply fine-tuning all pre-trained parameters. [0] https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805

niutech 3y ago
BERT paper is from 11 Oct 2018 and the OpenAI paper it was referring to "Improving Language Understanding by Generative Pre-Training" is from 11 Jun 2018: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised...
- lolinder 3y ago
  Yes, but that paper that you link to doesn't ever call it GPT or Generative Pretrained Transformers. It talks about training Transformers with Generative Pretraining, both of which are pre-existing concepts by this point. I also looked on the OpenAI website in Sep 2018 and could find no reference to GPT or Generative Pretrained Transformers, so I think OP might be right about BERT using it first. http://web.archive.org/web/20180923011305/https://blog.openai.com/language-unsupervised/ http://web.archive.org/web/20180923011305/https://blog.opena...

lolinder 3y ago

That paper is using a pre-existing term "Generative Pretraining" [0] and applying it to Transformers (a Google innovation [1]). As far as I can see from a search, they don't even use the term GPT or Generative Pretrained Transformers in that paper, and they don't in the accompanying blog post either [2]. A sibling [3] claims that the BERT paper in Oct 2018 was the first to use the term GPT to describe what OpenAI built, and that sounds reasonable since a cursory look through the Sep 2018 archive of openai.com turns up nothing. [0] See this 2012 example: http://cs224d.stanford.edu/papers/maas_paper.pdf http://cs224d.stanford.edu/papers/maas_paper.pdf [1] https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf https://proceedings.neurips.cc/paper/2017/file/3f5ee243547de... [2] http://web.archive.org/web/20180923011305/https://blog.openai.com/language-unsupervised/ http://web.archive.org/web/20180923011305/https://blog.opena... [3] https://news.ycombinator.com/item?id=39381802 https://news.ycombinator.com/item?id=39381802

kurthr 3y ago

Yeah, but my 2 year old Generative Potty-Trained Transformer needs an acronym too! Seriously, like TLA domain names and other TLA acronyms, there just aren't enough to allow trademark. Imagine, if 5 letter LASER had been trademarked? Would that have also covered LASIK (Laser assisted sub-epithelial keratectomy)?