4 ms·

Not that it matters much, but I'm curious who first coined the specific "GPT" term? The first occurrence of that particular term I can find on Google Scholar i

by breck 4y ago

Not that it matters much, but I'm curious who first coined the specific "GPT" term? The first occurrence of that particular term I can find on Google Scholar is in 2019 ("Documentation is all you need" https://scholar.google.com/scholar?q=%22Generative+pre-trained+transformers%22&hl=en&as_sdt=0%2C5&as_ylo=2010&as_yhi=2019 https://scholar.google.com/scholar?q=%22Generative+pre-train...) Edit: it looks like it was OpenAI. (Radford et al, 2018).

anxrn 4y ago
Yes, this paper [1] introduced the term. https://scholar.google.com/citations?view_op=view_citation&hl=en&user=dOad5HoAAAAJ&citation_for_view=dOad5HoAAAAJ:W7OEmFMy1HYC https://scholar.google.com/citations?view_op=view_citation&h...
- runnerup 4y ago
  Weird. The term “Generative pretraining” seems to also exist in this 2016 proceeding from 2 years prior: https://ui.adsabs.harvard.edu/abs/2016arXiv160202644D/abstract https://ui.adsabs.harvard.edu/abs/2016arXiv160202644D/abstra... Google Trends shows it first got searched for in measurable quantity back in 2008.
  - reaperman 4y ago
    Here's "generative pretaining" in an IEEE publication from 2012. https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf Geoffrey Hinton wrote it. I wonder if he'll have anything to say about OpenAI trademarking the term 11 years after he wrote this. Here's an earlier publication from January, 2012 talking about Hinton's 2011 "Generative pretraining" work: https://www.microsoft.com/en-us/research/publication/three-classes-of-deep-learning-architectures-and-their-applications-a-tutorial-survey/ https://www.microsoft.com/en-us/research/publication/three-c...
    - hn_throwaway_99 4y ago
      > Geoffrey Hinton wrote it. I wonder if he'll have anything to say about OpenAI trademarking the term 11 years after he wrote this. But they're not applying to trademark "generative pretraining". They're only applying to trademark GPT, which seems (I have very limited knowledge if there are other competing uses of the term) reasonable given that most people associate "GPT" with specific AI implementations created by OpenAI. There were other "business machine" companies that existed 100 years ago, but that is very different from trademarking "IBM".
mhink 4y ago
Interestingly, the record for OpenAI's application (for "GPT") has a note under "Goods and Services" that reads: "FIRST USE: 20181000. FIRST USE IN COMMERCE: 20181000" [1] The paper "Improving Language Understanding by Generative Pre-Training" appears to have been released on June 11, 2018. [2] 1: It looks like I can't link directly to it, but it should be pretty easy to find by searching for "GPT" at the parent's link 2: https://openai.com/research/language-unsupervised https://openai.com/research/language-unsupervised
gpm 4y ago
I was curious and discovered that Google has a (non-AI, advertising related) product called GPT that has existed for ages, here's a reference from 2015 https://github.com/davidecantoni/googletag https://github.com/davidecantoni/googletag I'm not sure if the industry is similar enough that this will be a problem for OpenAI (I suspect the fact that people are using GPT to refer to generative pre-trained transformer's in general is more of an issue for this application). GPT partition tables also come to mind.
- hgsgm 4y ago
  There are many many overlapping trademarks on initialisms.
- hn_throwaway_99 4y ago
  Good find. As the other commenter noted, however, trademarks are limited to the narrow field to which they apply. So I'm assuming OpenAI will be successful in getting a trademark for GPT in the field of generative AI, but it will be limited to that scope.
- gcoakes 4y ago
  Thank you for mentioning GPT partition tables. The whole time I was reading this thread I kept feeling like there was something I was forgetting. I even just nuked one of my linux tinkering laptops this weekend and reinstalled.
dr_dshiv 4y ago
“GPT, like Geppetto” Geppetto carved a puppet of wood, And brought it to life as best he could. Geppetto's dreams, so long ago, Still echo in our quest to know, Can we create a being so smart, That it rivals the workings of the heart?

by breck 4y ago

anxrn 4y ago

Yes, this paper [1] introduced the term. https://scholar.google.com/citations?view_op=view_citation&hl=en&user=dOad5HoAAAAJ&citation_for_view=dOad5HoAAAAJ:W7OEmFMy1HYC https://scholar.google.com/citations?view_op=view_citation&h...

runnerup 4y ago
Weird. The term “Generative pretraining” seems to also exist in this 2016 proceeding from 2 years prior: https://ui.adsabs.harvard.edu/abs/2016arXiv160202644D/abstract https://ui.adsabs.harvard.edu/abs/2016arXiv160202644D/abstra... Google Trends shows it first got searched for in measurable quantity back in 2008.
- reaperman 4y ago
  Here's "generative pretaining" in an IEEE publication from 2012. https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf Geoffrey Hinton wrote it. I wonder if he'll have anything to say about OpenAI trademarking the term 11 years after he wrote this. Here's an earlier publication from January, 2012 talking about Hinton's 2011 "Generative pretraining" work: https://www.microsoft.com/en-us/research/publication/three-classes-of-deep-learning-architectures-and-their-applications-a-tutorial-survey/ https://www.microsoft.com/en-us/research/publication/three-c...
  - hn_throwaway_99 4y ago
    > Geoffrey Hinton wrote it. I wonder if he'll have anything to say about OpenAI trademarking the term 11 years after he wrote this. But they're not applying to trademark "generative pretraining". They're only applying to trademark GPT, which seems (I have very limited knowledge if there are other competing uses of the term) reasonable given that most people associate "GPT" with specific AI implementations created by OpenAI. There were other "business machine" companies that existed 100 years ago, but that is very different from trademarking "IBM".

mhink 4y ago

Interestingly, the record for OpenAI's application (for "GPT") has a note under "Goods and Services" that reads: "FIRST USE: 20181000. FIRST USE IN COMMERCE: 20181000" [1] The paper "Improving Language Understanding by Generative Pre-Training" appears to have been released on June 11, 2018. [2] 1: It looks like I can't link directly to it, but it should be pretty easy to find by searching for "GPT" at the parent's link 2: https://openai.com/research/language-unsupervised https://openai.com/research/language-unsupervised

gpm 4y ago

I was curious and discovered that Google has a (non-AI, advertising related) product called GPT that has existed for ages, here's a reference from 2015 https://github.com/davidecantoni/googletag https://github.com/davidecantoni/googletag I'm not sure if the industry is similar enough that this will be a problem for OpenAI (I suspect the fact that people are using GPT to refer to generative pre-trained transformer's in general is more of an issue for this application). GPT partition tables also come to mind.

hgsgm 4y ago
There are many many overlapping trademarks on initialisms.
hn_throwaway_99 4y ago
Good find. As the other commenter noted, however, trademarks are limited to the narrow field to which they apply. So I'm assuming OpenAI will be successful in getting a trademark for GPT in the field of generative AI, but it will be limited to that scope.
gcoakes 4y ago
Thank you for mentioning GPT partition tables. The whole time I was reading this thread I kept feeling like there was something I was forgetting. I even just nuked one of my linux tinkering laptops this weekend and reinstalled.

dr_dshiv 4y ago

“GPT, like Geppetto” Geppetto carved a puppet of wood, And brought it to life as best he could. Geppetto's dreams, so long ago, Still echo in our quest to know, Can we create a being so smart, That it rivals the workings of the heart?