3 ms·

It is revolutionary for many. GPT2 was just another "looks cool" thing, GPT4 is just so jaw dropping that I am scared what GPT5 would look like.

famouswaffles 4y ago
Yeah lol. I didn't want to argue further because hey it's his/her opinion. But saying GPT-2 to 4 in 4 years isn't revolutionary just seems contrarian to me. Bespoke NLP of any kind is out the door with a single model. That's fucking crazy.
- timr 4y ago
  Yours is also just an opinion. But having a baseline understanding of what came before is important to having an informed opinion. To that end: GPT is not the first tech to combine "NLP" into a single model. For example, from 2019 [1]. This stuff is evolutionary. [1] https://pub.towardsai.net/unified-language-model-pre-training-for-natural-language-understanding-and-generation-f87dc226aa2 https://pub.towardsai.net/unified-language-model-pre-trainin...
  - famouswaffles 4y ago
    You linked a paper that came months after the release of GPT-2. It is also a modified transformer, the basis of most LLMs today and an architecture that was released only two years prior. and this is without mentioning that NLP is far from the only things LLMs do/excel at. Frankly, nothing you have said so far has made me think your opinion is more informed than mine.
    - timr 4y ago
      No. I linked to a blog article which mentions many different papers, many of which were from 2018 or before. ELMO, BERT, UNILM...all examples of multi-target NLP models that pre-dated GPT. I didn't dig this up because I thought the date on the webpage was some kind of rebuttal. I picked the first article I saw that mentioned models I knew about, with dates. > It is also a modified transformer Well, yes. Exactly. We're in the echo of that work, which was the core advancement that happened circa 2017. As I keep saying, it only seems revolutionary if you haven't been paying attention to the last 5 years.

famouswaffles 4y ago

Yeah lol. I didn't want to argue further because hey it's his/her opinion. But saying GPT-2 to 4 in 4 years isn't revolutionary just seems contrarian to me. Bespoke NLP of any kind is out the door with a single model. That's fucking crazy.

timr 4y ago
Yours is also just an opinion. But having a baseline understanding of what came before is important to having an informed opinion. To that end: GPT is not the first tech to combine "NLP" into a single model. For example, from 2019 [1]. This stuff is evolutionary. [1] https://pub.towardsai.net/unified-language-model-pre-training-for-natural-language-understanding-and-generation-f87dc226aa2 https://pub.towardsai.net/unified-language-model-pre-trainin...
- famouswaffles 4y ago
  You linked a paper that came months after the release of GPT-2. It is also a modified transformer, the basis of most LLMs today and an architecture that was released only two years prior. and this is without mentioning that NLP is far from the only things LLMs do/excel at. Frankly, nothing you have said so far has made me think your opinion is more informed than mine.
  - timr 4y ago
    No. I linked to a blog article which mentions many different papers, many of which were from 2018 or before. ELMO, BERT, UNILM...all examples of multi-target NLP models that pre-dated GPT. I didn't dig this up because I thought the date on the webpage was some kind of rebuttal. I picked the first article I saw that mentioned models I knew about, with dates. > It is also a modified transformer Well, yes. Exactly. We're in the echo of that work, which was the core advancement that happened circa 2017. As I keep saying, it only seems revolutionary if you haven't been paying attention to the last 5 years.