3 ms·

Even developers at Open AI played almost no part in the developments at Open AI. 99.9999% of the work was done by those who created the content it was trained o

• HarHarVeryFunny 4y ago
  If that was true we could have had GPT-3/etc years ago. It's a bit like saying that college graduates are dumb because after all what have they learnt but a bunch of knowledge in text books. The success of these LLMs comes down to the Transformer architecture which was a bit of an accidental discovery - designed for sequence-to-sequence (e.g. machine translation) NLP use by a group of Google researchers (almost all of who have since left and started their own companies). The "Attention is all you need" Transformer seq-2-seq paper, while very significant, was an evolution of other seq-2-seq approaches such as Ilya Sutskever's "Sequence to Sequence Learning with Neural Networks". Sutskever is of course one of the OpenAI co-founders and chief scientist. He was also one of Geoff Hinton's students who worked on the AlexNet DNN that won the 2012 ImageNet competition, really kicking off the modern DNN revolution.