3 ms·
I guess I'm interested to see if this performs qualitatively better than GPT-3, given how many more parameters it has. However, I think this is really a dead-e
by bobm_kite9 5y ago
I guess I'm interested to see if this performs qualitatively better than GPT-3, given how many more parameters it has.
However, I think this is really a dead-end: throwing more hardware at this is just going to generate better-sounding nonsense. Yes, we are learning the "model" of the English language - which words go with which others, but successively larger transformer models don't really expose much more about the nature of intelligent conversation.
I think we need a better algorithm now.
- jamesbriggs 5y agoAlso agree with this, it's almost like a marketing ploy - especially from OpenAI. They produce awesome stuff but things with GPT can get silly sometimes, like when they wouldn't release the larger versions because 'they were too powerful', and in the end you ask it how many eyes my foot has and it says seven... It producing interesting results, but doesn't really progress the field - although who knows, maybe skynet is actually a 100T parameter transformer
- simonh 5y agoListening to conversations and learning how they flow is only one aspect of language learning. What these things are missing is the interactive part. Next generation systems need to be able to form hypotheses about what appropriate responses should be, try out various responses and then see what the results are. They can only learn so much from passive consumption of training sets, so I agree this approach is going to hit a wall of diminishing returns.