3 ms·
Since the whole GPT3 thing blown up, I'm thinking from time to time... how I am generating what I say. I'm sure many smart people wrote papers on that. I did no
by aflukasz 4y ago
Since the whole GPT3 thing blown up, I'm thinking from time to time... how I am generating what I say. I'm sure many smart people wrote papers on that. I did not read any of them, mind you, will just share a short thought of my own here, hopefully providing some intellectual entertainment for someone.
It seems from my point of view that, broadly speaking, I maintain four things at the moment of talking to someone:
1. A graph of concepts that were used / are potentially going to be used by me or by my interlocutor.
2. Some emotional state.
3. Some fuzzy picture of where I'm going with what I'm saying in the short term of say 20 seconds.
4. Extra short term focused process of making sure that the next 2-3 words fit to the one I just said and are going to fulfill requirements stemming from (3) and (1); this happens with some influence form (2), ideally not too much, if I consider current state of (2) not helping to be constructive.
GPT3 obviously lacks (2). My limited understanding of LLMs is that it does (4), maybe (3) and probably not (1) (?).
So I'm just wondering - are those LLMs really that far from a "human being"?
Again, not an expert. Happy to be corrected.