2 ms·
I believe the naming is perfect: the word attention clearly maches the intention of the model structure. It’s still clearly missing more efficiency, but we had
by xiphias2 3y ago
I believe the naming is perfect: the word attention clearly maches the intention of the model structure. It’s still clearly missing more efficiency, but we had to have ChatGPT / ChatGPT4 working and used to make the further research directions clear (increasing context length and decreasing hallucinations).
- pico_creator 3y agoHaha, yea - naming things is hard Every-time someone comes up and say "this is not attention, because it does X and not Y" My response is, ok, what would you call it then? Because no one (including me) seem to to able to find a better term, that fits its use case.