3 ms·
GPT3 was released 3 years ago now. There have been major advancements in scaling attention so it would be strange if they didn't use some of them
by space_fountain 3y ago
GPT3 was released 3 years ago now. There have been major advancements in scaling attention so it would be strange if they didn't use some of them
- MichaelZuo 3y agoIt doesn't matter how many major advancements they made in scaling, as long as one component is O(n^2) or higher.
- cubefox 3y agoIt's not the scale itself, it's the scaling architecture.
- MichaelZuo 3y agoThe same applies.