4 ms·
The author mentions that the shape of the curve from 117M parameters to 175B is interesting, but doesn't show it. Does anyone have the graph?
by ladberg 6y ago
The author mentions that the shape of the curve from 117M parameters to 175B is interesting, but doesn't show it. Does anyone have the graph?
- bubba1236 6y agoprobably this https://www.microsoft.com/en-us/research/uploads/prod/2020/02/TurningNGL_Model__1400x788-5e418cff76a2a-800x550.png https://www.microsoft.com/en-us/research/uploads/prod/2020/0...
- deleted 6y ago[deleted]
- Tarean 6y agoPresumably it's one of these graphs? https://i.imgur.com/r7xJfi1.jpg https://i.imgur.com/r7xJfi1.jpg https://lh6.googleusercontent.com/VHmmdKYio39kz017ECfPwCxGPT_l7FYl7KDvlXM-ruC5ttiAYJXz5hsfPiWkrxkVKCU5fL-24XRXtw7gEe9KwJXqWW1bLzukITyL1Xz-YFWg0BCP5-RrBXOAdDK2paV7HpT1F4AM https://lh6.googleusercontent.com/VHmmdKYio39kz017ECfPwCxGPT... There are a lot of graphs in the paper, though.
- yunusabd 6y agoI think it's this one [1], there are also more detailed graphs in section 3 of the paper. [1] https://imagehost.imageupload.net/2020/06/04/IMG_20200604_093458.jpg https://imagehost.imageupload.net/2020/06/04/IMG_20200604_09...
- nostalgebraist 6y agoI wasn't referring to one specific graph. The GPT-3 paper (https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf) has a very large number of graphs showing parameter count on the horizontal axis and some kind of prediction quality metric on the vertical axis. Most of the interesting ones are in Appendix H. "The point isn’t the performance at 175B, but the shape of the curve as it passes from 117M to 175B" was referring to a general point about how to interpret any/all of those graphs, not a particular one of them.