3 ms·
It's not so strange if you visualize the actual weights of a model. Notice that most of the visualizations on https://battle.shawwn.com/sdb/visualizations/2020-
by sillysaurusx 6y ago
It's not so strange if you visualize the actual weights of a model. Notice that most of the visualizations on https://battle.shawwn.com/sdb/visualizations/2020-09-16-117m-gpt-revnet/index.html https://battle.shawwn.com/sdb/visualizations/2020-09-16-117m... are solid colors.
- yorwba 6y agoIf you zoom in until you can distinguish individual pixels, it's clear that they aren't actually solid colors. It's just that all the variability is at the pixel level and there are no large-scale structures. (There couldn't possibly be, because the ordering of rows and columns is arbitrary: you could shuffle them around and get an equivalent model. The most you could expect is some banding where values in the same row or column are related.)