10 ms·
likely https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_maps https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_... although the quoted sen
by evertedsphere 3y ago
likely https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_maps https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_...
although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch
- moralestapia 3y agoYou're right, that's a bit ambiguous. Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them. The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...
- evertedsphere 3y ago"if you look at the (chart) transition maps for different atlases on this manifold, they tend to look similar for different atlases" what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?
- moralestapia 3y agoSee, https://transformer-circuits.pub/2023/may-update/index.html#feature-manifolds https://transformer-circuits.pub/2023/may-update/index.html#.... They also reference, https://distill.pub/2020/circuits/equivariance/ https://distill.pub/2020/circuits/equivariance/. I'd also add, https://distill.pub/2021/multimodal-neurons/ https://distill.pub/2021/multimodal-neurons/.
- codethief 3y agoYeah, that's the definition I was thinking of, too, but the sentence didn't make sense to me, either.