I'm really excited about the dream that we'll be able to learn from neural networks. Shan Cater and Michael Nielsen wrote a really inspiring article on this (ht

I'm really excited about the dream that we'll be able to learn from neural networks. Shan Cater and Michael Nielsen wrote a really inspiring article on this (https://distill.pub/2017/aia/ https://distill.pub/2017/aia/). I also wrote something about this a while back (http://colah.github.io/posts/2015-01-Visualizing-Representations/ http://colah.github.io/posts/2015-01-Visualizing-Representat...). One of the amazing things about this project exploring CLIP was seeing some hints of this. For example, one day I was studying one of the Africa neurons and it generated the text "IMBEWU" -- it turns out this is a popular TV show in South Africa (https://en.wikipedia.org/wiki/Imbewu:_The_Seed https://en.wikipedia.org/wiki/Imbewu:_The_Seed). That's a trivial example, but it begins to hint at something interesting. I'd really love to see what a domain expert analyzing CLIP would make of things. For example, I'd love to hear what ethnographers think of the region neurons, or what historians think of the time period neurons. Especially for future, larger models.