I think it is less relevant to Winning Ticket Hypothesis (in my opinion, the most important paper on deep learning theory in the last 5 years), but about archit

I think it is less relevant to Winning Ticket Hypothesis (in my opinion, the most important paper on deep learning theory in the last 5 years), but about architectures that work to some extend with random weights. See "Weight Agnostic Neural Networks" (https://weightagnostic.github.io/ https://weightagnostic.github.io/): > Inspired by precocial species in biology, we set out to search for neural net architectures that can already (sort of) perform various tasks even when they use random weight values.