thats not surprising to me: if we view the weights in the whole network as "individual cells" in a population, and if we pretend that at each update of network

thats not surprising to me: if we view the weights in the whole network as "individual cells" in a population, and if we pretend that at each update of network weights, before the update each weight undergoes cell division such that one daughter cell / weight is an increase in weight, and the other a decrease, then each component of the gradient descent vector can be viewed as the fitness function for that specific cell or weight: an increase or a decrease. From this perspective each cell forms its own niche in the ecosystem, and its no surprise that replacing a cell with its ancestor is roughly compatible with the final network cells: the symbiosis goes both ways. The reason for Bengio demonstrating this on a complete layer is obviously to demonstrate that this is NOT due to redundantly routing information to the next layer (think holographically, for robustness). And using non-ancestor random weights illustrates that the ecosystem fitness suffers if redundant / holographic routing is prevented while also using non ancestral cells / weights...