Also you can still compare neurons to see if any of them are very similar. Another thing that might work would be to prune away many of the connections. Set all

Also you can still compare neurons to see if any of them are very similar. Another thing that might work would be to prune away many of the connections. Set all of the connections that are very small to zero, or encourage it to set weights to zero with something like weight decay. As a bonus this might even improve generalization.