I don't know much about actually merging networks. I knew a guy who trained multiple networks in parallel for his M.S. and then combined them by just averaging

I don't know much about actually merging networks. I knew a guy who trained multiple networks in parallel for his M.S. and then combined them by just averaging the corresponding weights. It seemed to work well. For this to work, the networks needed to have the same architecture (same number of layers, nodes in each layer and same connectivity between layers). Now if you just want to train multiple neural networks (or other classifiers) on different datasets (to have different strengths) then you can keep them separate and build a composite system that lets each network "vote" on an answer to a given problem; the decision of the overall system is a weighted some of the components. See [0]. [0] - https://en.wikipedia.org/wiki/Ensemble_learning https://en.wikipedia.org/wiki/Ensemble_learning