Yes you can! The "mix" part is key. It's the sequential learning which screws up networks today. If you randomly sample from tasks you're fine, or if you can re

Yes you can! The "mix" part is key. It's the sequential learning which screws up networks today. If you randomly sample from tasks you're fine, or if you can replay older tasks while you're learning new ones (essentially another form of random sampling) the network can learn multiple tasks. But the moment you drop a task from a distribution of training data you're going to start losing competency on it. By default neural networks don't have mechanisms to protect data (weights) from being over-written.