You would have to fundamentally change how the models are structured/trained and how the data sets are presented to the training process in order to make this w

You would have to fundamentally change how the models are structured/trained and how the data sets are presented to the training process in order to make this work and it would dramatically limit the model's generality as a result.