Does it take less compute to train N agents vs one large model? Seem like a big win. Can the majority of the training be done independently or in distributed fa

Does it take less compute to train N agents vs one large model? Seem like a big win. Can the majority of the training be done independently or in distributed fashion?