Training from scratch could presumably mean including the new design attempts and old designs mixed in. So no contradiction: pretrain on old designs then finet

Training from scratch could presumably mean including the new design attempts and old designs mixed in. So no contradiction: pretrain on old designs then finetune on new design, vs train on everything mixed together throughout. Finetuning can cause catastrophic forgetting. Both could have better performance than not including old designs.