There is some research on layer duplication on existing pre-trained models. This paper also references layer skipping: https://aclanthology.org/2025.findings-em

There is some research on layer duplication on existing pre-trained models. This paper also references layer skipping: https://aclanthology.org/2025.findings-emnlp.967.pdf https://aclanthology.org/2025.findings-emnlp.967.pdf