I am curious, if folks have tried hybrid methods using ensembles. Train the main model using FP8 (or other quantized approaches) and have a small calibrating m

I am curious, if folks have tried hybrid methods using ensembles. Train the main model using FP8 (or other quantized approaches) and have a small calibrating model at FP32 that is trained afterward.