3 ms·

You should add what version of the model you are testing For example you mention Jon Durbin Airoboros L2 70B But is it 1.4? 2.0? 2.1? Etc.