The point still stands, Phi3 is an excellent model and shows that good models don’t need that many parameters You should see the work on ReFT coming from manni

The point still stands, Phi3 is an excellent model and shows that good models don’t need that many parameters You should see the work on ReFT coming from mannings group showing that you can instruction fine tune models by modifying like, 0.00001% of the parameters. By doing it this way, you significantly mitigate the risk of catastrophic forgetting.