3 ms·

I suspect we're seeing Goodhart's Law in action here combined with a multiplier based on hardware gating, similar to what's happened with Stable Diffusion finet

I suspect we're seeing Goodhart's Law in action here combined with a multiplier based on hardware gating, similar to what's happened with Stable Diffusion finetunes. There are some fantastic 1.5 finetunes if you're seeking to gen a particular style/subject but these finetunes are horribly overfitted and fall flat when used outside of their specific niche (big tiddy anime girl most often). These models come into being because the barrier to entry is low and the model is focused on a narrow set of metrics. This is the reason we see smaller models "outperforming" larger ones. I have tried a lot of these LLMs locally and I have rarely ever found the smaller models to outperform the bigger models when used casually/generically. The same is generally true for 1.5 vs SDXL the moment the prompt strays away from some variant of beautiful woman.

22c 3y ago
I agree for the most part, but there is also the other side which is how useful it can be. If you fine tune your model on anime girls, it becomes a very useful model for generating anime girls and perhaps it's now worse at drawing stick figures or cave paintings, but a lot of people don't need the model to draw stick figures or cave paintings and if they did, they could get a model fine tuned to do that. On the LLM side, there's a similar phenomenon where a huge amount of training could be on SEO spam tags, or /r/counting or producing nonsense or generating obfuscated C or reciting excerpts of Shakespeare word for word, etc. If what you're measuring is generally more useful then you can end up with a better model using these methods, likely at the cost worse performance on things that you aren't measuring. I would gladly take a 10b parameter coding assistant that doesn't know how to write in iambic pentameter or recite digits of pi, or translate words from Swahili to Turkish etc. but is much better at code completion.
- thot_experiment 3y ago
  Totally valid points! However I think a lot of the real "magic" comes from the sort of cross-domain "thinking" that the larger models are capable of and I think that axis hard to benchmark. (though because it seems so hard to quantify, there's a high chance i'm making shit up)
  - 22c 3y ago
    Valid, but how much can you distill a model while retaining (or refining) usefulness? You don't need to know how to paint Rembrandt to draw an anime girl, and you don't need to know about the biological taxa of North American Ducks to write a for loop. The cross-over might come to a point where you might need to explain what "duck typing" is, but even then, you only need to know that a duck is something which quacks (What does "quack" mean? Who knows..) If a model forgets how to speak French but gets much better at generating unit tests, that might be perfectly fine for the type of work we want the model to perform. The problem is we can't easily know what the model "forgets" when it gets better at doing something else. The best thing we can do is benchmark/measure their output and hope that those benchmarks cover what users care about. I suspect high quality benchmarks will quickly become almost as important as the tuning process itself.

22c 3y ago

I agree for the most part, but there is also the other side which is how useful it can be. If you fine tune your model on anime girls, it becomes a very useful model for generating anime girls and perhaps it's now worse at drawing stick figures or cave paintings, but a lot of people don't need the model to draw stick figures or cave paintings and if they did, they could get a model fine tuned to do that. On the LLM side, there's a similar phenomenon where a huge amount of training could be on SEO spam tags, or /r/counting or producing nonsense or generating obfuscated C or reciting excerpts of Shakespeare word for word, etc. If what you're measuring is generally more useful then you can end up with a better model using these methods, likely at the cost worse performance on things that you aren't measuring. I would gladly take a 10b parameter coding assistant that doesn't know how to write in iambic pentameter or recite digits of pi, or translate words from Swahili to Turkish etc. but is much better at code completion.

thot_experiment 3y ago
Totally valid points! However I think a lot of the real "magic" comes from the sort of cross-domain "thinking" that the larger models are capable of and I think that axis hard to benchmark. (though because it seems so hard to quantify, there's a high chance i'm making shit up)
- 22c 3y ago
  Valid, but how much can you distill a model while retaining (or refining) usefulness? You don't need to know how to paint Rembrandt to draw an anime girl, and you don't need to know about the biological taxa of North American Ducks to write a for loop. The cross-over might come to a point where you might need to explain what "duck typing" is, but even then, you only need to know that a duck is something which quacks (What does "quack" mean? Who knows..) If a model forgets how to speak French but gets much better at generating unit tests, that might be perfectly fine for the type of work we want the model to perform. The problem is we can't easily know what the model "forgets" when it gets better at doing something else. The best thing we can do is benchmark/measure their output and hope that those benchmarks cover what users care about. I suspect high quality benchmarks will quickly become almost as important as the tuning process itself.