Training on chats with an LLM is considered useless in my circles (enthusiast level). The argument is that it already knows its answers, also user input is just

Training on chats with an LLM is considered useless in my circles (enthusiast level). The argument is that it already knows its answers, also user input is just too bad to train on, because it’s mostly clueless questions, corrections and rants, due to the nature of chats with LLMs. It just isn’t a discussion worth including into a dataset. Some people experimented with loras trained on such chats and reported degradation.