2 ms·
How about weighting each input training data item using some sort of "accuracy" metric? Rather than treating data from the onion and reddit the same as Wikipedi
by danjl 2y ago
How about weighting each input training data item using some sort of "accuracy" metric? Rather than treating data from the onion and reddit the same as Wikipedia, we could more accurately emulate what we do as humans. We spend an inordinate amount of time learning how to differentiate between different sources. Currently, LLMs treat all inputs identically and hence weight only by quantity.