2 ms·
In the future researchers and big corps will have much more heavily annotated data sets that can provide models with things like aesthetics and accuracy scores,
by CuriouslyC 4y ago
In the future researchers and big corps will have much more heavily annotated data sets that can provide models with things like aesthetics and accuracy scores, and low quality AI generations won't impact output much unless prompted for (e.g. "early 2020s style AI art image with deformed hands and phantom limbs")
- marginalia_nu 4y agoThe language you use makes it seem like this is an inevitability. What makes you say that?
- CuriouslyC 4y agoBecause data sets are very heterogeneous, and a lot of work already goes into curating them to try and get the best model possible. Curation is limited though, because it reduces the data set size and the model is still equally weighting everything in the data set, just without the worst stuff. By annotating the data set, you avoid reducing data set size while simultaneously teaching the model the difference between good and bad content. Many models already use annotated data, but LLMs haven't leveraged it much thus far because they're still new-ish tech with architectures in flux, and they've been focused on improving general quality by scaling size/data. As LLMs become more commercially important and we run out of high quality training data, annotating existing data will become the cheapest route to improving output quality.