5 ms·

See my comment here: https://news.ycombinator.com/item?id=34640699 https://news.ycombinator.com/item?id=34640699 tl;dr: So far, we might be overplaying AI. The

See my comment here: https://news.ycombinator.com/item?id=34640699 https://news.ycombinator.com/item?id=34640699 tl;dr: So far, we might be overplaying AI. The post above can be coupled with the "AI Kessler Effect" in that AI text/code will be basing itself on AI-generated training sets. This will lead to a quality drop without human supervision (same old jobs as search-engine result checking, except maybe more technical). It will also lead to result stagnation - if fewer devs are writing code, then everything is a rehash of old code. Of course, the argument is that devs are writing new code and automating the boring parts, which is somewhat true, but doesn't lead to a loss of jobs. Also, if anyone knows of companies that they believe to be leading in AI, or products than can generate whole apps, please share. P.S. I am yet to see AI image generation to build visual novels. That would be the mark of it being "good". I spend very little time in the space though, so again, if anyone knows of anything, please share. (and no, not the kids book about AI, by AI, that is both bad and hand-picked / callibrated out of 1000s of results).

CuriouslyC 4y ago
In the future researchers and big corps will have much more heavily annotated data sets that can provide models with things like aesthetics and accuracy scores, and low quality AI generations won't impact output much unless prompted for (e.g. "early 2020s style AI art image with deformed hands and phantom limbs")
- marginalia_nu 4y ago
  The language you use makes it seem like this is an inevitability. What makes you say that?
  - CuriouslyC 4y ago
    Because data sets are very heterogeneous, and a lot of work already goes into curating them to try and get the best model possible. Curation is limited though, because it reduces the data set size and the model is still equally weighting everything in the data set, just without the worst stuff. By annotating the data set, you avoid reducing data set size while simultaneously teaching the model the difference between good and bad content. Many models already use annotated data, but LLMs haven't leveraged it much thus far because they're still new-ish tech with architectures in flux, and they've been focused on improving general quality by scaling size/data. As LLMs become more commercially important and we run out of high quality training data, annotating existing data will become the cheapest route to improving output quality.

CuriouslyC 4y ago

In the future researchers and big corps will have much more heavily annotated data sets that can provide models with things like aesthetics and accuracy scores, and low quality AI generations won't impact output much unless prompted for (e.g. "early 2020s style AI art image with deformed hands and phantom limbs")

marginalia_nu 4y ago
The language you use makes it seem like this is an inevitability. What makes you say that?
- CuriouslyC 4y ago
  Because data sets are very heterogeneous, and a lot of work already goes into curating them to try and get the best model possible. Curation is limited though, because it reduces the data set size and the model is still equally weighting everything in the data set, just without the worst stuff. By annotating the data set, you avoid reducing data set size while simultaneously teaching the model the difference between good and bad content. Many models already use annotated data, but LLMs haven't leveraged it much thus far because they're still new-ish tech with architectures in flux, and they've been focused on improving general quality by scaling size/data. As LLMs become more commercially important and we run out of high quality training data, annotating existing data will become the cheapest route to improving output quality.