4 ms·
>Even if they did – so what? Amplification of biases, propagation of errors, echolalia and over-optimization, lack of diverse data, overfitting
by Jimmc414 4y ago
>Even if they did – so what?
Amplification of biases, propagation of errors, echolalia and over-optimization, lack of diverse data, overfitting
- RosanaAnaDana 4y agoI mean maybe. There also might be something to this. OpenAI has been very opaque about training techniques.
- paxys 4y agoThat's just the base concern with every single model regardless of where they sourced their data from. Garbage in, garbage out.
- educaysean 4y agoSure. Does that fact mean we're prohibited from expressing concerns about data quality? ShareGPT isn't representative of authentic, quality writing.
- Jimmc414 4y agoRight, but training an LLM on the output of another LLM can certainly exacerbate these issues
- paxys 4y agoMaybe, but we are fast approaching the point (or more likely have crossed it already) where distinguishing between human and AI generated data isn't really possible. If Google indexes a blog, how does it know whether it was written with AI assistance and therefore should not be used for training? Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that of other LLMs)?
- abduhl 4y agoYour argument would have a lot more force if we were past that point rather than fast approaching that point. Concerns about training data errors being compounded are much more important when you're talking about the bleeding edge. And your question about how OpenAI prevents their training data from being corrupted is one we should be asking as well!
- madeofpalk 4y ago> If Google indexes a blog, how does it know whether it was written with AI assistance and therefore should not be used for training Yes, this is an existential problem for Google and training future LLMs. See also, https://www.theverge.com/23642073/best-printer-2023-brother-laser-wi-fi-its-fine https://www.theverge.com/23642073/best-printer-2023-brother-... and https://searchengineland.com/verge-best-printer-2023-394709 https://searchengineland.com/verge-best-printer-2023-394709
- ethbr0 4y agoOr Google can just materialize the expected page into existence at search time. ... it's uncanny how it always finds what you thought you were looking for!
- rightbyte 4y ago> Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that or other LLMs)? Seems trivial. Only use old data for the bulk? Feed some new data carefully curated?
- toxik 4y agoFuture job: token selector / archiving
- notahacker 4y ago<meta name="generator" content="human brain"> I'm only half joking.... I think we likely will end up with flags for human generated/curated content (and it will have to be that way round, as I can't imagine spammers bothering to put flags on AI-generated stuff), and we probably already should have an equivalent of robots.txt protocol that allows users to specify which parts of their website they would and wouldn't like used in the training of LLMs.
- shubhamkrm 4y agoReminds me of the old “evil bit” RFC[1] [1] https://www.ietf.org/rfc/rfc3514.txt https://www.ietf.org/rfc/rfc3514.txt
- jfk13 4y agoIf content with a "human-generated" flag is rated more highly in some way -- e.g. search results -- then of course spammers will automatically add that flag to their AI-generated garbage. How do you propose to prevent them?
- notahacker 4y agoI assume, like the actual meta generator tags, it wouldn't actually be a massive boon for regular search results
- groestl 4y agoAnd if it's not: why bother.
- chatmasta 4y agoI think something like this will definitely happen, and your suggestion is the cleanest implementation idea I've seen for it. I imagine there will be a service provided by Google and OpenAI where they verify your identity as a human and then grant you a token to put into your meta tags (wait a second... this sounds like sama's worldcoin idea...). It will need to be based somewhat on the honor system (just because someone's proved they're a human doesn't mean they won't put their attestation on auto-generated text), but it definitely sounds better than nothing. They'll still need to incentivize it somehow, though. Why do I as a human want to add that meta tag? If the answer is "better search ranking" then it renders the whole scheme mostly pointless because obviously spammers will want to acquire the attestation and attach it to their auto-generated content.
- p1necone 4y agoPre-AI data is going to become like pre-nuclear steel.
- funkyjazz 4y agoNot to mention it's embarrassing. Google playing second banana to OpenAI.
- nicehill 4y agoI think Amazon was first in the (free) banana business
- jrirhfifj 4y agoyou joke, but first producy they changed on whole foods were the bananas. before: organic (south america) and regular (central ou SEA) for 69, 59. then: both chikita's brand with regular and organic stickers (clearly the same produce, always from SEA) for 49 and 39 cents. thats was days after the announcement
- b112 4y agoDid you inadvertently reverse to regular/organic order, or was organic cheaper after?
- ithkuil 4y agoThat assumes that training on the output of another language model somehow gives you the ability to improve your model and to catch up somehow
- iandanforth 4y agoIt does. In general this is known as teacher-student training or knowledge distillation. It works better if you have access to the activations of the model but you can work with just outputs as well.
- ithkuil 4y agoDoes it work even if you don't control the inputs?
- jrirhfifj 4y agoyou talk like chatgpt was some bastion of curated perfectly correct content. get a grip. web scraping is web scraping.