4 ms·

> If your models are influencing the generation of data that they are later retrained on, it gets harder and harder to even maintain model performance. Why don

• ArekDymalski 3y ago
  > Why don't humans suffer from this problem, then? Can we be sure that we don't? For example, depending on perspective our language is either evolving towards clarity or devolving towards primitivity, compared to XVIII/XIX century. The same could be argued about other aspects of society and culture.
• dragonwriter 3y ago
  > Why don't humans suffer from this problem, then? They do, but the problem is poorly stated. The problen isn't “If your models are influencing the generation of data that they are later retrained on”, its “If output produced by your models with inadequate filtering for quality/accuracy is dominating the data that they are later retrained on.” Humans definitely experience the same problem: we see it when societies become closed, inward-looking, and locked into models of information filtering that don't involve accuracy or fitness for purpose other than confirming an in-groups priors. There are some examoles in stagnation of large societies, but probably the clearest examples are cults that socially isolate their membership and degrade into increasingly bizarre behaviors terminating in mass suicide, murder/suicides, etc. LLM’s experience self-reinforcing degradation more acutely because they are individually less capable than humans and they are less diverse.
  • js8 3y ago
    But humans can have a discussion about what is natural and what is not, and decide to mitigate it. The whole argument is based on the idea that the AI (either collectively or individually) can't even understand that, so the examples of cults etc. just do not apply. It's really the paperclip maximizer argument again. "Superintelligent AI" will supposedly change the world, but will lack enough common sense to understand a simple instruction with ethical constraint. Same here, the crappy AI content will somehow take over the world, despite the fact that most humans can actually mostly recognize original content from a generated one. The art that humans created doesn't replace nature, just like tons of kitsch don't pose a problem for new artists. In the same way, crappy AI content is not gonna replace good content, just augment it, and any decent AI (which has good enough common sense to be called "intelligent") will be able to tell the difference. Nobody's gonna care about crappy content in a few years, we are all gonna just throw it away, and this process happens continuously.
    • olddustytrail 3y ago
      > But humans can have a discussion about what is natural and what is not, and decide to mitigate it. In theory. In reality what you get is the Climate Change "debate".
      • Jensson 3y ago
        And it works, people are more and more convinced that we should do something about climate change. Many measures have already been put in place. You can lament that it isn't happening fast enough for your liking, but you can't say it isn't happening. Humanity do change their mind about things all the time, just that it takes decades instead of years or months.
        • olddustytrail 3y ago
          That's a fair point. I suppose my complaint is how slow the progress.
    • dragonwriter 3y ago
      > But humans can have a discussion about what is natural and what is not, and decide to mitigate it. (1) accuracy or fit for purpose, not natural, is the issue. (2) evaluation, not having a discussion, is the issue. (3) LLMs can definitely do evaluation, and probably can also have discussions and decide to, if given a proper prompt and access to ground truth in the first case, and if also used in a properly constructed agent framework, for the second. (4) if you did all that, and used LLM decisions and not human decisions to guide retraining, I suspect with current LLMs you’d actually have made the problem worse not better, because you’d reduce the influence of higher-capacity and more diverse humans on the training cycle and increase the influence of less diverse, lower capacity LLMs.
• raincole 3y ago
  > we have little problem (the exception are possibly things like crop circles) to recognize what is a natural phenomenon and what is generated culturally. How much of your taste to the opposite sex's (if you're straight) physical appearance is cultural and how much is in your genes?
• ben_w 3y ago
  > and yet we have little problem (the exception are possibly things like crop circles) to recognize what is a natural phenomenon and what is generated culturally. We have such a huge problem with this, that (with apologies) I cant help but think that you also don't know. If I look out of the window, what do I see that's natural? There's less nature in my view than most, because I'm in a city: ignoring the concrete, glass, steel, and tarmac, the traffic and the clothing, I see… humans, a tree and some mosses and grasses, the clouds above. Can humans even be said to be in a natural state or not, in contexts like this? I don't know. We domesticated ourselves, and it was our nature to do so, and also our nature to make the tools and clothing that led to our loss of body fur and other divergences from the other primates. But what I can say is that the tree was planted (if it was cultivated or selectively bred to this state, I wouldn't know); and a third of the CO2 in the air I breathe is from human actions, influencing the atmosphere and precisely when and which clouds I see (no contrails visible today, which would be more explicitly artificial). If I look a little further afar, I find entire hills in this city made by piling up rubble from the second world war and then covering it in topsoil, the only indication of which is the large signpost present to tell all of this[0]; and there are other hills both here and elsewhere that are made from trying to turn landfill sites into something less unpleasant to be around, with varying degrees of effectiveness in their disguises. If I think back to my childhood bedroom in a suburban home: there was a lawn (placed there by humans, then kept short with an unnatural Flymo) with two apple trees (cultivated and planted by humans), a vegetable patch and a herb garden (each plant likewise cultivated and placed by humans), surrounded by wooden fences (cut and placed). In the distance there was a row of trees, which might have been self-seeded or planted (I wouldn't know), enshrouding a small victorian folly covered in vines, and separating us from a series of fields (unnatural) where horses (selectively bred) were being stabled (unnatural structures); far beyond them was the ruin of an ancient tower destroyed centuries ago[1] — clearly built, but ask yourself: while stones are natural, are those specific larger stones on the corners, naturally like that? In a more abstract sense, if I look at foods in the supermarket, some will say "made from natural ingredients": if that thing is meat, such a claim ignores the selective breeding of the animal (and the conditions they were raised in, which would be a separate sticker saying "free range", though even then that's not like being wild). And even then, if it's made from ingredients plural, that's not natural either: bread doesn't grow in wheat fields, sushi rolls don't grow in rice paddies. Even if is a single ingredient, there's often processing involved: wheat (already selectively bred) has to be sorted from chaff, then ground to become flour. Even "mineral water" probably has had something done to it, even assuming there's not some small fine-print on the label saying "from a municipal source" (or whatever the clause is that means "actually just tap water"). [0] https://en.wikipedia.org/wiki/Fritz_Schlo%C3%9F_Park https://en.wikipedia.org/wiki/Fritz_Schlo%C3%9F_Park [1] https://en.wikipedia.org/wiki/Warblington_Castle https://en.wikipedia.org/wiki/Warblington_Castle